Kudos, great start. Ambitious effort, for it to be successful you could zero-in on a specific application and make that better than pytorch for training/inference.
Many users of pytorch are scientists and folks only peripherally interested in software. For them, python scripting is great because of its low barrier to entry. The torch ecosystem is large, and consists of folks like that. They are unlikely to learn things like rust. So, instead of wholesale reimplementation of pytorch, a better bet would be to replace parts of pytorch that are slow and bloated. Those parts would be invisible to end-users, and would be a great candidate for a rust refactor.
I think if I wanted to turn this into a "real" project that picking a direction would make sense. For the moment I am just trying to learn as much as I can about how something like PyTorch works.
Any new rust based ML library is great in my book. I've taken a brief look at the candle tensor implementation and just took an even briefer look at the can-t tensor implementation. What was the reasoning behind a global equation to store all tensors grads etc ? My initial assumption would be that it might make more sense to store that data scoped within a `Graph` type or similar.
Basically the idea, and I need to write up something on this, is that as much as possible client usage of the library should feel like you are writing PyTorch. For better of worse, people are used to that lib, and its patterns. So if anyone wanted to bring code over, it is easier if can basically copy and paste it over.
Kudos, great start. Ambitious effort, for it to be successful you could zero-in on a specific application and make that better than pytorch for training/inference.
Many users of pytorch are scientists and folks only peripherally interested in software. For them, python scripting is great because of its low barrier to entry. The torch ecosystem is large, and consists of folks like that. They are unlikely to learn things like rust. So, instead of wholesale reimplementation of pytorch, a better bet would be to replace parts of pytorch that are slow and bloated. Those parts would be invisible to end-users, and would be a great candidate for a rust refactor.
I think if I wanted to turn this into a "real" project that picking a direction would make sense. For the moment I am just trying to learn as much as I can about how something like PyTorch works.
Any new rust based ML library is great in my book. I've taken a brief look at the candle tensor implementation and just took an even briefer look at the can-t tensor implementation. What was the reasoning behind a global equation to store all tensors grads etc ? My initial assumption would be that it might make more sense to store that data scoped within a `Graph` type or similar.
Basically the idea, and I need to write up something on this, is that as much as possible client usage of the library should feel like you are writing PyTorch. For better of worse, people are used to that lib, and its patterns. So if anyone wanted to bring code over, it is easier if can basically copy and paste it over.
Oh that's pretty elegant. Good job.