If your goal is to get the network as small as possible, you could try using NEAT[1]. A cool example of a small network for a hard task using NEAT is MarI/O[2].
Neat! Maybe consider doing a longer postmortem/blog post on this.
Small bit of feedback: it might help to add an explicit small (i) icon for areas which contain tooltips especially for interfaces (phones) that don't support hover events. I discovered that the models (such as pareto-r1) had additional information by complete accident~
I definitely will be - my experimentation outpaced my understanding a bit, so I’m slowing down to understand and verify my exact claims.
One of the most fun projects I have ever worked on and I’m excited to share the discoveries.
What kind of postmortem would you like to see? A blog or a hackernews post? And what interests you the most? Is it the compression, the visuals, the neural networks, the discoveries, or something else?
I will update the frontend based on the icon feedback for sure :) thank you!
- 14 Bytes does not include the runtime!
- All that counts towards the 14 Bytes is the model weights
- Lack of git history is because my actual repo is private, this is just the frontend for now + models
Thank you I will update the frontend with a legend and some better alignment understanding with the visuals.
As a quick note, they are mostly immediate environment observations like “is there a wall in front of me, to my left, to my right” etc or “was my last move illegal”.
But every model is different, some have very rich input information like “there is a wall diagonal to my front right” and some only have very coarse inputs.
Because I have trained over 2000 models now I need to go back and get an LLM to label the inputs for me, definitely not a manual task :)
If your goal is to get the network as small as possible, you could try using NEAT[1]. A cool example of a small network for a hard task using NEAT is MarI/O[2].
[1]: https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...
[2]: https://www.youtube.com/watch?v=qv6UVOQ0F44
I will be reading this one asap, my goal is as small as possible ! So far 14 bytes has been hard for me to beat
Neat! Maybe consider doing a longer postmortem/blog post on this.
Small bit of feedback: it might help to add an explicit small (i) icon for areas which contain tooltips especially for interfaces (phones) that don't support hover events. I discovered that the models (such as pareto-r1) had additional information by complete accident~
Thank you for your feedback!
I definitely will be - my experimentation outpaced my understanding a bit, so I’m slowing down to understand and verify my exact claims.
One of the most fun projects I have ever worked on and I’m excited to share the discoveries.
What kind of postmortem would you like to see? A blog or a hackernews post? And what interests you the most? Is it the compression, the visuals, the neural networks, the discoveries, or something else?
I will update the frontend based on the icon feedback for sure :) thank you!
Thanks! This is inspires me to learn about AI / NN. Can you share some information on your tech stack?
Facts:
- 14 Bytes does not include the runtime! - All that counts towards the 14 Bytes is the model weights - Lack of git history is because my actual repo is private, this is just the frontend for now + models
I would love to understand what the inputs actually are. Very neat visualization.
Thank you I will update the frontend with a legend and some better alignment understanding with the visuals.
As a quick note, they are mostly immediate environment observations like “is there a wall in front of me, to my left, to my right” etc or “was my last move illegal”.
But every model is different, some have very rich input information like “there is a wall diagonal to my front right” and some only have very coarse inputs.
Because I have trained over 2000 models now I need to go back and get an LLM to label the inputs for me, definitely not a manual task :)
[dead]