Hello! I just released wyat-2, a fine-tune of Ouro-1.4B (a looped transformer) trained on a single RTX 4070 Ti.
The link is the report. As far as I know it’s the first decision model built on a looped transformer: not the best, but the first.
I realized I would need heavy data work to make it the best, so I’d prefer not to spend too much time on it for now. Maybe later, if it catches interest.
Hello! I just released wyat-2, a fine-tune of Ouro-1.4B (a looped transformer) trained on a single RTX 4070 Ti.
The link is the report. As far as I know it’s the first decision model built on a looped transformer: not the best, but the first.
I realized I would need heavy data work to make it the best, so I’d prefer not to spend too much time on it for now. Maybe later, if it catches interest.
Model: https://huggingface.co/wyat-io/wyat-2-1.4b