There was a great paper around compiled AI (use LLM as compiler, not as runtime) that I really liked, but it's really hard to explain the benefit to people. This open source project should change that, giving everyone a chance to play around with it locally. You can create your own compiled ai plans, run them deterministically, and it also allows you to connect it to your agents so that if they have something where they can use less tokens or be more deterministic in the future they will.
We choose Rust btw bc it works so well for both agents and humans. The compiler alone is super helpful, it's like getting a development harness for free that you can just extend.
It's still more of a demo what it can do, so expect failures and quirks, but looking forward to feedback and how it's working for you
Glad the “LLM as compiler, not runtime” idea is landing! The goal is to make “compiled AI” available as something you can actually play with and use locally instead of just reading about in papers. It´s still a demo but looking forward to your feedback and what kind of work you are going to compile
Super happy to see this. Feels like there's a strange gap in infrastructure projects for AI.
Not everyone has infinite money to throw at tokens!
There was a great paper around compiled AI (use LLM as compiler, not as runtime) that I really liked, but it's really hard to explain the benefit to people. This open source project should change that, giving everyone a chance to play around with it locally. You can create your own compiled ai plans, run them deterministically, and it also allows you to connect it to your agents so that if they have something where they can use less tokens or be more deterministic in the future they will.
We choose Rust btw bc it works so well for both agents and humans. The compiler alone is super helpful, it's like getting a development harness for free that you can just extend.
It's still more of a demo what it can do, so expect failures and quirks, but looking forward to feedback and how it's working for you
Agree so much compiling the plan once into something deterministic and inspectable flips the economics and the reliability profile.
Glad the “LLM as compiler, not runtime” idea is landing! The goal is to make “compiled AI” available as something you can actually play with and use locally instead of just reading about in papers. It´s still a demo but looking forward to your feedback and what kind of work you are going to compile