Hey, so I have been tinkering lately with locally running LLMs for programming purposes, on a Strix Halo machine with 128GB of RAM. I quickly realised that in order to get more speed out of the setup I need to save on stuff that ends up in model context. I built (with agents!) this simple bash command runner that swallows large output, passing only the relevant stuff to agent.
Think pytest only outputting OK if all tests pass, but in case of error getting the necessary error output only.
It is a cli, basic use for agents:
oo pytest
oo cargo test
etc…
Custom patterns configurable and there is also learn -flag to use Anthropic models to have LLM write the configuration for your custom command.
Nothing too complicated or special, but works in my LLM workflow saving on token use & speeding models!
Hey, so I have been tinkering lately with locally running LLMs for programming purposes, on a Strix Halo machine with 128GB of RAM. I quickly realised that in order to get more speed out of the setup I need to save on stuff that ends up in model context. I built (with agents!) this simple bash command runner that swallows large output, passing only the relevant stuff to agent.
Think pytest only outputting OK if all tests pass, but in case of error getting the necessary error output only.
It is a cli, basic use for agents:
oo pytest oo cargo test etc…
Custom patterns configurable and there is also learn -flag to use Anthropic models to have LLM write the configuration for your custom command.
Nothing too complicated or special, but works in my LLM workflow saving on token use & speeding models!
Have a great weekend!