I wonder if decision models can help bridge the gap between prose, and schema or a lint rule.
The problem with strong typing is sometimes you enforce a rule that is too rigid.
A good example is with UTC: suppose you use strong typing to enforce across ALL datetime fields that they must be UTC.
Then you need to store a birthday. A birthday stored as a UTC timestamp will show up a day early for users in another timezone.
With Agents people are starting to enforce a lot more prose style requirements in their AGENTS.md or CLAUDE.md files, which have the advantage of being nuanced and flexible.
The problem is these natural-language rules are prose, often ignored by agents, or arbitrarily applied.
Perhaps a natural language rules-engine that runs everything through a decision model asking "Does this meet this criteria" has the strength of both.
The validator could be enforced by a script to run on every commit, and it could instead enforce "All date fields should be stored in UTC, except for instants or local dates."
It could even enforce things like "we never store PII in any field not in our secure-store," and other high-level criteria.
Actually, we use our own product—Moonbounce.ai—to do exactly this and it's been part of our coding workflow for some time. The first way we did this was with cursor hooks that ran checks after each turn completed so that it wasn't up to the agent whether to do the check or not. But there are plenty of ways to implement the check itself.
We've found lots of "rules" that can't be enforced well (or get over-enforced by static checks) that this approach is great for. And it works far better than just jamming a prompt into `AGENTS.md` or another markdown rules file as those tend to just get ignored as context grows.
One example from my usage is dependency injection. Our coding agents were constantly taking the easy route and using singletons (they are evil, I don't need to restate why). As a general principle, we want to use dependency injection and so we set up a check to enforce this. Essentially, it meant I didn't have to read the code, find the violation of our principle, and then go back to the agent and say "hey, I asked for dependency injection but you did X." Saved a lot of time.
While our platform uses a continuous learning loop to "teach" the decision model about your judgment over time, I'd expect that you could do this with any decision model if you can figure out the prompts and state that you need.
TL;DR—We've made it work. Saves us money on LLM code reviews. It's great. Highly recommend. 10/10.
Absolutely - the most valuable and expensive datasets to produce are catastrophic real-world failures. We need to encode _at least_ those direct patterns. We also need to extrapolate, simulate, and reinterpret those precious datapoints as the surrounding landscape changes. To the exent we fail to do so, we will be augment the world's catastrophic failure dataset with heartbreaking new datapoints. Victims and their families, such as the plaintiffs in the Boeing Max lawsuits, will want to see those datapoints encapsulated into future processes.
In the past some of these issues were discovered in real life - in blood. I wonder what role some kind of simulation can have in predicting and running scenarios to unlock opportunities low effort high risk. We have a lot of computational power to run this as well as make predictions of what can fail and how bad it could be.
This is super relevant to cyber. Historically we've written rule after rule following every incident but that makes it close to impossible to catch things you haven't seen before. Nice write up!
If this is how society learns lessons (and I can see why this has made sense until now) then we're waiting for a paperclip problem before we invest in a kill switch that works.
I wonder if decision models can help bridge the gap between prose, and schema or a lint rule.
The problem with strong typing is sometimes you enforce a rule that is too rigid.
A good example is with UTC: suppose you use strong typing to enforce across ALL datetime fields that they must be UTC.
Then you need to store a birthday. A birthday stored as a UTC timestamp will show up a day early for users in another timezone.
With Agents people are starting to enforce a lot more prose style requirements in their AGENTS.md or CLAUDE.md files, which have the advantage of being nuanced and flexible.
The problem is these natural-language rules are prose, often ignored by agents, or arbitrarily applied.
Perhaps a natural language rules-engine that runs everything through a decision model asking "Does this meet this criteria" has the strength of both.
The validator could be enforced by a script to run on every commit, and it could instead enforce "All date fields should be stored in UTC, except for instants or local dates."
It could even enforce things like "we never store PII in any field not in our secure-store," and other high-level criteria.
Actually, we use our own product—Moonbounce.ai—to do exactly this and it's been part of our coding workflow for some time. The first way we did this was with cursor hooks that ran checks after each turn completed so that it wasn't up to the agent whether to do the check or not. But there are plenty of ways to implement the check itself.
We've found lots of "rules" that can't be enforced well (or get over-enforced by static checks) that this approach is great for. And it works far better than just jamming a prompt into `AGENTS.md` or another markdown rules file as those tend to just get ignored as context grows.
One example from my usage is dependency injection. Our coding agents were constantly taking the easy route and using singletons (they are evil, I don't need to restate why). As a general principle, we want to use dependency injection and so we set up a check to enforce this. Essentially, it meant I didn't have to read the code, find the violation of our principle, and then go back to the agent and say "hey, I asked for dependency injection but you did X." Saved a lot of time.
While our platform uses a continuous learning loop to "teach" the decision model about your judgment over time, I'd expect that you could do this with any decision model if you can figure out the prompts and state that you need.
TL;DR—We've made it work. Saves us money on LLM code reviews. It's great. Highly recommend. 10/10.
Absolutely - the most valuable and expensive datasets to produce are catastrophic real-world failures. We need to encode _at least_ those direct patterns. We also need to extrapolate, simulate, and reinterpret those precious datapoints as the surrounding landscape changes. To the exent we fail to do so, we will be augment the world's catastrophic failure dataset with heartbreaking new datapoints. Victims and their families, such as the plaintiffs in the Boeing Max lawsuits, will want to see those datapoints encapsulated into future processes.
In the past some of these issues were discovered in real life - in blood. I wonder what role some kind of simulation can have in predicting and running scenarios to unlock opportunities low effort high risk. We have a lot of computational power to run this as well as make predictions of what can fail and how bad it could be.
[flagged]
Really good read- the bottleneck for decision making is totally a new paradigm and I struggle with it daily now.
This is super relevant to cyber. Historically we've written rule after rule following every incident but that makes it close to impossible to catch things you haven't seen before. Nice write up!
[flagged]
My p(doom) just increased.
If this is how society learns lessons (and I can see why this has made sense until now) then we're waiting for a paperclip problem before we invest in a kill switch that works.
[flagged]
[flagged]
[flagged]