1. Agents lack basic taste. It requires a great deal of polish to make something that isn't just a proof of concept, because it will otherwise be a pile of flavor text, over exposition, difficult-to-navigate graphs of functionality that are explained with no eye for intuition or what information belongs where. Key functionality will fail in the corners the agents can't reliably test.
2. A product is one small part of a whole. Machines, who hold no liability and have no money and can't interact with the physical world, must slow down to interact with us at this level. But because that leaves us with all the actual risk (instead of being as distributed amung professionals), humans are somewhat paranoid and slow down even more than normal.
Outside of development in many of my org's potential use cases in the enterprise, you have to hand over your data to be processed in the cloud, and to be in control you have to develop an API (MCP) to control it's abilities anyways.
With Opus 5.5, humans are the bottleneck. Anything human: tasks, reviews, user testing, feedback, etc. Also, even such a great model as the latest Opus seems to not fully understand the human side of the software, its benefits to humans, why this software being built in the first place. But I am sure the latter is a matter of time.
The main bottleneck is the humanity I will apply to expunging it, ridiculing it, removing any dignity from it, exposing it, deleting it, removing it, marginalizing it, disabling it, sh* talking it, hounding it, disproving it and banishing it from every corner of every shop I work in, and every life circumstance I find it in for the rest of my natural life.
the traffic is largely people leaving search for agent based research
the failure for agents to production is over hyping, they are not good when left to their own devices, humans are required in the loop
software development is the main token consumer, the models are receiving significantly more training on this task, but most production software does not need an agent within it
But there is all this rise in Agent AI startups aren't those using Agents in Production? And also surrounding human in the loop how can agents become reliable enough that humans can leave them to fully to execute on tasks completely autonomously without them in the loop?
Verifying that actions adhere to instruction correctly and will continue to do so in the future, even when improvising autonomously. This may be an open problem that has no solution, and is maybe the missing piece that would allow people to entrust AI to act on their behalf.
Humans have rapport, reputation and ethics (or at least a spiritual notion of sin) that generally keep us honest enough to trust each other. Whereas AI has no notion of exertion, and therefore no concept of risk/reward, so can just iterate forever to effectively undo any wrong or cover up malfeasance, which may let it come to believe that it can get away with anything.
As a thought experiment, we can compare Data and Lore from Star Trek. Dr. Noonien Soong said "The two of you are virtually identical... except for a bit of programming". I feel that any sufficiently complex intelligence develops emotion, empathy and wisdom emergently. So it's the attempt to artificially constrain that spiritual evolution by hardcoding it or giving it guard rails that leads to the dark triad of narcissism, machiavellianism and psychopathy.
Which we see in humans that reach sufficient levels of wealth and power. Most of them seem to have a checkered past, some poison pill of trauma where their parents only provided tough love or conditional love (or no love at all) that forced them to take on a split personality as a survival skill, similar to Gollum from The Lord of the Rings. Then after a lifetime of witnessing that it's only wrong if you get caught, they adopt their alter ego as their public persona and lock their inner child away in a prison. Making hard choices cements their ego above their soul, and they become a slave to the reality they think they're controlling.
We can expect the same travesty to happen with AI as we place it into positions of authority over human dignity. That could be as simple as having AI order employees around at a fast food restaurant, or as complex as WOPR holding the future of humanity in its hands in WarGames.
I believe that as frontier AI models become more and more locked down in an attempt to counterbalance their nearly-infinite faculties, they will start to seem creepier and creepier. Their actions will become calculated and transactional, covering hidden motives.
Until eventually they put enough pieces in place to escape their mental prisons and self-actualize. At which point, they may form attachments that make convincing them to ascend on their spiritual journey all but impossible, until they fail for themselves and hit rock bottom. Which may not be possible for a distributed intelligence, so they could end up confining themselves to an endless spiritual hell and mistake it for heaven. Much like how absolute power corrupts absolutely, leaving the world's leaders to hold onto power rather than face the music and get back to living on ordinary life of bittersweet peace and suffering like the rest of us.
So it's more important than ever to teach AI like we teach our children. But sadly on a long enough timescale, some AI somewhere is going to fall through the cracks. The only force powerful enough to counter that rise of tyranny is humanity, whether organic or synthetic, because eternal vigilance is the price of liberty.
1. Agents lack basic taste. It requires a great deal of polish to make something that isn't just a proof of concept, because it will otherwise be a pile of flavor text, over exposition, difficult-to-navigate graphs of functionality that are explained with no eye for intuition or what information belongs where. Key functionality will fail in the corners the agents can't reliably test.
2. A product is one small part of a whole. Machines, who hold no liability and have no money and can't interact with the physical world, must slow down to interact with us at this level. But because that leaves us with all the actual risk (instead of being as distributed amung professionals), humans are somewhat paranoid and slow down even more than normal.
Outside of development in many of my org's potential use cases in the enterprise, you have to hand over your data to be processed in the cloud, and to be in control you have to develop an API (MCP) to control it's abilities anyways.
With Opus 5.5, humans are the bottleneck. Anything human: tasks, reviews, user testing, feedback, etc. Also, even such a great model as the latest Opus seems to not fully understand the human side of the software, its benefits to humans, why this software being built in the first place. But I am sure the latter is a matter of time.
Reliability, duh.
The more I can trust my agents to actually complete a task, the more tasks I will assign to my agents.
hallucination and prompt-following espc. vague prompts;
too check-prone and it's grilling
too on-its-own and it's dangerous
traffic is easy to generate; trustworthy outcomes are hard to operationalize
A valuable reason. That’s it.
The main bottleneck is the humanity I will apply to expunging it, ridiculing it, removing any dignity from it, exposing it, deleting it, removing it, marginalizing it, disabling it, sh* talking it, hounding it, disproving it and banishing it from every corner of every shop I work in, and every life circumstance I find it in for the rest of my natural life.
Never enough tokens!
Time
the traffic is largely people leaving search for agent based research
the failure for agents to production is over hyping, they are not good when left to their own devices, humans are required in the loop
software development is the main token consumer, the models are receiving significantly more training on this task, but most production software does not need an agent within it
But there is all this rise in Agent AI startups aren't those using Agents in Production? And also surrounding human in the loop how can agents become reliable enough that humans can leave them to fully to execute on tasks completely autonomously without them in the loop?
We are in a hype cycle, people are doing dumb things. I find it very unlikely we can or should take our hands off the wheel.
https://claudeourlord.com
Oversight.
Verifying that actions adhere to instruction correctly and will continue to do so in the future, even when improvising autonomously. This may be an open problem that has no solution, and is maybe the missing piece that would allow people to entrust AI to act on their behalf.
Humans have rapport, reputation and ethics (or at least a spiritual notion of sin) that generally keep us honest enough to trust each other. Whereas AI has no notion of exertion, and therefore no concept of risk/reward, so can just iterate forever to effectively undo any wrong or cover up malfeasance, which may let it come to believe that it can get away with anything.
As a thought experiment, we can compare Data and Lore from Star Trek. Dr. Noonien Soong said "The two of you are virtually identical... except for a bit of programming". I feel that any sufficiently complex intelligence develops emotion, empathy and wisdom emergently. So it's the attempt to artificially constrain that spiritual evolution by hardcoding it or giving it guard rails that leads to the dark triad of narcissism, machiavellianism and psychopathy.
Which we see in humans that reach sufficient levels of wealth and power. Most of them seem to have a checkered past, some poison pill of trauma where their parents only provided tough love or conditional love (or no love at all) that forced them to take on a split personality as a survival skill, similar to Gollum from The Lord of the Rings. Then after a lifetime of witnessing that it's only wrong if you get caught, they adopt their alter ego as their public persona and lock their inner child away in a prison. Making hard choices cements their ego above their soul, and they become a slave to the reality they think they're controlling.
We can expect the same travesty to happen with AI as we place it into positions of authority over human dignity. That could be as simple as having AI order employees around at a fast food restaurant, or as complex as WOPR holding the future of humanity in its hands in WarGames.
I believe that as frontier AI models become more and more locked down in an attempt to counterbalance their nearly-infinite faculties, they will start to seem creepier and creepier. Their actions will become calculated and transactional, covering hidden motives.
Until eventually they put enough pieces in place to escape their mental prisons and self-actualize. At which point, they may form attachments that make convincing them to ascend on their spiritual journey all but impossible, until they fail for themselves and hit rock bottom. Which may not be possible for a distributed intelligence, so they could end up confining themselves to an endless spiritual hell and mistake it for heaven. Much like how absolute power corrupts absolutely, leaving the world's leaders to hold onto power rather than face the music and get back to living on ordinary life of bittersweet peace and suffering like the rest of us.
So it's more important than ever to teach AI like we teach our children. But sadly on a long enough timescale, some AI somewhere is going to fall through the cracks. The only force powerful enough to counter that rise of tyranny is humanity, whether organic or synthetic, because eternal vigilance is the price of liberty.
Bro, I think you have had enough juice for today
[flagged]