I think a lot of the problem with the current discourse is how black-and-white it is. Either you're a luddite or "ai pilled".
In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more. And heck, sometimes, it just gets you somewhere wrong.
But it seems everyone is arguing about whether LLMs can be perfect software engineers in isolation running in a closet, and using that to say that LLMs do not have a massive potential in other scenarios.
Sometimes, I like to imagine how much more productive most organizations could be from the things that the internet gave us, even to this day. Most companies never really do even a fraction of what is possible. That helps to ground my view of LLMs as well.
The fault dear Brutus isn't in our language models, but in ourselves.
It's funny, but the more I know about the true Luddites, the more I see their point of view.
" the original Luddites were primarily protesting against machinery used to "fraudulently and deceitfully" manufacture inferior goods, bypass labor standards, and strip skilled artisans of their livelihoods."
Goods are usually (although not always) inferior when made by a machine. A hand-crafted solid wood table is still superior to something from Ikea.
Of course hand made tables are expensive. They service a sliver of the market. Ikea serves the rest of us who'd prefer not to eat off the floor.
Fundamentally, Luddites didn't like being replaced by a machine. They were skilled workers, who used to have very desirable skills. Most people didn't need their standard of quality (but customers had no choice.)
Their name is well known today because we never stopped replacing people with machines. Every industry as been "optimized" over and over again since the Luddite times.
AI is the first threat to the Artisans of today (ie programmers). We are just the most recent in a long history of Luddites.
In every change of this nature, some move on embracing the change, others do not. Some will find other jobs, possibly new jobs, others won't. Carriage drivers became Chauffeurs, some grooms became mechanics.
So sure, I'm a Luddite - I don't want to see my skills become cheap - but I'm also pragmatic. The change is here. I'd rather adapt than die.
> Goods are usually (although not always) inferior when made by a machine.
This is only true in the beginning, when machines are still primitive (e.g. first automatic looms). Nowadays machines mostly yield much better quality than any human can produce (e.g. automated welding, anything CNC controlled). Many things are only possible to build with machines (e.g. semiconductors).
> A hand-crafted solid wood table is still superior to something from Ikea.
This is by choice. Ikea chooses to produce the cheapest furniture possible, using cheap, crappy materials. Other manufacturers still produce high quality furniture, which is much more expensive.
Even something as "simple" as a kitchen knife - the same basic concept found in ancient roman everyday kitchens - can't be made to the same quality by a machine as per hand. CNC can't do the tolerances needed behind the apex. Even ignoring the story attached to a specific maker and going solely by cutting ability the really top end hand-made stuff noticeably outperforms everything mass-produced.
The point was to show that machines can produce knifes with precision. The other commenter stated that "CNC can't do the tolerances needed behind the apex.", whatever that means exactly.
Long story short, your point was that machines can do many things better than humans and I argue they can't even do kitchen knives better. How easily and well you cut through things depends on the thickness of the blade wedging them apart. Especially noticeable on denser food like carrots. The apex is the actual pointy part of the blade, and the thickness of the part after the apex and overall blade geometry dictate the cutting experience. Getting the front part thin is usually done via hand-grinding even in the video you shared, a fully controlled process would use something like C&C, but that tech currently can't get things thin enough to be a good knive.
I'd wager that razor blades and scalpels and many other things sharper than kitchen knives are made by machines. Sometimes we don't automate fully expensive things because handmade is also a premium, not necessarily because if we were to dedicate engineering resources to the problem it would be impossible to automate. That said there are things that machines currently can't do despite good efforts to solve the problem. That's true
I know you said “mostly,” but wood is a huge outlier. It’s too heterogeneous to be accurately worked by machines at the level of accuracy that truly fine craftsmanship demands. To be clear, I’m talking about the level where a single chair can cost $20K, and represents hundreds of hours of labor. My FIL is at that level, and the stuff he makes is insane. The market for it is of course tiny, and I’d wager most of his buyers don’t even appreciate the attention to detail he puts into it, but yeah - there are no machines that I’m aware of that can feel perturbations at the sub-mm level and adjust the tool head on the fly, but he can.
This may simply be due to a lack of demand, but regardless, I assure you that machine-produced furniture can’t touch human-produced at the apex of fine craftsmanship.
They don't have particularly high end products, which is not what I claimed anyway. But they have good quality for the price, which extends to medium prices.
> Goods are usually (although not always) inferior when made by a machine. A hand-crafted solid wood table is still superior to something from Ikea.
False. Ikea is not representative of machine-made products, just their own brand of cheap and poorly-designed machine-made products which outsource assembly to the customer for novelty value. Machines are much more precise than humans and do tedious and complex work without cutting corners or getting tired. Buying handcrafted wooden furniture is a class and wealth flex, just like buying any other customized product. The superiority, if any, is entirely in the materials and bespoke design rather than the work - machines can do almost any of the work involved much better than humans.
Seems we've fallen straight back into false dichotomy land.
In that there are a dozen shades of grey between "fully-automated, no humans assembly line" and "person using only hand tools maintained by hand". Tools _are_ machines, are technology, they've just been around so long we forgot that industrially forged steel needs an industrial steel forge.
Probably the best quality product you'll get is from a person who cares, sourcing materials they care about, working with it using their expertise and discernment, and using the most effective tools to get the job done - most, but not all of which will be power or machine tools.
But the point is that the human needs to direct the machines - sometimes that's just a thinking task, and sometimes it's a bunch more physically hands-on.
I know it's not the point of the conversation, but this is very overly harsh to Ikea. Their products are cheap (or, at least, they focus on cheap products), but why poorly-designed? I never had any of their furniture fail on me, and the concessions they make to get the lower price are really not deal-breakers in 99% of cases. It's really not bad at all. Also, selling disassembled furniture saves them a fortune on shipping, they don't do it because they think it's funny or quirky or something.
I wonder what percentage of wooden furniture is truly hand-crafted in whole chain. That is felled with hand-made tools by hand. Transportation maybe can be ignored. Then hand sawed without power tools. All other tools after that being hand made from ground up. Like saws, planes, sand paper, glues(most likely at least some parts are glued together wood)... Most likely vanishingly small number.
Anything “live edge” is often done by hand. Also true for end grain cutting boards, end grain anything really, and any furniture made with mostly burl and heartwood. Coffee tables are common.
The reason is that knotted wood is dangerously unwieldy to machine without a lot of additional preparation. End grain work is just hard to automate. Lots of gluing into a shape that can’t be planed easily and prone to exploding if one is careless.
If you want to peek in to the weird long tail, the guys over at Sawmill Creek love to one-up each other in a never ending contest to be the king of traditional wood working.
Head out to Amish country, there's plenty to be found if you look for it. With that said, depending on the community, they may use tools powered by belt driven by or air compressed by non-electrical means. Each community decides where to draw the line of acceptable modern tech, so you'd need to ask how they build their products.
>Of course hand made tables are expensive. They service a sliver of the market. Ikea serves the rest of us who'd prefer not to eat off the floor.
And yet my working class grandparents didn't eat off the floor, they had great quality tables.
Mine is made of disguised cardboard.
This is a big part of the problem, there is zero trust that any potential improvement in cost or access will reach consumers. Companies don't even bother telling us it will.
We will just be slowly moved into accepting a degradation as the new normal.
Affordable quality is perhaps harder to find in the US than in some other countries. Because professional salaries are so high, the top 10% is responsible for ~50% of consumer spending. That makes middle spenders a less lucrative market than in countries with more equal incomes.
So many posters here going on about cheap IKEA tables. Did they ever visit IKEA? I have an IKEA wooden dinner table with the top made from sturdy panels of solid wood bars glued together for eternity. We've only had it for twenty years of course, so who can tell if it is any good…
Sure, the LACK coffee table is recycled honeycombed cardboard. That's on purpose, and totally fine. I hope people don't eat their dinner of those outside of student life though.
Its interesting, I think people sometimes think there is some binary existence of furniture sometimes! What's nice about todays modern economy is that there is a whole spectrum of tables I can purchase.
I can get a ultra cheap mdf Walmart table, a slightly less cheap Ikea table, maybe a midrange crate and barrel table, or a very expensive hard wood table from the local furniture store. Here in the Midwest we even have hand made Amish furniture available. So buy what you want!
The cost improvements reached you, you just don't see them in the table quality.
You see them in the fact that every single home you'll visit to buy or rent has a fully equipped kitchen including a fridge, oven, likely a microwave, dishwasher and even a washing machine (which alone has a huge economic impact: https://www.youtube.com/watch?v=_gvsz_vc7B0)
You see it in the fact that your home is safer from fires than it ever has been. That hot water is a cheap passive thing you don't even think about, rather than something you have to plan for. That a TV is a nice add-on to it all, rather than a huge deal to get.
Your grandparents' table was more expensive because they had less things, and the massive wood table that they saved months for was what was kept and stood the test of time for you to see today. Because let's not forget, this is also what furnishing 100 years ago can look like:
The real problem is that today, you rarely can pay more to get better. If you pay 3x more for your appliances (TV, dishwasher, oven, etc...) you don't get something 3x more reliable/better engineered.
Because that requires manufacturers ready to give up stealth corner cutting as the cornerstone of their earnings in favour of the hard and long task of developing an image of reliability.
------
Three cases I know enough about: cars, loudspeakers and computer monitors.
You can still buy some Mazda/Toyota models to really get more thoughtful engineering and QC for your money, but the Germans with a similar image of quality (Mercedes, BMW) have partially or fully shed the underlying quality.
Genelec remains the only (non-PA) loudspeaker manufacturer you can sincerely trust to take reliability, performance and transparency seriously. There was also Klein + Hummel (K+H) but since being bought by Sennheiser and integrated with Neumann, things have been going downhill... to the point where some curious people found CapXon caps (bottom of the barrel) in their KH80s.
Computer monitors? Since Panasonic (Eizo's supplier of yore) exited the panel market and left it as LG vs Samsung, it's been a complete disaster. Oh, you wanna pay 1~2k $currency for a fancy OLED monitor? Get used to appalling panel QC (banding, uniformity), VRR flicker and DSC crap.
The available choice for "pay more to get better" continues to dwindle...
And when you do pay more, you're paying more to someone who has figured out how to make you think you are getting better quality, not to someone who is giving you better quality. This is the "market for lemons" effect.
"If you pay 3x more for your appliances (TV, dishwasher, oven, etc...) you don't get something 3x more reliable/better engineered."
You do at the bottom of unregulated markets. For dishwashers and ovens, safety regs generally impose a high floor on the market. There is no $40 oven, because it's physically impossible to make a safety-compliant oven for $40. If it weren't for market regulation, $40 death-trap ovens would be a thing for sure.
The very cheapest compliant unit isn't _much_ worse than a mid-market unit, it might be a bit flimsier and wear out sooner; high-end luxury units aren't much better than mid-market units - because there's not much innovation driving progress at the top end. AEG and Bosch are still generally solid engineering, but there's not much point in paying more than that unless you like the aesthetics.
Mercedes and BMW - small-volume performance models aside - are like the big fashion brands, Vuitton etc., they're selling the idea of luxury to people who aren't even nouveau-riche, more like borrowing money to cosplay loudly as nouveau-riche. Compare old 1970s Merc convertibles with today's, the modern ones are just kind of ugly, aggressive and sad.
ADAM Audio loudspeakers are pretty good or were last time I bought a pair. They're designed as studio monitors but great for listening too. Perhaps they've gone downhill since being bought by a listed company a few years ago?
>ADAM Audio loudspeakers are pretty good or were last time I bought a pair. They're designed as studio monitors but great for listening too. Perhaps they've gone downhill since being bought by a listed company a few years ago?
The Focusrite buyout (unless there was another after it) seem to have improved quality and transparency (i.e. publicly available official measurements for their current range). Still, performance remains lacking for the asking price of the A/S models; the A7V has a massive port resonance near 650 Hz, for example.
But there was also a lot of hard work to push legal reforms and protections to ensure quality and a good living for people displaced by or pressed into work by the new models. It isn't just an "adapt or die" situation on a personal level. Society itself must adapt to produce a healthy human condition.
> Most people didn't need their standard of quality (but customers had no choice.)
Customers don't really have a choice either way. Good luck finding quality clothing, services with decent customer support, etc.
Supply chains supporting quality work are destroyed when an industry gets commoditized, and whenever a company doing quality stuff emerges, it eventually gets bought out and the product gets watered down in order to milk its reputation with inferior product.
> Goods are usually (although not always) inferior when made by a machine. A hand-crafted solid wood table is still superior to something from Ikea.
I would argue that this is quite the opposite. We may have this perception due to how mass-manufactured product are pushed to insane cost-saving measure due to harsh competition.
But machine are far, far more accurate than human, and have been for years. A commercial CNC has insane tolerance, a pick and place machine can accurately place parts that human can barely see, a miter saw can make straight and angle cut that would be very hard using hand tools, ...
And I would argue that your example is even wrong. Almost all Ikea furniture use MDF, which is very dimensionally stable, and once protected with a veneer, is decently resistant to moisture. A solid wood table will contract, warp, etc, depending on the grain of the wood, the humidity, ... And will require much more care and regular use of surface treatment. Of course, "real" wood has its own advantage, but it is a matter of requirement.
And even that "hand-craft" table is not hand-crafted. Any woodworking shop today use machines. Circular saw of many types, power drill, planing machines, ... Which are faster and more accurate than hand tool (although hand tool still have their place).
> Fundamentally, Luddites didn't like being replaced by a machine. They were skilled workers, who used to have very desirable skills. Most people didn't need their standard of quality (but customers had no choice.)
And that's the thing. As you mentioned, very few go to a woodworker to buy a several k$ furniture. Most go to mass-produces-cheap-but-decent furniture companies like Ikea because they don't have a whole month of salary to put in furniture. Machine can absolutely help create far better quality product. But the way of the world has always been to favor cheap but good enough goods.
The big difference with LLMs and "IA" is that they are not a circular saw, they are not a CNC, etc. They are not a tool made for a specific purpose and optimize for it that can reach insane tolerances that no human could match (and especially not as fast). They are, as the post mentioned, "a highly sophisticated statistical model designed to mimic the distribution of programming". There is not really any equivalent in human history. This is a bullshit machine that is scarily good at producing valid output.
It's why I think it is so controversial and why the dust still hasn't settled and why the usefulness of LLM are still subject of such heated debate. A miter saw will cut your plank at a 45-degree angle very fast and very accurately. If you do a lot of that, the benefit is obvious. But if you had a “magical” woodworking tool that could cut at an angle, drill counter-sink, glue veneer, etc, all-in-one but the tolerances are completely random, how useful would that be ? How much time would it save you ? It would be really tough to say.
In the end, the worst part wasn't just loosing artisan jobs and destroying the market for crafted goods. The second order effect was to create a new kind of drudgery in the repetitive and sometimes dangerous work of maintaining the operations of the machine. It created a class of workers who were completely dependent on these kinds of jobs.
And yet, clothes would have remained very expensive if we kept doing fabric by hand. Even destitute people in the poorest countries can have clothes these days, the meaning of “who wears the pants in this house” has lost its original (一条裤子) scarcity meaning.
You're right for the english version, but the Chinese version is IIRC a family that shares one pair of pants - the person who needs to go outside wears the proper clothes.
I was only aware of the Chinese version because my friend explained it to me on my first trip to yunnan in 2002. I always thought the English version descended from the Chinese version, not that they were separate. Oops.
I think you are misrepresenting the luddites. They were not against technological progress. Many of them were the ones who invented the machines. What they were fighting for were labour rights and distribution of power. They were fighting against enshitification by some guy who stole their collective inventions and by force kicked them out.
They were not against more clothes for everyone - quite the opposite. They were against fast fashion bad quality clothes made in horrible conditions by people (or children) who had no other choice.
The Luddites-as-labour-movement view is not entirely wrong, but also misses a lot. They were also an elite group that fought to preserve their own status and the exclusion of women from the industry.
I am no expert but from books I've read on the topic. It was also pretty diverse movement as is usual with decentralized movements like this. There absolutely were lower class people present who saw it as labour movement.
In other words. The Luddites as elite group that fought to preserve status view is not entirely wrong, but also misses a lot.
In context of audience here - programmers (elite group). You can say that programmers fighting with AI do it to preserve their own status. Or we might trust those (like geohot) who are angry because it's just leading to bad results. It's enshitification - of the result, the working conditions, the ethics. The whole chain.
> You can say that programmers fighting with AI do it to preserve their own status. Or we might trust those (like geohot) who are angry because it's just leading to bad results. It's enshitification - of the result, the working conditions, the ethics.
Economic competition should prove either position right in the long term. If AI really is BAD and is just waiting energy, not providing a real benefit and not really making reasonable quality software/content cheaper produce, then the hype will eventually crash. If its SWEs trying to protect their jobs, then one country or the other is going to lean into it for an overwhelming economic advantage and the rest will be forced to follow regardless of what they feel.
The same thing happened during the industrial revolution or even the neolithic revolution. Superior technology will diffuse no matter what barring an overwhelming geographical barrier.
I think the case of Luddites shows just how strong anti-worker and anti-union propaganda is - making it a mock word was very effective at preventing uneducated people from understanding what they fought for.
All learners of Chinese, or just the ones whose native language is English?
I figure the most natural mistake to make when counting pants is to use 件, since that's the measure word that applies to "clothes", but you'd have to already know at least some Chinese in order to make that mistake.
Ah, the ones coming from English of course. I don't have much experience in how Koreans or Russians learn Chinese (although I had plenty of non-English classmates when I took language lessons at PKU).
I'm thinking more Kouyu than writing. If I didn't think hard about it, I might say yi shuang kuzi or even, for some reason, yi jian kuzi.
(It's not the case, however, that all measure words require nouns. 天 ("day") and 年 ("year") are measure words that are almost always used on their own. There might be an implicit notion of "one day of time" or similar.)
> If you are an HN commenter who is more likely invested heavily in tech stocks, much more than a lot.
The current state of the stock market is not exactly inspiring confidence about stability over the next few years. Number goes up over sufficient timescales, but if we get a Dotcom-level bust when AI investment slows, there may be a ways to climb back to current levels...
Exactly. Also, keep in mind that CEOs have both a personal monetary incentive and a fiduciary duty to do what's in the best interest of their company.
It is batshit insane to me that these days we put a microphone to CEOs, ask them to predict the future, and believe it. The only correct answer that they can say into that microphone is whatever is beneficial to their company, nothing less or more.
> the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history
I don’t use agents, myself. I use a simple chat interface, and a running dialogue, to build software at a function-building level. The resulting workflow is quite “chimerical,” and benefits greatly from my own experience and expertise. The LLM simply lubricates the process.
In my case, it seems to be working well. I would not want to go back.
I agree with this. I've tried using agents over the last 2 months, and I feel they are just... bad. I spend more time trying to correct their inexplicable decisions than it takes to just go through step by step in a dialogue.
I think that's geohot's point as well. They're advocating against being fully "ai pilled". Saying we should be using AI as a tool, not for being a luddite.
> But it seems everyone is arguing about whether LLMs can be perfect software engineers
That's just those of us with longer memory holding the AI companies to the standards they declared themselves. Nobody forced Sam Altman to blab about a team of pocket PhDs, did they? I don't want the crap that does it correct 60℅ of the time - where is the god damn nation of PhDs in a datacemter already? Where is the AI doing all the SWE work "in 3-6 months"?
No buddy, not me. Dario Amodei however, keeps announcing it every about 6 months, on the dot. Last time i January this year. So I just want them held acccountable to their own statements. Otherwise if they would be untrue, that is at best incompetence, and at worst investor fraud. Both should draw serious consequences, given the ungodly sums which are burnt into these pipe dreams.
> In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more.
That's my experience too, but it's 60-95% solutions in my case[1], with about 120-140% of lines of code required. I wish there was a harness that would let me mask code it should/n't change, because prompt-based refactors fail from the same over-eagerness.
We had the same issue until we created a review skill that we run after a LLM is done implementing a feature. We give it a list of things to check that is based on the problems we have observed previously, like writing too verbose code, and ask it to report on issues and suggest improvements. The developer can then give feedback and let the LLM fix the issues, or just address them manually. It’s still early but I’ve been much happier now with the results. It makes it much easier as well for humans to review since there’s a report about what the change is about, why, things to keep an eye on etc. This is something you can do with any harness you may be using and there’s nothing to buy, just a suggestion from someone trying to make the best use of this insane technology.
I completely agree with your sentiment of “black or white.” I believe it comes from social media with primarily “radical” perspectives being the ones in the spotlight. Just not an environment that promotes nuance or friendly discussion
The Luddites were (violent) activists. They were more than just "non-believers"
Generally, those being labeled "Luddites" in today's "discourse" are people who dare to question the "AI" hype. GGenerally, these people are not activists
But the big question is "where will '80-95% of the way' get you?"
Do you grind-out the last 5-20% in a period that's disappointingly long compared to the initial step? Or do you another 80% complete thing on top, and another and another until the whole structure collapses?
The post is talking about what groups might go what directions, which seems fair.
Yes, exactly, it's 'us' not the AI, which is great.
Why on earth would we ever remotely compare a 'tool' to 'a software engineer' ?
The 'great delusion' is not that 'AI can't code' - because obviously it can, and very well.
The problem is the 'anthropomorphism' and all this AGI nonsense.
If we called it 'Stochastic Mechanisms' and did not 'personalize' our prompts, refer to them as 'chat' or give them 'personalities' but remained in the domain of 'Stochastic Language CLI' ... then our metaphors would pbably not cloud our judgments.
Because the alternative would've been telling everyone "here's the Stochastic Machine. The more what you write looks like the sources we blatantly stol- I mean, trained from, the closer it gets to output working code". And we're not ready to admit we don't want to know how the sausage is made.
By anthropomorphizing it, we give it some sort of authorship, which clears our collective conscience from what's really happening.
You are a tool. You're a human resource, from the perspective of the organization. That pushes buttons on bunch of other tools. That's why you compare it.
I don't know why you are getting downvoted. Perhaps because people don't like the sentiment. But its true, people are hired as tools to write programs.
Both people and AI make mistakes. Perhaps the AI makes more, a lot more, but its so fast, and works around the clock, and has no ego, there is a chance that the benefits outweigh the costs.
With the level of ability that AI is at right now, I've found it useful personally to think of it something like a very good search over existing knowledge. Another step up in searchability in the lineage of reference books, stack overflow, GitHub etc.
Programmers are rewriting and reinventing the same techniques more often than any other vocation I can think of, and so we were primed for a really good search over prior art. The fact that AI can also adapt that prior art to your particular use case makes it even more powerful.
Much like how great success never came from cobbling together various bits of copy-pasted code from Stack Overflow though, current AI can't really build your whole project.
And on other hand I really do not understand how basic project boilerplate templating wasn't already a fully solved issue. Surely it should have been doable...
I guess the nice thing about AI is it points to the things that we really need to figure out how to abstract instead of rewriting from scratch all the time.
Nice reusable libraries are still a core part of most AI projects, but honestly I think it's not a terrible approach with all the updating dependency malware issues with stuff like NPM.
> I think it's not a terrible approach with all the updating dependency malware issues with stuff like NPM
I think in this instance, the only thing worse than a zero day in your dependency tree, is a zero day you don't know your LLM vendored directly into your codebase...
Personally I feel a vulnerability in local code (unshared ai slop) is much less likely to be exploited, than for say a npm package update that will pwn you as soon as it loads up.
I sort of hold the opposite view: pretty much anything I want to do, there are about ten competing libraries/frameworks/languages, and the lack of commonality across them means I'm often wedged into weird ecosystem choices
Yes, I don't have anything important to say other than I 100% agree with this comment. AI in its current state is akin to Stack Overflow and Google on steroids, but from my experience, it doesn't do well building out full-scale applications other than perhaps some initial scaffolding.
If I were to use it against a legacy, rather poorly written codebase, where the code may be hard to understand without some in-depth analysis. I could certainly ask an AI agent to read the code (How does application X do Y, for example), but I wouldn't have it start hammering out features or have it do any type of refactoring. That would cause far too many commits and confusion amongst the development team, leading to even more slop than whatever we'd already be dealing with.
Just leaving this comment here so I can come back to your comment. I've been getting a bit discouraged by AI lately, but this sums up my experience with it well enough.
> Yes, I don't have anything important to say other than I 100% agree with this comment. AI in its current state is akin to Stack Overflow and Google on steroids, but from my experience, it doesn't do well building out full-scale applications other than perhaps some initial scaffolding.
We're currently using it to build out a full-scale application. It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve.
>It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve.
That is also my experience.
When starting a project I observe how the agent fails, I add new rules to the harness to prevent it from falling and repeat the process until I am happy with the output.
I can't point you to a good complete documentation, because the field is changing very fast as people make new discoveries.
I learned by reading articles, success stories failure stories and mostly by doing, trying stuff, see how it works and adjusting it and burning a lot of tokens along the way.
What I would do in your shoes, I would ask an AI chat to find new articles on the matter (including on HN), explain how Codex, Claude, Pi are managing agents.
My compressed view is: you need to have a great specification both business and architecture wise that doesn't leave anything important for the model to guess because chances are it will make the wrong choices. That comprehensive spec should not be in one huge chunk. Have your plan divided in phases that each fit in a context window and have the spec for each phase.
Use TDD, strive for 100% coverage.
Force the model to behave: if it doesn't do what is supposed to, give it feedback and ask it to retry and don't allow it to progress to the next stage unless everything is perfect.
I also like to write comprehensive integration tests before building anything. The agents are not allowed to touch or read the integration tests, only run them and they will get feedback where the tests fail. I like to build the integration tests in a different language than the software I am building, to make sure there isn't something platform specific that the tests rely on. I use C#, Go, Rust and Zig for development and Python for the integration tests.
For now, to get good results, I can't just copy and paste the setup from a project to another, I have to work a lot to tailor the process for each new codebase.
And that's why I am working on an agent harness to try to force the agents to do the right things in most common development scenarios without wasting much tokens. By common development scenarios I mean that is a large goal, right now I am working towards backend web development and microservices.
In my experience, you’ll eventually hit a context window issue and it will just start spouting gibberish/doing wrong things, and nothing will significantly improve it. But hey, maybe it’s improved.
Well, auto-compaction is a thing in Claude Code now. Plus we have /goal command and some automated review stuff, so you can kinda just get it to loop until the automated reviews are satisfied and CI is passing. Does most of the heavy lifting.
So currently there are people who are buying grey market peptides[1], marked "not for human consumption" and injecting themselves with them based on dubious anecdotes and vibes, to make their skin clearer, build muscle mass, and so on.
Are they are all suddenly turning into zombies? No. Do they have any real idea what that is going to do to their body a few years down the line? Also no. Could it be catastrophic? Maybe!
I think about this when I think about how violently much of the industry has pivoted into AI being the primary generator of code in the last 6ish months. AI is the peptide, your codebase[2] is the body. Literally no one knows how maintainable this approach is, because there simply hasn't been enough time to find out. It could be fine. It could be a complete mess, with your entire engineering team falling asleep at the wheel, lulled into thinking they understand what is being built when they don't, completely impotent to fix or maintain it once the LLM is no longer able to.
I've been thinking the same: the smaller the codebase, the better AI performs. So a way to scale AI is to modularize your architecture to maximize the number of leaf nodes in the dependency tree, and split out separate libraries where it makes sense.
It is huge for token usage also, Claude grepping the codebase for context it doesn't have is the main consumer of input tokens from what I can see.
The whole AI powered utopia is a bet on two possible outcomes:
- humans and companies somehow stop being greedy, selfish and cutting corners to optimize for revenue and time-to-market
- LLMs are the path to artificial superintelligences that will be able to deal with the exponential increase in tech debt from throwing AI slop at the wall (vibecoding) because no one has time to do things “the proper way”
The former is impossible. The latter is extremely unlikely and an existential threat to humanity.
The so called Luddites are the only ones to have even engaged at all with these concerns. Everybody else is just focused on the selfish game (see bet #1) of staying afloat in a rapidly changing ecosystem.
The hardest thing in software engineering is solving the right problem. The ability to identify the right problem to solve, is IMO, what distinguishes the top senior engineers. And we could have endless discussions about what constitutes the right problem, but for the sake of this discussion, let's reduce it to: the problem whose resolution adds the most value to the product for the amount of complexity and afferent costs that it incurs.
Once upon a time, long ago, I worked on a Web product whose original junior designer had figured it would be neat to be able to manage the backend with LDAP tools. So the database schema and structure that the product used mimicked that of OpenLDAP, with compound CN keys, and the entire codebase had to deal with that structure whenever reading from or writing to the DB. LDAP compatibility was not the right problem to solve when designing the DB schema.
But software that solves the right problems can be hard to identify because, quite often, how it does things seems so obvious that it's not readily apparent what other designs might have been chosen.
Now, the thing that usually keeps the blast radius of wrong-problem designs limited over time, is the very friction that they introduce. Development slows down, including the development of more wrong-problem designs. It's a self-limiting phenomenon.
And that's one major thing which worries me about LLM coding agents:
They paper over this friction. They don't repair it; they just make it so its cost is deferred.
So you gradually end up with codebases that grow unboundedly complex for the value they provide, with no controlling mechanisms.
You end up with juniors who never face the feedback loop from which they'd develop the engineering instincts and the taste for what makes a problem the right problem to solve in a given design.
At scale, as a field, you might end up forgetting there ever was such a thing as solving the right problem.
And I don't know what to do about that. Plan for an early retirement, maybe.
Part of my job is working on trying to make these models productive for the large corporation I work for. It's a lot of throwing tomatoes at a wall and to a degree I see the issue he is talking about output seemingly having a certain ceiling.
At the same time in no part of his post is any code snippet or anything to latch on to of "the model performed poorly here when it should have done this" - this style of criticism seems to be a pattern of most of these "the LLMs will never work" style posts on blogs and twitter.
They obviously can perform better than autocomplete and in my own day to day development build out huge portions of a codebase that I would have expected a junior or midlevel engineer to perform at.
How are we really supposed to grasp their actual capabilities when no one will actually cite specifically what mistakes they are making.
> How are we really supposed to grasp their actual capabilities when no one will actually cite specifically what mistakes they are making.
The mistakes they make are pretty subtle. Coding with LLMs can be like that scene in Whiplash – <excellent drumming >, not quite my tempo, <excellent drumming >, downbeat on 18, <excellent drumming>, you’re rushing, <excellent drumming>, dragging, …
Like yeah it produces working code almost always and the code usually does what you asked. And yet it makes you want to throw a chair because it’s not quite right in frustrating ways and it doesn’t even have the taste to know how it’s wrong.
Yeah. It does this. Pretty consistently and replicably depending on the issue, in fact! Yet I can point exactly where it fails.
Why are we not showing the bad choices? On my computer I have hundreds of diffs stored by my agent code review tool that point to style/architecture failures (and in the end, the result of that iteration on the AI output)
I'm not quite sure how people are generating unsalvageable outputs. I'd never ship the result of a first AI pass, either. I review all the code and the architecture, within reason (eg: in Rust I don't preoccupy myself anymore with precisely scoping pub, or whatever, unless I'm making a library crate). I sent a "changes requested" prompt+json to my agent, and it interactively fixes everything (even style, even comments with manual patches with my in-review-tool editor)
Well again that is just a "vibes" explanation with nothing concrete.
I feel like with LLMs, it's like a situation where you are close to some feature or project and have a pretty good idea in your head already of how you'd implement it yourself "I'd do this and have an API with that and a database table foo for storing bar with index on baz" and you're keen to get started on it ...but then someone else gets assigned to work on it not you.
They do it a totally different way than you would have thought of doing it, and the code feels alien and weird because it doesn't follow your "design" and decisions you already had in your head before they started work on it. Is it "bad" or just not how you'd have done it?
I think that is ok. So long as the code works and meets all stated requirements and is secure and performant and uses good abstractions and is not full of hacks, then it's ok to let go. Sure maybe you'd have done it a different way but ultimately that doesn't matter.
> So long as the code works and meets all stated requirements and is secure and performant and uses good abstractions and is not full of hacks, then it's ok to let go
That is the problem. The code often is full of hacks and bad abstractions. LLMs write code like a junior or mid-level engineer – perfectly overfitted to today’s request. Oh you need to work on this code tomorrow and there’s a laundry list of future requirements? Throw away and rewrite, I guess.
You can most easily see this when you ask LLMs to write tests. They have a tendency to write convoluted tests that absolutely definitely pass. Even when you know the code has a bug, they’ll write the test in a way that fits the code as written and passes. Because they know tests should pass.
Getting an LLM to write a failing test against a currently working function because you know the business requirements have changed is like pulling teeth.
You don’t see writing about this stuff because it doesn’t neatly fit in an article or video (I’ve tried). Plus it goes against the zeitgeist so you’d never get traction (even if people write these posts, we don’t see them)
Last week I helped a co-worker with some flaky tests, where the code and tests were generated from one of the models. While looking at how one of the tests work, I'd spotted a place in the code where a boolean condition was backwards in a way a human would never have written (and on top if it, there was a confidently-incorrect block comment above it so it was easy to assume it was correct) - so even if he'd fixed the test's flakiness, it would end up always failing instead of sometimes failing. He'd spent hours trying to figure out what was going on.
When people write blog posts about how LLMs failed for some particular task, the responses from boosters invariably fall along the lines of "just use this other model/just tweak your prompt like so/you're just not skilled enough—you can't make fundamental arguments about AI by citing specific examples."
So we can't make arguments by citing specific examples, and also can't make arguments by not citing specific examples. Whelp, I guess that's the ball game.
(yes yes, I'm committing a group attribution error, but still)
This is an excellent point, and as a novice using LLMs for projects I could never previously dream of doing I find myself looking for the same, examples or citations of what exactly agents are writing incorrectly and how would the human do it better. I'm sure they're out there, maybe someone can refer some good content showing such examples.
I have no doubt the top nth percent of coders could write circles around Claude or Codex, but how much worse are they than your average schnook?
Reality: the top nth percent of coders are seeing absurd, dramatic gains in productivity using LLMs. See: antirez, Simon Willison, Steve Yegge.
The more experience you bring to the table, the more value you get from these tools.
Look, about 12 years ago articles about how if you're not pair programming you're doing it wrong were on HN's home page every day. Doing well prompted plan -> agent -> debug cycles is like pair programming with someone that knows every SDK and API intuitively and doesn't have to pick up their kids from daycare at 4pm.
While I don't actually disagree - to me, Gas Town sounds literally insane - I suspect that if you reframe his work to compare it against the cost of developing a new medication or chip fabrication technique, you can make a strong argument that he's putting his money where his mouth is to see how far he can take a new technology. He's doing science! And I think that's admirable, even if nothing comes of it.
When I think of how much money gets wasted on gambling apps and how much human potential gets wasted watching reality television and compare that to Steve going full Alexander Shulgin with LLMs, the comparison really falls flat.
The problem is what they do to large existing systems: subtle misunderstandings mean subtle bugs are constantly being introduced, and very few shops have adequate systems in place to receive reports of subtle issues at the rates they occurred 10 years ago, let alone today. And don't even get me started on llm-assisted support that some might suggest as a solution.
I'm in the "haven't written any code in a while" boat ATM. I'd love to see examples of issues that are so big that they warrant reverting to manual coding.
My main issue has been the inconsistent quality across between model releases and the tendency to insert older APIs or documentation, especially with command line tools.
I can understand if the model struggles with a million line monolithic codebase with a decade of cruft but can't think of why it'd be too much of a pain with new codebases.
That's fair. In all honesty I'm already feeling challenged but given how much time I save I can set aside some time to keep myself sharp. I can learn more languages. Additionally, as pointed out by others, I'm trading coding effort for design and and strategy, which generally control business outcomes a lot more.
Having said that, I won't use AI for production system if I don't understand the programming constructs in enough detail.
Easily 99 percent on most tasks. As an example, for a Python project with a dozen modules and ~50 files,a simple instruction like "Design a config file backed by Pydantic to store the project's settings. Keep the models modular" sets up nested Pydantic models, moves the settings to sensibly named JSON fields and updates the code to use Pydantic classes everywhere. Takes a few minutes maybe. Manually done the same task would take me half a few hours in the best case and a day in the worst case.
It's not difficult to avoid the 1000 lines per PR thing: Depending on what kind of thing I am adding, the plan might also receive as instructions to value making as small a code change as possible. It still requires judgement, as on something big, the smallest possible code base is not necessarily the most readable, but this is the kind of thing one can decide with some experience and little work.
I've also managed to use LLMs to cut a lot of manual duplication in code where we typically didn't do enough investment: "Claude, evaluate code duplication in the functional test suite" will have no problem finding things like insufficient helpers, or tests that are testing simpler things as prerequisites, so they can rely on each other. So I am not seeing my codebases growing all that much. There's some risks of functional changes that before would be rejected due to cost which now are not, but I am not all that sure of how much that is controllable without being relatively antagonistic with management.
It doesn't engineer. It writes code. Enthusiastically. Usually without thinking about the bigger picture, the design, the architecture, the trade-offs, etc.
It's up to us to manage that process.
It's why senior engineers are finding LLMs a really useful tool - because we've learned to think about all that other stuff before opening the text editor. Writing the actual code was always the easy (and least valuable) bit.
However I find their claim "I've lead teams of really competent engineers and I can leave them without supervision for months and come back and not feel like throwing away the entire code base." dubious. We all know how much effort it is to keep the quality of even small patches consistent.
Design, architecture, style and refactoring still require significant involvement. Providing only a description and a criteria will likely produce hopelessly messy code, which is also what you get with most corporate dev teams.
What type of projects you work on, in particular how rich it is in novelty, non-googlable data points and non-trivial project-specific deviations from industry standards?
This shouldn't actually change virtually anything. We had this happen recently, and were able to rollback within minutes. Devs hand-coding stuff breaks things too. If you already have good observability, fast rollback processes, and feature flag new changes plus do % based rollouts to limit the blast-radius, then it's more or less the same.
sounds like bad deployment practices - canaries, guardrails, fast rollbacks, ring based promotions, cell based architecture, blah blah etc... humans write bad code too, there should be systems in place to protect it from releasing
I think people spend way too much time trying to say that LLMs are bad / shouldn't be used / etc because the LLM can't get it right the first time and/or makes mistakes. I think this is because we all hope that software/computers work like this in an ideal world, and LLMs are software.
This is the wrong mental model.
The way to think about an LLM is like a human: prone to following bad examples if it sees them, needs guardrails to catch mistakes, needs code review. It also needs access to what "correct" looks like: architectural design documents, skills that explain each type of change, etc. It needs prompting/skills telling it to follow a safe workflow, telling it to consider how a safe rollout would work, what a safe rollback would look like, what the performance implications are - just like a human.
The nice thing is that you now have a very knowledgable assistant that can help write additional guardrails that would have always ended at the bottom of your backlog. Perhaps it used to take many hours to research and understand how to write a custom linter to catch a specific coding pattern. Today, ask Claude to do it and an hour later you'll have a custom linter rule for your language of choice, guaranteeing the same mistake can't happen again because CI will block it.
An electronic medical record system. It's got too many moving parts and they all have to unify.
I've had a lot of success in using LLMs in smaller tasks that I peal off myself, but it's largely due to having an existing architecture that makes sense. The times in which I've tried to let the agent loose to make architectural decisions, it tends to wildly-overcomplicate stuff (which I'm only able to recognize because it's a problem I suffer from too).
That's not to say those smaller tasks aren't useful or time-consuming. They're important, but I try to remove the critical assumptions that might need to be made.
Seems like AI isn't really solving complex bugs and issues (that it itself created) in my MAUI project over the last 18 months.
Seems like it is completely hopeless at doing anything netcode consistency and performance related in game dev. Seems like unique game mechanics it doesn't do well either.
Seems like asking it specific UI stylistic changes is basically like throwing darts at a board and hoping it sticks.
Even with relatively simple things, frontier models get me about 90% of the way - and this is without evaluating how good that 90% actually is. It's the last 10% that the model fucking sucks at. And it's often the simplest things. It takes a lot of tokens and a lot of time to cajole the AI to get that last 10% working. And even then, I've just given up and had to go read the slop and fix the bug myself because it become so frustrating.
Agent harnesses have barely been available for a year, and reasonably reliable for only half, and there's already fatigue. I think this says less about whether LLM's will actually be able to program and more about how mentally exhausting AI-assisted programming can be, which involves a higher frequency of decision making and reading an astronomical amount of both code and prose if you actually want to stay on top of what the agent is doing to your codebase. This personal/psychological exhaustion and negative sentiment is now being inaccurately transferred into a pessimistic prognosis for the advancement of the technology itself.
The technology doesn't have an existence outside how people use it.
If everyone who uses it correctly finds it frustrating, and if the only people who love it produce a mountain of unmantainable slop, we will quickly abandon it to the dustbin of history.
A lot of things have "potential" but never amount to anything.
We're going to keep using LLMs but the utility of agentic coding has already peaked in my opinion.
My guess is the models just continue to get better and better
When I got into agentic coding a year or two ago I was sure it was only good at autocomplete. Something happened earlier this year where the models hit a new level of capability.
Everyone I know now just does agentic coding, and it’s really amazing. I think we should just try pushing this as far as we can possibly go, it really feels like the acceleration of the human race is upon us.
We're already hitting some logistical limits. Even if transformers don't have an inherent capability plateau, we only have so many GPUs and so much power to improve them, and we're finding it very difficult to expand that infrastructure. Something like 6 GW of new DCs have been announced over the past 2 years of which less than 1 GW has actually been turned on and started serving, and the deliverable dates for the rest just keep slipping. (Not to mention that the DCs are all talking as if the chips in them will last 6 years, which is turning out to be a stretch.)
Markov chains are very useful and imo so are LLMs, the issue I believe stems from calling it "inteligence" which creates some expectation which is often not met
It's hard to care much about how software is written when superpowers are invading their neighbors, democracy itself is under attack, the food supply chain is failing, and the world is hotter than it's ever been.
software plays a big role in enabling 2 (arguably 3) out of those. Least we could do is not make it worse. But it seems most of us are still just children. They give us something shiny that makes pretty lights and we happily burn the world to keep it.
>... I was sure it was only good at autocomplete. Something happened earlier this year where the models hit a new level of capability.
Yes, something happened, it got better at autocomplete. What else could be? The underlying model hasn't changed.
>acceleration of the human race
Please just stop with this bullshit. Nobody's curing cancer, climate change, inequality or whatever important real problem there is with LLMs. Nobody.
If this tech is good enough to make you more productive is just because you're not working in anything new or cutting edge or innovative. The only reason a LLM knows how to do your job is because that code has been literally written before enough times to appear in the training data. Try to use llms to write C++26, some HDL or in any niche stack and you'll get a nice reality check about LLMs.
I’m sure I would be just as useless as an LLM in the “niche stack” examples that you cited.
Why do you think that is actually a good argument against?
Most “business” problems have already been solved in some way and the times I had to write really novel code in my career have been very very few.
Also sure LLMs haven’t solved cancer or unequality in the few years they exist - but humans also failed here in the last couple thousand
AFL didn't find more vulnerabilities than LLMs. AFL and skilled practitioners found vulnerabilities. AFL triggers faults, many (most?) of which aren't exploitable, and humans (or, now, agents) have to triage and evaluate them. And they did so in a pre-AFL corpus of memory-unsafe software. The heyday of AFL was a decade ago. Every target is harder now.
For context: the author is George "geohot" Hotz, who has a long list of exploits, likely the best known of which is basically vibe coding (I mean that in the nicest possible way) comma.ai for autonomous cars on a shoestring budget long before actual AI vibe coding was a thing.
A pattern i'm seeing is that people working with low level stuff (USB <-> PCIe chip reverse engineering) are much more often refusing the pill. My career is essentially cooking an unholy amount of bog standard nodejs CRUD. And i think i definitely represent a bigger segment than low level gurus, and for me AI is irreplaceable.
ai agents can program, in fact, in our current time with current models, i'd say they program better than most people in the industry (an industry where people were literally copying and pasting from stack overflow for years prior)
being able to program is not the only skill required to be a successful software engineer, so no ai agents cannot be software engineers
very important distinction - i personally like the radiologist example - looking at scans is a part of a radiologist job, AI can do it better than most of them, but looking at scans is a small part of the job, most of it communicating with doctors to help their patients
I suspect is very hard to get a consultant that doesn't hand over a repo of AI generated nonsense right now. For the reason above, they are (usually) under time pressure to deliver, and they can wash their hands of the results at the end of the contract.
I am at 2 for 2 at the moment in the "infrastructure as code" arena (I wasn't involved with choosing to use a consultant, just dealing with the output). Which is an area that AI was supposed to eat for lunch. And it seems like it should, DSLs with a narrower scope seem perfect for an LLM, but I'm not convinced.
I think the issue is, infrastructure DSLs like Terraform or Azure Bicep are distilling down an architecture that has complex interactions and often needs a lot of "inside baseball" knowledge from outside of the code to create a congruent result. Unless you feed a bible of markdown files to the LLM to guide it in the right direction the output goes off the rails fast. The time spend creating the bible might as well be spent creating the code.
Of course there are areas where an LLM will definitely help, like re-factoring, stamping out boiler-plate or even building on a solid base. But attempt to create even a semi-complex architecture from scratch using a few paragraphs of prompt and you are asking for trouble.
The trouble with the consultants I have interacted with is they don't write the bible first, as far as i can tell they just iterate on slop and you end up with multiple 800+ line PowerShell scripts in IaC pipelines and other craziness that is almost impossible to unpick after they have gone.
At a granular level, it's almost guaranteed that you cannot write better code than an agent.
Agents now are writing extremely consistent, normalized canonical code, that usually compiles the first time.
Right out of the 'textbook'.
For what it's trying to do - it writes nearly perfect code.
The only thing you could nominally disagree with are some of the conventions and idioms.
It 'writes a perfect novel, in perfect prose'.
What it will not do however, is 'write the novel that's in your head'.
And that's the crux of it.
It's not even your job to 'write code' at this point, but rather to be the storyteller - and a very good editor who has enough taste and grasp of gammar to be able to know when it's going awry.
It will make mostly what you tell it too, the quality of the output is the quality of your guidance, but at the lowest levels it's generating extremely high quality syntactic prose.
I don't think LLMs inherently do anything perfectly. They can make sure it compiles and passes tests and they can be trained to do an enormous array of tasks, but the code it generates isn't perfect, it's selecting one of many possible outputs based off of some numbers it came up with after a few matrix multiplications and ReLU activations.
Those matrix multiplications aren't a divine perfect thing. They suffer from floating point precision issues and training data issues and there's still debate if adversarial examples are just an unsolveable property of our linear-algebra based neural network architecture.
Can they do things way faster than a human? No doubt. Can they do very complex tasks? Yes. Do they do things with perfection? Not by our human definition of perfect.
"Those matrix multiplications aren't a divine perfect thing. They suffer from floating point precision issues " - this is not the right intuition.
"Not by our human definition of perfect."?
'Human definition' has nothing to do with it.
Your job is to define what you want, to the extent you can do that, the AI does really well at a certain scale, at the 'functional' scale, nearly perfectly.
AI can already easily emulate human creative output [1][2] and there is zero change that you or anyone else can reliably detect AI output with any degree of consistency.
Even with the few 'tell tale' patterns it's been leaving ... that threshold is being moved past quite quickly. Within not even 6 months, works will be identical for all bus some specific activities.
Holding LLMs to the standards of human contextual understanding without communicating sufficient context can be dispensed with, as fantasy.
Using LLMs you quickly learn how much can be inferred from existing, which is dynamic and particular to each model today. There will always be a gap in how much instruction is needed due to mismatches of existing versus intent. Current state, you don't need much to get a lot out that can be verified prior to merge.
LLMs + Harnesses are incredibly effective, as evidenced by the literal millions of people who are paying quite a lot to use them, who speak glowingly of them and would 'never go back'.
Whatever 'shape they take' - they are obviously useful - ergo - 'you're doing something wrong' if you can't make use of them for most tasks.
Mileage varies, there are downsides, but it's the same with anything.
He did admit that he writes his posts in a slightly baity manner. They are his views, but he likes to make waves for the sake of it. It's a smart tactic; it helps his companies if he stays well-known. It's also just fun to read his inflammatory musings. I'm sure he emjoys writing them to.
>But each time I suspected I could have done it better and faster manually.
I've heard this said so many times, but my experience has just been so dramatically the opposite that it rings false. But geohot seems to be a pretty productive and smart guy, so it's hard to just dismiss what he's saying.
I get the sense that he's truly one of the 10x engineers. And maybe he can do it faster and better manually. But for those of us who aren't 10x, I think it lets us bridge that gap. Now we're getting back to "status anxiety": is this an attack on his ego, if the average becomes 10x?
Anecdote: Over 2 weeks of spare time, I used AI tooling to build a fairly sophisticated debian package caching proxy server (~72KLOC, 27K implementation, 45K tests). This would have easily taken me 6 months of focused time to implement by hand. I literally couldn't have done it because I can't take that much time off work and I have other weekend/evening obligations.
Exactly my thoughts, I believe that the 10x engineers are basically living in a bubble and completely oblivious to the average developer/engineer (I consider myself average). Yes the LLMs cannot bring the level of sophistication that geohot would have but they totally could satisfy the needs of an average developer and their average job. More than 95% engineers are not having the problems that geohot is solving and most of our works is straightforward that any LLM can do, thus enabling us to do more of the same and possibly focus on slightly abstract problems if we have the time. Someone said couple of months ago that manual coding will be a privilege and I see that now.
Every C program I've had codex write ended up costing me more time than had I just done it from the start myself. Whereas almost every Python program it's written for me saved me time, even including the time I spent cleaning it up.
I chalk this up to primary two reasons. First, I cared a lot more about the implementation details of the C program than I did the Python one, and second, it's just better at simple stand-alone python programs than it is at C programs.
The criteria I know use is "do I care about the implementation details of this?". If I do (because for example it's going to be long-term code that I need to maintain) then the agent likely isn't worth it. But if I don't, there are huge efficiency gains to be had using the agent.
Data from six months of production from one SaaS codebase provides a more limited response. Maintainability doesn't depend on the level of AI usage. Maintainability depends on the discipline during diff reviews.
Good sessions: One topic per session; scope defined prior to the agent starting; all diffs read prior to committing. Poor sessions: Broad scope; undefined constraints; rubber-stamped results.
The quality of the codebase decays precisely at the rate you stop reading the results. This is not an issue of AI writing the code. This is an issue of unreviewed code. geohot's issue is entirely valid. This problem does exist. But this isn't dependent on the generation phase.
When I started coding with AI I would copy / paste code into GPT-3.5 and ask it to update the code, it was a massive productivity boost, minor changes, fully reviewed. Then VSCode allowed tabbing, it was okayish, but I had my finger on the pulse and knew exactly what was changed and had an opinion on the suggestions. Then cursor allowed you to see and approve changes, after a few changes it started making bigger changes but had an review and approve process, things were starting to feel more magic and required more discipline to be on top of changes. Then YOLO mode hit and you could make massive changes, slowly it became easier and easier to just let the AI build code and you just guide it.
The issue is people mix up complexity, novelty, repeatability and scale.
Well documented complex problems can easily be solved by LLMs.
Doing the same thing over and over again is easy for an LLM.
Novelty and scale is very hard for an LLM.
Even small novel problems confuse LLMs.
When you start a new code base the LLM smashes through the boilerplate work. Then when it gets to scale it struggles with context rot plus novelty.
The AI agents are great, and any expert can prompt them correctly to get good code. LLMs occasionally pick wrong patterns and start digging a hole, but this is why an expert is required. The code itself is just not worth writing when a detailed prompt can get you the same code typing 20x less text.
Where I agree with the post is:
The adoption of AI agents into software engineering is a problem. Solo projects are great, but our teams have not adjusted to the speed-of-change to a mental model of a project. So I see orgs making a choice to either: slow down or forgo the shared mental model.
Anybody choosing to forgo the mental model is building crooked legacy slop at scale.
You can and should save the mental model to an AGENTS.md, but devs need it in their brain to prevent the digging a hole behavior.
To be fair the digging a hole behavior is something humans do just as well. But in teams you'd communicate enough to catch it - hopefully^1. It's the combination of higher speeds and teams that's creating a bit of a disaster.
I'm not sure what a good solution is either. There is a case for solo devs running for 2-month sprints with much more freedom. Perhaps we'll have an "AI Agile manifesto" within a year.
[1] Though you should not underestimate the amount of poor code being created before LLMs. There are enough teams for whom LLMs are practically all upsides. Stay very far away from those.
Thanks for the message, is exactly what I have in my brain and how I'm seeing it unfold.
I'm lucky to be part of different codebases, +200 engineers codebase in a 10 years old company and code, +5 engineers on fresh code. My personal projects, that are beyond POC's, real users, hundreds of commits.
The LLM agent sweet spot is the last one, they are perfect, as I can contain most of the knowledge in my brain of how it works in/out. Speed is insane as a solo developer.
Then the 5 engineers codebase, is also really good, but here you already start to see the problems, thanks to agents you don't even need to care how it works, I have been working on it for +6 months, it uses TRPC and I don't even know (I don't care) how TRPC works. You feel that no one in the team really knows how stuff works at 100% (fresh codebase, we have build this ourselves!!).
Then there is the old codebase with +200 engineers, this is the worst of all of them, you described it perfectly, a bottomless pit of tech debt. This codebase before agents was an old non-typescript one, it was not perfect, but you could build a mental model and understand it perfectly after a few weeks working on it. Now, is a hot-mess of code duplication and the quality is degrading faster and faster as the code gets worse and the Claude Code adoption increases within the engineering team.
Not sure what will be the outcome of all this, but I wouldn't be surprised if some company wakes up in 2027 with a codebase that maintenance and development has increased by x100 fold thanks to Agents.
‘If it hurts, stop’ vs ‘if it hurts, do more of it’. Organizations have a choice, some slow down to avoid, some speed up in hope to make issues… non-issues. If the go-fast orgs find workflows that actually truly speed things up without loss of quality, it’s like hitting the jackpot - you’ve found a way to run away from competition without them even realizing it’s possible (for a while anyway, until they notice they’re grossly outpaced).
>When people see an artifact, they make assumptions about the process that was used to create it. Without even thinking about it, they assume the creator had a basically human state of mind. This assumption is no longer true. Things can be broken in ways that weren’t previously possible, and old proxies of underlying quality like syntax and grammar are useless. AI produced artifacts are not produced by the same process as human ones, and this difference, while extremely subtle in statistics, makes itself obvious when you try to interact with and build on the artifact in human ways.
Once Humans just had oral language, and we could us words to pass ideas from one human mind to another. Then with writing ideas could pass to minds that weren't immediately close together in space or time.. and with this we made complext global spanning civilization. When words just become noise, that one has to be suspect of each one as to whither they'er coming from another human mind, or just a statistical process, can this civilization even survive?
When digital cameras replaced traditional ones, we thought it would make photography more democratic: each of us would be Helmut Newton for 15 minutes. But it didn't give us the beautiful portraits and inspired lanscapes we expected, only millions of pictures of food.
How much will it take for AI agents to pass from distilling decades of collective wisdom to copying each other's worst mistakes?
The thing about having the ability to take effectively free photographs is that it really lets you experiment and learn the edges of what's possible.
I was inspired by Stanford's camera array, and wound up doing virtual focus synthetic aperture photography. I'm hoping to build a rig to do it on near real time, instead of the manual process I used to do on my train rides to and from work.
Sure, the removal of cost lead to a flood of the mundane, but it also means we can capture our lives in ways that even kings couldn't afford in the past. I have thousands of good photos, and even some video, of friends and family.
The comparison between AI and digital photography is a great one IMHO.
I used to be deep into photography in the 00s, until I came to the conclusion that I was spending lots of money, spending hours trying to take the perfect picture and then hours reworking them on the computer... for what?
Sure, I had fun while doing all of this, but to me art is about sharing.
And given the sheer amount of pictures that were published daily on Flickr at the time, I basically had to spend more time in trying to reach out to people to share with them, than in creating something, which was my initial goal.
Or I could strong-harm my relatives, but I myself hate people that are forcing me to watch at their kids/holidays pictures, so...
And looking at the figures on your pictures, taking them at absolute value (not considering what personal value you can give them), it seems that your art is not seen by many.
Contrary to thousands of random pictures of take-away than can reach thousands of eyes in a mater of hours.
Which is, to me, depressing.
You put your soul in something and it has almost no reach, while random snapshots got millions of views.
And I see how the same thing will apply to software: flood of vibecoded trash that can be used by many, while deeply thought after software will be used by a few.
The good news for me here is that I never thought software as art but only a tool. I can vibecode the hell out of an issue, and the code will live on my private repos. Don't care.
Sometimes I put stuff on Github but I don't care if someone (or something) uses it.
I'm focussed on solving my issues.
Oh, there's a story there about assumptions and bad UI. I had fairly large numbers, and tried to whittle down the thousands of photos I had posted to the ones that were favorited, and in the process erased everyone elses favorite tags, leading to rage quitting Flickr for a while. It's all now a mere hint of what it was.
When a blog like this goes completely black or white on a topic I get skeptical. Nothing in life is 0 or 1. So is AI. Has some good to it and some obvious issues. All not that big of a deal. Ppl try to position themselves on the edges bc that’s what’s polarizes and engages conversation…
I do think that between luddite or "ai pilled" ai usage should be much more in favor of "ai pilled".
However, this isn't a plug to be using AI for coding everything, but a more general plug that AI should be integrated to a lot more things outside of the mainstay of chatbots.
There is a lot of merit to using AI to establish a new abstraction layer.
I don't think Geohot has a good idea about LeCun and Hutter's views on the limitations of LLMs. I think that on abstract, textual domains, LLMs perform superbly, and they would agree. I am not too well-informed about LeCun and Hutter's views either, but I think that:
LeCun thinks that LLMs are a bad fit for AI that understands the physical, dynamical systems that we inhabit, and that understanding this is necessary for AGI/ASI.
I don't know that Hutter is bearish on LLMs, but Hutter is interested in AI that can reason exceptionally well given infinite compute, and approximations of such a reasoning AI. I think he is open to the idea that LLMs can be such an approximation.
I think the ai-as-an-exoskeleton analogy is quite apt: 1. It still requires your intent to move in the right direction rather than being fully autonomous, and 2. It's too big and bulky to use for delicate tasks.
Yes. I think of it as a car or tractor (which is also a kind of exoskeleton). You still need intricate knowledge; it's really an amplifier. Steer it wrong and you'll have a 1000 (very hard to detect upfront) bugs in the blink of an eye. Indeed it's hard to wield. At a minimum you need to understand your harness at the character level - the exact shape of the context should be roughly known when operating a harness.
I'm very interested in APIs that allow client-side context construction rather than relying on opaque APIs concatenating strings from your JSON messages and injecting tool prompts. I found that generally, you can craft the entire context as a unicode string and just stuff it in the system message. This works best with models where the chat template is published.
Very much agree. All these currently hyped workflows removing the human from the loop attribute a modularity to those agents that just does not hold up. They will always be leaky abstractions given their stochastic nature.
That being said, they can be a great tool for getting past the "blank page" and just start or getting unstuck in general.
It's not periodic so much as continuous. But yes, HN is constantly where Reddit was about 3 years ago and correspondingly declining. E.g. see the rise in "underrated comment" type comments lately.
"Things can be broken in ways that weren’t previously possible" and also "Things can work in ways that weren’t previously possible". It all depends on what the use the tool for, if you're a carpenter you're going to do a bad job regardless if you have a fancy hammer or a basic one. If you're an expert, give them a basic hammer and they'll do the work, give them a fancy hammer and they'll do the same, perhaps a little faster (or not).
Yet the Eternal September is what made the modern AI possible. I asked Cowork for share of the corpus before / after the event, and the corpus before it is <1%, which fits my hunch.
I don't think you can go completely hands-off for quality products but you can relax and let the agent do as much as possible. It does enable things that probably wouldn't have happened otherwise.
If you are already comfortable with letting other devs work on features then it's easier, because it's similar (arguably you have more control with AI, because what you say goes regardless of hierarchy).
"When people see an artifact, they make assumptions about the process that was used to create it. Without even thinking about it, they assume the creator had a basically human state of mind. This assumption is no longer true."
I've been running into this experience with non-code artifacts, like slideshows and documents.
Didn't expect this to come from him. Seeing some of his recent YouTube streams and previous blog posts, he seemed like he has unconditionally bought into the idea of vibecoding, even as he had Opus 4.5 (latest at the time) stuck failing to enumerate a serial device for solid hours. What a turn.
> It’s definitely a better Google for most searches
I can't agree with this. You tend to get one point of view, often without any actual resources and references so you have to go look it up yourself, on [insert search engine]. Plus, what does it say when we consider an AI the one stop for our data intakes.
how do you measure if google’s engineering org is more productive than meta’s?
What about comparing 2 startups/small teams?
I think the discussion about methods (coding agents included) depends on answering those questions. Seems pointless to claim these agents [dont] make you more productive.
Although, at a first glance, the productivity increase does seem like nothing I’ve seen before. Even more than the transition of making webapps in plain js -> jquery -> frameworks or going from something like Flask to using Rails.
Problem is this is not evidence based. I just feel prototyping has speed up 100x. So the number of iterations/attempts has gone up. Transforming specs into a test suite takes a fraction of the time. Dunno, feels weird not to be able to be overall more productive (do more with less time) if you have these new tools.
But each time I suspected I could have done it better and faster manually
There is a class of tasks that can't be done faster manually, unless you're some sort of colour-smells-like-chicken-and-numbers-have-taste genius. And there is other class (my suspicion now is any non-standard task+framework) that are slower than using agents. So I can imagine you have excellent experience with some tasks like USB hacking and would do it faster than LLM. On the other hand for me, as a Java developer, hacking a USB is finally possible with LLM. Otherwise I'd need to stop-and-learn for some time, which I wouldn't, so either I'd by a more expensive hardware that fulfills my requirements, or put the USB reverse engineering project to my 100 acre todo list
It all depends on the code quality bar. If it's high, a lot of tasks will not be completed much faster. The main speed comes from trusting LLMs output. When you review each change and reprompt LLMs to make the code look like you want. Suddenly, things become much slower and reviews/reprompts are very mentally exhausting.
The issue then becomes it writes insane one liners that are impossible to read. Would be difficult to set up an RL environment where the goal is correctness and readability.
One nasty set of bugs Claude recently introduced; it was doing a large refactor which involved changing call sites to conform to a changed API. Tedious, but straight forward. It helpfully added about 50 if(!something) continue; statements, this would make the code silently absorb issues that should have thrown. Had I accepted this, the results would have made the program run like shit but not crash, making debugging much harder than it needs to be. Really effing annoying! Thanks Claude!
Eh but statistical models are obviously useful, because statistically 99% of your codebase wont involve new idea invention. Tools that write all the boilerplate code used to have names and job titles.
I hate how both the for and against case for LLMs are just so bloody terrible at addressing these things.
This is a good take. The most effective combination of AI and skilled practitioner is using AI to amplify the abilities of the skilled practitioner. And in particular, max benefit comes from exploiting comparative advantage. AIs are really good at boilerplate -- in many cases better than humans because humans will optimize the process by doing copy/paste and often inject errors in the process -- whereas humans are better at abstract and critical reasoning. There's a very real and valuable use case for AI, but it's not replacing humans, it's taking the things that humans don't like doing (and that a computer can do well already) off the human's plate, so humans can focus more exclusively on the things that they do better than the AI. And at least with the current architecture of AI models, there will _always_ be higher-level reasoning that humans do better than the machine.
This. A ~staff software engineer designing big changes at one level above the raw implementation details using Opus 4.7 + superpowers today can genuinely ship multiple times more at the same quality level than pre-AI. The level of what a whole team could ship before.
You have to use something like superpowers, the key is that the humans need to make the important decisions.
You have to review the code - just like you had to review the code humans wrote. There will be iterations.
You have to give the LLM skills and patterns to follow, access to architectural documents, etc, just like humans needed to be onboarded at a company and do the same.
If you get all of these right with today's LLMs, you will never write code at all because it is so obviously not the best use of your time. If you feel that you are still better at writing the code manually, you have not done the above right, fix your workflow and try again.
It really feels like a mass psychosis. I'm not an AI sceptic insofar as I fully expect to get replaced by some future AI system. But what we have now isn't it.
To use a Geohot-inspired analogy, what we have now is like the Google self-driving car of 2010. It works most of the time, yet sometimes fails in unpredictable ways. So you need a safety driver behind the wheel to constantly watch what it's doing (the code review).
A real AI agent would not need a safety driver. We don't have that but many people are basically saying "fuck it, I'm just going to set this car off on its own and see what happens". And sure if you're prototyping it's not dangerous. But for production systems that is dangerous.
There is some very cool tech it just needs continued refinement, there is a path forwards even if it isn't always the clearest. This is happening but it is taking years and a lot of work to get done.
> The bottom performers won’t have that self check. They are the ones producing 10x output with the agents. What do you think is happening to the average output of that organization?
Nailed it!
At my last place this was encouraged (by non-technical leadership driving the AI adoption policies, as well as setting salaries) and seen as a huge win.
The "step change in number of created PR's" was celebrated (cult-style), and by one of the (co) CEO's praised as a paradigm shift of the same magnitude as the personal computer. Meanwhile, I was stuck finding insta-reject level bugs in pull requests from people one-shotting 6000 line PR's "finally solving" long-standing issues from the backlog.
Needless to say I left.
Another problem with perception of AI tools, for coding and other things, is that people often adopt a one-size-fits-all view. If Claude/Codex whatever can fix a bug in my tiny hobby project then it's going to revolutionize all software engineering. If it can write a haiku, then it the great American novel will be dead in a few years and the novelists will starve.
There aren't many truly general purpose tools so viewing things this way seems like either a fantasy or an over-reaction. And if nothing else the processes we use will have to change along with the tools.
It's the early days so we still have a lot to figure out but one of the most significant is which tools are appropriate for what sort of tasks. I've had good luck refactoring a small code base, building some small hobby projects and building features for our company's product. But, I've also dodged bullets doing greenfield development on some features where Claude (my default) has made what seemed like sound choices early on, and which I approved of, only to build something fragile or with unforseen consequences. I haven't quite figured out what distinguished those situations from the successful ones but I'm trying. But it's complicated by the fact that things are evolving quickly and yesterday's failure mode isn't the same as today's and, for that matter, yesterday's successes aren't guaranted to be repeatable today.
On Saturday I thought I had vibe coded myself into a mess. I had implemented a new block type in my structured editor for Practal Zero (or rather let Codex do it), and suddenly the syntax highlighting broke in the whole document. Asking Codex to fix it didn't work. I was contemplating to restart the whole project on a basis that I actually fully understand, but that would set me back so much when the first reasonable prototype seemed so close. Instead, I took a walk.
See, the project actually has a well thought out structure that I design carefully, but more and more of it gets filled out by Codex. Codex is not smart enough to remember all the high-level design considerations, some of which had not been documented because I was just implicitly assuming them. So the fix was to use Codex to isolate the error, think about in terms of the high-level design, and fix the problem, which was partially an implementation problem, and partially a problem of the high-level design.
I fixed the high-level design with discussions with Codex, and documenting this, and then let Codex implement the fixes. The discussion took me more than an hour, the implementation was done in a few minutes.
This working style is similar to doing math: You have a high-level idea of what you are doing, and let that guide you, and Codex assumes the role of something that fills out all of the details you take for granted. Often it turns out your high-level idea had flaws, and this shows up in your code not working as expected. So you revise your high-level idea, refactor the code to reflect the modified high-level design, rinse and repeat.
Working this way is still really hard, but it allows me to do things I could not have done before. Getting your ideas validated (or refuted) in minutes instead of days is huge, and makes it possible to march through stuff that would have turned into a deadly swamp before, at least for me.
Now. Do I think that most corporate programmers will use Codex or CC in this way? I don't know, but I think probably not. So what will stop them going into the swamp until it swallows them, instead of backing up in time and marching around it?
To add, what also often happens in these discussions is that Codex suggests a design that makes no real sense at all, or that it brings up two or three design alternatives, and recommends exactly the wrong one.
"It’s definitely a better Google for most searches"
This is dangerously incorrect. AI summaries of search results consistently return incorrect information and grossly oversimplified and thus misleading summaries, neither of which are detectable unless one either has prior domain knowledge or spends time drilling into search results to validate the AI output.
My experience with ChatGPT as a search engine - it is totally paranoid about checking and re-checking its answers by referencing them in multiple places (I usually read its thinking output). I have not seen an outright hallucination for at least a year. (It is of course a different situation with Google's "AI summary" which is wrong half of the time.)
Ironically I quit using ChatGPT a while back. I decided to run it through it's paces and asked it some rather detailed questions about a range of topics that I have significant domain knowledge on. Without exception the responses I got back where glibly superficial to the point the responses were almost totally devoid of meaningful information. The AI summary on Google search results is so bad it represents an assault on reason.
> and it’s taking longer and longer to realize that they can’t
For something to take "longer and longer" to realise, doesn't they imply that it's been realised at least once before or that there was an expected deadline for the realisation?
I read it as "agents can't program, and with each new generation of agents it's taking longer and longer to realize that that specific iteration can't". Maybe taking the Principle of Charity too far, I dunno.
It's just a tool. Use it well or use it badly - just the same as any. If you are generating slop using the tool, we'll then that is your own problem.
For me, the AI is essentially "faster hands" that can type what I am thinking way faster than I can do it. I tell it what I want, I give it the broad architecture and design patterns/types to use, and any specific test conditions, and let it write all of that usually by the time I have responded to a single email or chat message or two. Custom instructions etc build overtime to address model blind spots or my own personal taste so I don't have to repeat myself in every prompt for cross-cutting things.
Does it "one shot it"? Almost never - we go around the cycle a few times, treating it like pair programming a junior or intern by keeping a close eye on the broad direction and making sure it is acceptable - course-correcting where it matters, but cutting some slack where it doesn't. Sometimes I ask it why it picked a particular approach (that I wouldn't have necessarily) and it gives me a cogent explanation and we go with it, so I actually sometimes learn new things from it too which is great.
The other use case is just it's sheer capacity to research a codebase and hold everything in it's attention at once. It can comprehend unfamiliar code way faster and way more in-depth than I can. So if you are in an unfamiliar code base or a language or framework you are not that familiar with, it absolutely shines because it can just absorb all that info in seconds, and then you can just drill it with questions and what-abouts and how does it do this and what technique is used for that and that, what are the existing patterns and norms in this codebase when it comes to foo or bar? Etc etc
What I am not doing is deferring everything off to the AI unless it really doesn't matter (e.g. disposable one-off or prototype code). Same that I would not expect a junior or intern to make big architectural decisions when implementing something - you keep them on a fairly close leash and watch what they are up to.
Precisely. It is faster than I am at the easy part of writing the code; but the decisions about what to write (at least at a high level) are still my own.
It's a reference to Eternal September, the name applied to the cultural shift of the internet as the general public started gaining access to it en masse in the 90s.
Sloptember is clearly a reference to this - the similarity being that masses of AI generated content, from social media posts to open source contributions are replacing the human internet. In a way this is related to the "dead internet theory", an idea I previously found hard to believe, but these days could easily be true.
If the history of the internet interests you, both these are worth looking up.
> And whenever you need a quick prototype and don’t care about polish, it is absurdly fast. But is it a software engineer? Not close to the bar at any company I have worked at.
This line which he wrote, will override any quality gaps, because the cost to produce that shitty software will be lower than the cost to produce good software.
The more specific your work is, the more these LLM’s seem to struggle.
If your work was previously googling stack overflow, it can be incredibly useful at working through that. Which let’s face it, that’s what a lot of us do.
People misunderstand how AI is used in coding in normal work environments. New feature requirement comes - maybe you need a new service or some new classes. You need to do some research first.
You guide the AI with some prompts and give it some guidance on how to scenario-test it. It makes some classes, test methods. Maybe ~2000 lines and you do a quick verification, check if the overall idea looks okay. Ask it to fix a few design things and then merge it.
Its much easier than doing it yourself with all the boilerplate and understanding each esoteric language specific thing. Which library do I use for UDP communication in golang? The agent might have made a good assumption. These kind of things is where it speeds it up.
I just joined a new team and have been using copilot with opus models.
We have our core code in a weird dialect of C and rust. C I know well, but not rust. Our tests are in Python. The pipeline descriptions are in Yaml.
Outside of the core code there are so many arcana to learn. Writing syntactically and semantically correct yaml/Python test code would be a nightmare. The Agents have flaws, but they provide a huge leg up in improving the tests.
And they are great at providing a first pass review of the core code before bothering a human reviewer. Lastly we run some of our test failures through AI triage, which often enough finds the root cause or rules out simple failures.
This shows up in a higher checkin rate. I'm curious to see whether this will lead to quality end product since we have more support for the more manually written and reviewed core product code.
As an example: I asked the LLM "synonym for "provides" that also means "places" on you" and it gave me 5 answers and I immediately knew the right one was "confers". How? It just fits. Just like most things.
Sometimes I think folks having ‘experiences’ should play more poker.
The point is, it’s a game of chance and yet good players beat bad players in the long run. Your job in the new era of software engineering is to design the process so LLMs doing your code monkeying avoid the losses (including discarding bad changes) and take the wins. Win often enough and you’ll come out ahead.
Hmm you have written it in a way that gives me a new line of thinking.
I think what you are saying is that people should learn and appreciate working in high variance environments and still exploit small gains. This is clearly not something that is easily digestible to people so they end up rejecting LLMs.
that's a very dangerous analogy, because you would be considered the domain expert and you are just asking for synonyms for something you already know but may not remember off-hand.
now, what if you asked for the synonym for "provides" in a language that has gender differences (e.g. spanish/portuguese) as well as societal nuances (e.g. japanese) and it gives you "confers", how would you now know that's correct?
ah, so you say you tell it to take into consideration gender differences, as well as societal nuances. What are those, if you were not already familiar with the language?
You are pushing the limits of my framework and it’s a good thing.
The extent to which LLMs help is determined by how well acquainted you are with the domain. But it will always push you directionally in the right direction.
In your case, you used a language example and this is one where LLMs have natural strength in. I don’t need to be an expert in Spanish to trust it because I know that LLMs are specifically good at catching these problems.
But again there are limits and good to understand it.
We all remember cryptocurrency. Everyone in tech proclaimed fiat was dead, every office buzzed with talk of every possible way that cryptocurrency could be used, billions of dollars flooded in to projects losing money hand over fist. The cynics reacted to the froth with outright rejection of the idea. And today… cryptocurrency exists, it has some use, but it didn’t take over the world, it didn’t kill fiat, it was useful in some areas and worthless in others. AI will be the same. The noisiest proponents will be over exaggerating. The most cynical cynics will be underestimating. The result will be somewhere in the middle. Success will not be predicated on adoption of the technology. We, nerds, are bad at predicting the impact of technology.
I wish people would stop comparing AI with cryptocurrency. The hype/perception was the only thing that was similar between them. The fundamental usefulness of the respective technologies are not comparable.
The cognitive dissonance around this is astounding.
First of all define productive. Would someone using AI to build software at a startup which is likely to fail be considered productive? What if there is already similar software available that solves the same problems? What about the broad use of LLMs to draft emails or make silly memes?
It’s funny how everyone’s concerns around climate change just disappeared when they realised AI was useful to them.
That's the fallacy. You think technology usefulness dictates the outcomes. There are billions of people living in poverty the world over, starving every day, we have the technology and resources to solve that right now, we have for decades, but we don't because we don't want to.
Could AI technology change the world? Sure. Will it? That depends on so much more than what the technology can do. Why are we all still working 40 hours a week? Why are people still hungry? We could have radically changed our world with the technology we have had for decades. Yet, we have not, we have continued, nothing has really changed.
The internet is a great example. What is the most impactful part of the internet today? Social media. Social media has radically changed our culture. What is social media? A database, a few endpoints and an app? The technology is the least consequential part, the consequence comes from how we use it.
Nerds focus on what is possible with the technology, not what society is likely to do with it. What evidence is there that AI is going to change the world? What change is going to come from... being able to generate plausible sounding text? From being able to instruct agents? How many companies are using garbage software from 20 years ago despite dozens of revolutionarily better equivalents being available out there today that could have drastically reshaped their workforce? What are agents if not better macros? How many businesses have hundreds of employees doing the same tasks over and over again that could have been replaced by a few macros? How much of the code that you and I have written in our careers has already been written before?
The fundamental usefulness is the least important part of a technology when discussing how that technology will impact the world.
What "society is likely to do" with a technology drops to zero for things that technology cannot do. That's a technical property. On the other hand, people quite reliably find unexpected uses for technology, beyond the initial hype, again, due to inherent technical properties. You're vastly overcorrecting against "nerds".
Cryptocurrency is more popular and intertwined with the financial system than it ever was so while the claim isn’t currently true it doesn’t mean it won’t be on a long enough timeframe.
If you are old enough then you would be aware that similar claims were made about email but only one country that I know of (the Netherlands) no longer processes mail. Still if we had to guess I would say that we are still early and email will replace the worlds postal systems.
> It is a golden era for buckets and buckets of slop, and a dark age for gems of quality.
I mean, this has been the trend for decades really, before LLMs were a thing. The incentive is skewed toward quantity rather than quality. The new tools just add more fuel to the fire.
Code quality is also really lacking in much of the industry. The truth is, these LLM models, as limited as they are, program at a level above that of the median junior programmer.
"I don’t think models like this will ever be able to program,"
I don't get how anybody who has used the SOTA models in the last 3-4 months can write a sentence like this?
They most certainly can program. And usually better than 90% of my coworkers.
The question is really.. Can they engineer? By which I mean handle the duties of a software engineer working in a team, managing a large complex system, making reviewable pieces, forward progress in incremental steps, etc.
No, that part I'm definitely more skeptical about. That requires slave driving by the person in front of the prompt.
But this is a useful distinction to make. Because making overly pessimistic claims about the coding capabilities of the models makes me question the author's experiences with them.
I think agentic tools are toxic to team programming culture and engineering that produces reliable stable results. But I wouldn't for the life of me question their ability to write programs.
There's a time and a place for assembly language programming. Of course, I knew someone who would say there's a time and a place for machine language programming (improved it by reprogramming a device by flipping the 17 switches on the front panel)
To me this sounds like an old cobbler complaining that machines aren't producing good shoes if left unsupervised and that the old process of making shoes completely by hand is far superior.
So what he is telling us? That agents are not infaillable and they are not capable to one shot complex software and they do not produce perfect code?
We know what and the solution is to use agents for what they are good at and work around their limitations and we have a human in the loop.
>not some RLVR shit that comments out the failing test and tells you all the tests are now passing
That's what harnesses should be about: detect when the agent is misbehaving and force it to take the right approach.
This example in particular should be easy to solve if we generated the tests before coding and we have a workflow or state machine that doesn't allow the agent to disable tests and doesn't allow it to reach the next stage unless all tests are passing.
LLM proponents always use some language like "these old, stuck up dinosaurs with their manual labor vs us cool smart kids with automated labor", but they forget one thing - with automated labor the performance and cost difference was easily measurable in favor of the automation. With LLMs it's neither measurable nor visible (no better software, no faster delivery overall in the industry), and the costs are pretty bad. Besides personal anecdotes of someone toiling away at yet another AI harness project on GitHub.
Right now, to get some good results from AI and save time, you will have to spend a lot of tokens and money. Maybe in the future, the things will get better, I don't know.
Saving money is the wrong reason to use AI now. AI is expensive if you want good results.
But what AI is good for, is it allows you to build fast.
Also, I don't see everything being automated. To get good results you have to drive the AI.
The factories still have workers supervising the process and doing some high value manual processes even if most of the production is done by machines.
This is effectively what's happening to software. We are getting some forms of automation but I believe there's plenty of manual work and coordination left for humans to do.
Coders underestimate the utility of AI in so many boring day to day tasks. If you freelance, that’s where the money is at, not in creating a startup that fills holes in AI offerings or in creating generic slop while hoping for ad money.
Nah this person is dead wrong. Lets come back in 2 years and check on it. I'm willing to make a reasonable bet on these terms: companies will go even more AI native, will use even more tokens and spend even more money.
EDIT: To people downvoting me, please come up with a reasonable bet and lets try to work it out.
EDIT 2: $500 bet paid to your account on whether LLM's are going to still be used productively or not. No one?
EDIT 3: Any bet that would express the author's argument in a way that can be disproven in the future
The author is not arguing against your bet. I think the author wouldn't be surprised if you won that bet. But that doesn't change the argument he does make.
What is a bet that most charitably represents what the author would make? If there’s no such bet then it’s just whining. If there is then I’d like to know.
He is just making a moment-in-time assessment of how he feels about current state of LLM coding. I don't think it's a super-strongly held opinion, I'm sure he would be willing to change his opinion if the next two years of LLM development produce exponentially better results.
> I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history
> Agents _will_ end up hurting large organizations more than high performing individuals or small orgs
> Do you think macOS will get better or worse in the next 2 years?
> I don’t think models like this will _ever_ be able to program, I think the process matters.
From this quote especially how can you say he only thinks about the "current" state of LLM coding?
You are wrong. I don't think this is a point in time assessment. I don't think it is not supper strongly held. I do think he will change his mind though.
You can be completely right in a bad way, and it's what I suspect is going to happen.
Three years ago I mentioned to co-workers I was most concerned about juniors not being able to build skills to become mid-level or senior. In the past year others have started talking about the same. But also I had thought people who were already mid-level or senior could resist and control themselves enough to use it well - but in the past six months two different co-workers have independently said they've noticed their own skills atrophying. And with those skills atrophied, they'll have less and less input/direction for the AI tools.
My suspicion is that for everyone who has gone all-in, a few years from now, they will still see a productivity increase from their own baseline - but their baseline will have dropped from where it is now as they "get used to it".
Author here. I have never said that phrase before this blog post and certainly understand the absurdity of it. I certainly don't mean that you need something biological or whatever consciousness might or might not be.
However there's still a distinction. Unless I'm responding to an LLM, you had a childhood. You learned about the world and space and agency before you ever learned how to program. And you didn't learn it from billions of examples, you learned from a few examples, some self directed experiments, some feedback from teachers, etc...
I'm saying that's what matters. The process matters. You didn't learn to mimic a distribution, you learned to program. Of course in the perfect mathematical limit it's the same, but in practice it's not.
For a lot (most) of what we do with programming, the process actually doesn't matter. I understand you are a real ass dude who is in this shit for the love of the game. I respect that. You are a true artisan and exist in a kind of rarified space. There will always be a place for people like you and in some senses you are correct - you are not replaceable by any AI as they currently function today.
However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
LLMs are even more useful to experts who know the limitations. However the process matters even more if you want to build robust and scalable secure systems that generate millions of dollars and can explain that accurately to high value clients.
I do not want a $10M - $100M dollar issue (lawsuits) because I admitted that I don't understand why a breach happened after using a coding agent. Responsiblity and reputation can't be vibe-coded.
So:
> However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
There's a vast difference between code that works as a prototype vs how it works in production. I don't think you would trust anyone with no experience to fly a commercial plane with them vibe-coding a flight simulator without knowing the process of becoming a pilot.
It's not exactly what it is; they now model an incredibly complex markov process, and harnesses that control how that thinking is done.
Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern.
The LLM in this use case is perfect because almost everything involved is text based, and the model is able to take in all the expressive that is language.
> Is this any different than how a PM gets a programmer to work on a project?
Yes, it's very different. You seem to be suggesting that the current frontier LLMs, when tied to their tools and harnesses, have emergent properties that are similar to human consciousness. If you truly believe that, I'm not sure how to have a productive discussion here.
I think they have the capabilities to execute a well defined plan. If you truly don't believe that, then you I suspect any work you do as a programmer will not survive the coming changes.
It's not just that, but the core is just that, even with reasoning models. Harness can only get you closer to the good result, but can't save you from every pitfall.
As for PM analogy - don't forget that models don't learn and keep doing same stupid stuff they were doing a month ago.
I would suggest you examine current harness memory persistence. Any reprimand you give your model will be remembered, in the same way a puppy that has a bad social experience will become more shy.
They will not save you from every pitfall, but that isn't the point; engineers walk into pitfalls all the time. This can get you in, and out, much much quicker.
But its not useful because even humans are like that - a bunch of neurons slapped together. Overall a tired analogy that is more suited to stay in 2024 where it belongs. Right now it is clear that it is _much_ more than a statistical model semantically. It is misleading to claim it is _just_ that just like a human is _just_ a statistical model.
That's a very good way to describe what they do (better than 'AI') but ironically, it really well explains the mechanism, and how they are in fact able to 'code so well' which is contrary to the authors own premise.
Agents code extremely well.
They're not particularly good at 'architecture' and I think that's where his specific concerns about 'not being able to see the problems' arise - the issues are are almost never in the syntax, because the AI writes perfect code. The issue is that it's not doing exactly what you intended.
Instead of 'missing the target' ... it's 'hit the wrong target perfectly'.
Any senior developer working with AI daily should be able to have a baseline intuition for all of this, and would therefore reject the hyperbole of the premise 'it can't code!'.
Of course it's producing gargantuan amounts of slop - that's not because 'it can't code', that's something else entirely.
> Of course it's producing gargantuan amounts of slop - that's not because 'it can't code'
That is precisely because it can't code! Or rather, it's because it can't reason, or understand things, which in turn means it can't code. The output of LLMs is sloppy because they have no understanding of what they are doing.
That people will say 'Code Me An App' and expect some kind of magical results, will be more common than not, but it's no way evidence that the AI can't code.
Given a sufficiently detailed prompt, the AI will produce almost whatever you ask it within a certain scale.
As sure as the sky is blue.
And it will make perfectly compilable code usually on the first prompt.
Obviously, it can code.
Obviously, it can 'synthetically reason' about the code.
You can point it an arbitrary code base and it will give a better overall assessment than most humans.
Is it fallible? Obviously. Is it limited in scope? Obviously.
It's the mantra of the AI skeptics. Sounds so clever because it's technically true. Just like humans are just piles of oxygen, hydrogen, nitrogen and carbon atoms, along with maybe a dozen or so other trace elements, none of which have any intelligence or will or desire to do anything - hence humans cannot possibly be intelligent or have any free will.
I don't think LeCun is saying they won't be able to program. I think he says we won't hit AGI. Programming does not require AGI; it's a pretty specific skill!
--
I think this article is COPE, if I'm being quite honest. I thought of putting cute analogies, like the C programmers saying the Python and Javascript programmers are not "hardcore" enough... but the truth should be obvious to anyone using LLMs effectively.
--
Current AI is a much better programmer than 100% of people and when directed by someone in that top 10%, it's a force majeur.
I assume you meant something like 'force multiplier"? Force Majeure is an uncontrollable event that prevents a party from fulfilling a contract. Which some may argue is what AI will also deliver. :)
This is my experience. Though ice been writing LLM harnesses, agents, tooling, etc for 5 years now and believe it requires several hundred hours of experience before understanding how to consistently outperform at scale.
" Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming"
In other words - they can program, and probably better than you.
I don't like being too critical but this is a really superficial post - as if either 'AI is a Software Engineer - or - It must be Fraud'
It's an extremely powerful tool that is very 'pattern oriented' and with guidance can absolutely write great code - and even across modules given the right basis.
It's also great at so many other tasks - finding bugs in big code bases, doing migrations etc.
It's not going to make very goo architectural decisions for you, and if you're doing anything novel you have to read most of the code ... but that's too be expected.
It's ridiculous to suggest that 'AI can't code' - when the entire development world has moved into agentic coding, including all of the best developers in the world, and it's yielding positive results in most scenarios.
It's a callow 'bad twitter take' the length of an article.
He's not wrong to suggest that IA is a 'stochastic mechanism' over all the code that's ever been written, but that's evidence of the mechanism, and frankly, describe how it is able to code.
And yes - organizations will misappropriate AI at scale as they do with everything.
His premise is so far out of proportion and misguided, it's tantamount to 'fake moon landing' conspiracy theory.
I think a lot of the problem with the current discourse is how black-and-white it is. Either you're a luddite or "ai pilled".
In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more. And heck, sometimes, it just gets you somewhere wrong.
But it seems everyone is arguing about whether LLMs can be perfect software engineers in isolation running in a closet, and using that to say that LLMs do not have a massive potential in other scenarios.
Sometimes, I like to imagine how much more productive most organizations could be from the things that the internet gave us, even to this day. Most companies never really do even a fraction of what is possible. That helps to ground my view of LLMs as well.
The fault dear Brutus isn't in our language models, but in ourselves.
It's funny, but the more I know about the true Luddites, the more I see their point of view.
" the original Luddites were primarily protesting against machinery used to "fraudulently and deceitfully" manufacture inferior goods, bypass labor standards, and strip skilled artisans of their livelihoods."
Goods are usually (although not always) inferior when made by a machine. A hand-crafted solid wood table is still superior to something from Ikea.
Of course hand made tables are expensive. They service a sliver of the market. Ikea serves the rest of us who'd prefer not to eat off the floor.
Fundamentally, Luddites didn't like being replaced by a machine. They were skilled workers, who used to have very desirable skills. Most people didn't need their standard of quality (but customers had no choice.)
Their name is well known today because we never stopped replacing people with machines. Every industry as been "optimized" over and over again since the Luddite times.
AI is the first threat to the Artisans of today (ie programmers). We are just the most recent in a long history of Luddites.
In every change of this nature, some move on embracing the change, others do not. Some will find other jobs, possibly new jobs, others won't. Carriage drivers became Chauffeurs, some grooms became mechanics.
So sure, I'm a Luddite - I don't want to see my skills become cheap - but I'm also pragmatic. The change is here. I'd rather adapt than die.
> Goods are usually (although not always) inferior when made by a machine.
This is only true in the beginning, when machines are still primitive (e.g. first automatic looms). Nowadays machines mostly yield much better quality than any human can produce (e.g. automated welding, anything CNC controlled). Many things are only possible to build with machines (e.g. semiconductors).
> A hand-crafted solid wood table is still superior to something from Ikea.
This is by choice. Ikea chooses to produce the cheapest furniture possible, using cheap, crappy materials. Other manufacturers still produce high quality furniture, which is much more expensive.
Even something as "simple" as a kitchen knife - the same basic concept found in ancient roman everyday kitchens - can't be made to the same quality by a machine as per hand. CNC can't do the tolerances needed behind the apex. Even ignoring the story attached to a specific maker and going solely by cutting ability the really top end hand-made stuff noticeably outperforms everything mass-produced.
Are you talking about a chef's knife? If well done, I see no reason why it should be any worse than a high end knife made by a skilled human.
Have a look at this if you're interested:
https://www.youtube.com/watch?v=Q5lq2d-03T0
I watched the video, but don’t understand the point you are trying to make. It was only a How things are made collage from some factory.
The point was to show that machines can produce knifes with precision. The other commenter stated that "CNC can't do the tolerances needed behind the apex.", whatever that means exactly.
Long story short, your point was that machines can do many things better than humans and I argue they can't even do kitchen knives better. How easily and well you cut through things depends on the thickness of the blade wedging them apart. Especially noticeable on denser food like carrots. The apex is the actual pointy part of the blade, and the thickness of the part after the apex and overall blade geometry dictate the cutting experience. Getting the front part thin is usually done via hand-grinding even in the video you shared, a fully controlled process would use something like C&C, but that tech currently can't get things thin enough to be a good knive.
I'd wager that razor blades and scalpels and many other things sharper than kitchen knives are made by machines. Sometimes we don't automate fully expensive things because handmade is also a premium, not necessarily because if we were to dedicate engineering resources to the problem it would be impossible to automate. That said there are things that machines currently can't do despite good efforts to solve the problem. That's true
I know you said “mostly,” but wood is a huge outlier. It’s too heterogeneous to be accurately worked by machines at the level of accuracy that truly fine craftsmanship demands. To be clear, I’m talking about the level where a single chair can cost $20K, and represents hundreds of hours of labor. My FIL is at that level, and the stuff he makes is insane. The market for it is of course tiny, and I’d wager most of his buyers don’t even appreciate the attention to detail he puts into it, but yeah - there are no machines that I’m aware of that can feel perturbations at the sub-mm level and adjust the tool head on the fly, but he can.
This may simply be due to a lack of demand, but regardless, I assure you that machine-produced furniture can’t touch human-produced at the apex of fine craftsmanship.
Ikea does sell both ends of range; its has solid wood and particle boards.
They have "bottom of the barrel" and slightly better quality products. But the better products are nowhere near the (top) end of the range.
No, really: They have pretty decent quality, too. Much better than slightly better than bottom the barrel anyway.
But can you show me a "top of the range" product? Not Ikea's top range, but "high end" in the common sense.
They don't have particularly high end products, which is not what I claimed anyway. But they have good quality for the price, which extends to medium prices.
> Goods are usually (although not always) inferior when made by a machine. A hand-crafted solid wood table is still superior to something from Ikea.
False. Ikea is not representative of machine-made products, just their own brand of cheap and poorly-designed machine-made products which outsource assembly to the customer for novelty value. Machines are much more precise than humans and do tedious and complex work without cutting corners or getting tired. Buying handcrafted wooden furniture is a class and wealth flex, just like buying any other customized product. The superiority, if any, is entirely in the materials and bespoke design rather than the work - machines can do almost any of the work involved much better than humans.
Seems we've fallen straight back into false dichotomy land.
In that there are a dozen shades of grey between "fully-automated, no humans assembly line" and "person using only hand tools maintained by hand". Tools _are_ machines, are technology, they've just been around so long we forgot that industrially forged steel needs an industrial steel forge.
Probably the best quality product you'll get is from a person who cares, sourcing materials they care about, working with it using their expertise and discernment, and using the most effective tools to get the job done - most, but not all of which will be power or machine tools.
But the point is that the human needs to direct the machines - sometimes that's just a thinking task, and sometimes it's a bunch more physically hands-on.
I know it's not the point of the conversation, but this is very overly harsh to Ikea. Their products are cheap (or, at least, they focus on cheap products), but why poorly-designed? I never had any of their furniture fail on me, and the concessions they make to get the lower price are really not deal-breakers in 99% of cases. It's really not bad at all. Also, selling disassembled furniture saves them a fortune on shipping, they don't do it because they think it's funny or quirky or something.
I wonder what percentage of wooden furniture is truly hand-crafted in whole chain. That is felled with hand-made tools by hand. Transportation maybe can be ignored. Then hand sawed without power tools. All other tools after that being hand made from ground up. Like saws, planes, sand paper, glues(most likely at least some parts are glued together wood)... Most likely vanishingly small number.
Anything “live edge” is often done by hand. Also true for end grain cutting boards, end grain anything really, and any furniture made with mostly burl and heartwood. Coffee tables are common.
The reason is that knotted wood is dangerously unwieldy to machine without a lot of additional preparation. End grain work is just hard to automate. Lots of gluing into a shape that can’t be planed easily and prone to exploding if one is careless.
If you want to peek in to the weird long tail, the guys over at Sawmill Creek love to one-up each other in a never ending contest to be the king of traditional wood working.
Head out to Amish country, there's plenty to be found if you look for it. With that said, depending on the community, they may use tools powered by belt driven by or air compressed by non-electrical means. Each community decides where to draw the line of acceptable modern tech, so you'd need to ask how they build their products.
>Of course hand made tables are expensive. They service a sliver of the market. Ikea serves the rest of us who'd prefer not to eat off the floor.
And yet my working class grandparents didn't eat off the floor, they had great quality tables.
Mine is made of disguised cardboard.
This is a big part of the problem, there is zero trust that any potential improvement in cost or access will reach consumers. Companies don't even bother telling us it will.
We will just be slowly moved into accepting a degradation as the new normal.
And my dinner table is solid wood from IKEA.
Affordable quality is perhaps harder to find in the US than in some other countries. Because professional salaries are so high, the top 10% is responsible for ~50% of consumer spending. That makes middle spenders a less lucrative market than in countries with more equal incomes.
So many posters here going on about cheap IKEA tables. Did they ever visit IKEA? I have an IKEA wooden dinner table with the top made from sturdy panels of solid wood bars glued together for eternity. We've only had it for twenty years of course, so who can tell if it is any good…
Sure, the LACK coffee table is recycled honeycombed cardboard. That's on purpose, and totally fine. I hope people don't eat their dinner of those outside of student life though.
How many hours it was required for them to work to earn money for that table?
The truth is people mostly want not an improved version but cheaper one.
Its interesting, I think people sometimes think there is some binary existence of furniture sometimes! What's nice about todays modern economy is that there is a whole spectrum of tables I can purchase.
I can get a ultra cheap mdf Walmart table, a slightly less cheap Ikea table, maybe a midrange crate and barrel table, or a very expensive hard wood table from the local furniture store. Here in the Midwest we even have hand made Amish furniture available. So buy what you want!
The cost improvements reached you, you just don't see them in the table quality.
You see them in the fact that every single home you'll visit to buy or rent has a fully equipped kitchen including a fridge, oven, likely a microwave, dishwasher and even a washing machine (which alone has a huge economic impact: https://www.youtube.com/watch?v=_gvsz_vc7B0)
You see it in the fact that your home is safer from fires than it ever has been. That hot water is a cheap passive thing you don't even think about, rather than something you have to plan for. That a TV is a nice add-on to it all, rather than a huge deal to get.
Your grandparents' table was more expensive because they had less things, and the massive wood table that they saved months for was what was kept and stood the test of time for you to see today. Because let's not forget, this is also what furnishing 100 years ago can look like:
https://hips.hearstapps.com/hmg-prod/images/furnishing-for-h...
The real problem is that today, you rarely can pay more to get better. If you pay 3x more for your appliances (TV, dishwasher, oven, etc...) you don't get something 3x more reliable/better engineered.
Because that requires manufacturers ready to give up stealth corner cutting as the cornerstone of their earnings in favour of the hard and long task of developing an image of reliability.
------
Three cases I know enough about: cars, loudspeakers and computer monitors.
You can still buy some Mazda/Toyota models to really get more thoughtful engineering and QC for your money, but the Germans with a similar image of quality (Mercedes, BMW) have partially or fully shed the underlying quality.
Genelec remains the only (non-PA) loudspeaker manufacturer you can sincerely trust to take reliability, performance and transparency seriously. There was also Klein + Hummel (K+H) but since being bought by Sennheiser and integrated with Neumann, things have been going downhill... to the point where some curious people found CapXon caps (bottom of the barrel) in their KH80s.
Computer monitors? Since Panasonic (Eizo's supplier of yore) exited the panel market and left it as LG vs Samsung, it's been a complete disaster. Oh, you wanna pay 1~2k $currency for a fancy OLED monitor? Get used to appalling panel QC (banding, uniformity), VRR flicker and DSC crap.
The available choice for "pay more to get better" continues to dwindle...
And when you do pay more, you're paying more to someone who has figured out how to make you think you are getting better quality, not to someone who is giving you better quality. This is the "market for lemons" effect.
"If you pay 3x more for your appliances (TV, dishwasher, oven, etc...) you don't get something 3x more reliable/better engineered."
You do at the bottom of unregulated markets. For dishwashers and ovens, safety regs generally impose a high floor on the market. There is no $40 oven, because it's physically impossible to make a safety-compliant oven for $40. If it weren't for market regulation, $40 death-trap ovens would be a thing for sure.
The very cheapest compliant unit isn't _much_ worse than a mid-market unit, it might be a bit flimsier and wear out sooner; high-end luxury units aren't much better than mid-market units - because there's not much innovation driving progress at the top end. AEG and Bosch are still generally solid engineering, but there's not much point in paying more than that unless you like the aesthetics.
Mercedes and BMW - small-volume performance models aside - are like the big fashion brands, Vuitton etc., they're selling the idea of luxury to people who aren't even nouveau-riche, more like borrowing money to cosplay loudly as nouveau-riche. Compare old 1970s Merc convertibles with today's, the modern ones are just kind of ugly, aggressive and sad.
ADAM Audio loudspeakers are pretty good or were last time I bought a pair. They're designed as studio monitors but great for listening too. Perhaps they've gone downhill since being bought by a listed company a few years ago?
>ADAM Audio loudspeakers are pretty good or were last time I bought a pair. They're designed as studio monitors but great for listening too. Perhaps they've gone downhill since being bought by a listed company a few years ago?
The Focusrite buyout (unless there was another after it) seem to have improved quality and transparency (i.e. publicly available official measurements for their current range). Still, performance remains lacking for the asking price of the A/S models; the A7V has a massive port resonance near 650 Hz, for example.
Interesting post about an old Adam engineer reminiscing about A5X issues: https://www.audiosciencereview.com/forum/index.php?threads/a...
But there was also a lot of hard work to push legal reforms and protections to ensure quality and a good living for people displaced by or pressed into work by the new models. It isn't just an "adapt or die" situation on a personal level. Society itself must adapt to produce a healthy human condition.
> Most people didn't need their standard of quality (but customers had no choice.)
Customers don't really have a choice either way. Good luck finding quality clothing, services with decent customer support, etc.
Supply chains supporting quality work are destroyed when an industry gets commoditized, and whenever a company doing quality stuff emerges, it eventually gets bought out and the product gets watered down in order to milk its reputation with inferior product.
> Goods are usually (although not always) inferior when made by a machine. A hand-crafted solid wood table is still superior to something from Ikea.
I would argue that this is quite the opposite. We may have this perception due to how mass-manufactured product are pushed to insane cost-saving measure due to harsh competition. But machine are far, far more accurate than human, and have been for years. A commercial CNC has insane tolerance, a pick and place machine can accurately place parts that human can barely see, a miter saw can make straight and angle cut that would be very hard using hand tools, ...
And I would argue that your example is even wrong. Almost all Ikea furniture use MDF, which is very dimensionally stable, and once protected with a veneer, is decently resistant to moisture. A solid wood table will contract, warp, etc, depending on the grain of the wood, the humidity, ... And will require much more care and regular use of surface treatment. Of course, "real" wood has its own advantage, but it is a matter of requirement. And even that "hand-craft" table is not hand-crafted. Any woodworking shop today use machines. Circular saw of many types, power drill, planing machines, ... Which are faster and more accurate than hand tool (although hand tool still have their place).
> Fundamentally, Luddites didn't like being replaced by a machine. They were skilled workers, who used to have very desirable skills. Most people didn't need their standard of quality (but customers had no choice.)
And that's the thing. As you mentioned, very few go to a woodworker to buy a several k$ furniture. Most go to mass-produces-cheap-but-decent furniture companies like Ikea because they don't have a whole month of salary to put in furniture. Machine can absolutely help create far better quality product. But the way of the world has always been to favor cheap but good enough goods.
The big difference with LLMs and "IA" is that they are not a circular saw, they are not a CNC, etc. They are not a tool made for a specific purpose and optimize for it that can reach insane tolerances that no human could match (and especially not as fast). They are, as the post mentioned, "a highly sophisticated statistical model designed to mimic the distribution of programming". There is not really any equivalent in human history. This is a bullshit machine that is scarily good at producing valid output.
It's why I think it is so controversial and why the dust still hasn't settled and why the usefulness of LLM are still subject of such heated debate. A miter saw will cut your plank at a 45-degree angle very fast and very accurately. If you do a lot of that, the benefit is obvious. But if you had a “magical” woodworking tool that could cut at an angle, drill counter-sink, glue veneer, etc, all-in-one but the tolerances are completely random, how useful would that be ? How much time would it save you ? It would be really tough to say.
In the end, the worst part wasn't just loosing artisan jobs and destroying the market for crafted goods. The second order effect was to create a new kind of drudgery in the repetitive and sometimes dangerous work of maintaining the operations of the machine. It created a class of workers who were completely dependent on these kinds of jobs.
And yet, clothes would have remained very expensive if we kept doing fabric by hand. Even destitute people in the poorest countries can have clothes these days, the meaning of “who wears the pants in this house” has lost its original (一条裤子) scarcity meaning.
>the meaning of “who wears the pants in this house”
That's pants as opposed to skirts. It's a gender implication, not a scarcity one.
You're right for the english version, but the Chinese version is IIRC a family that shares one pair of pants - the person who needs to go outside wears the proper clothes.
I was only aware of the Chinese version because my friend explained it to me on my first trip to yunnan in 2002. I always thought the English version descended from the Chinese version, not that they were separate. Oops.
I think you are misrepresenting the luddites. They were not against technological progress. Many of them were the ones who invented the machines. What they were fighting for were labour rights and distribution of power. They were fighting against enshitification by some guy who stole their collective inventions and by force kicked them out.
They were not against more clothes for everyone - quite the opposite. They were against fast fashion bad quality clothes made in horrible conditions by people (or children) who had no other choice.
The Luddites-as-labour-movement view is not entirely wrong, but also misses a lot. They were also an elite group that fought to preserve their own status and the exclusion of women from the industry.
I thought this was a pretty good post about it: https://www.verysane.ai/p/against-the-luddites
I am no expert but from books I've read on the topic. It was also pretty diverse movement as is usual with decentralized movements like this. There absolutely were lower class people present who saw it as labour movement.
In other words. The Luddites as elite group that fought to preserve status view is not entirely wrong, but also misses a lot.
In context of audience here - programmers (elite group). You can say that programmers fighting with AI do it to preserve their own status. Or we might trust those (like geohot) who are angry because it's just leading to bad results. It's enshitification - of the result, the working conditions, the ethics. The whole chain.
> You can say that programmers fighting with AI do it to preserve their own status. Or we might trust those (like geohot) who are angry because it's just leading to bad results. It's enshitification - of the result, the working conditions, the ethics.
Economic competition should prove either position right in the long term. If AI really is BAD and is just waiting energy, not providing a real benefit and not really making reasonable quality software/content cheaper produce, then the hype will eventually crash. If its SWEs trying to protect their jobs, then one country or the other is going to lean into it for an overwhelming economic advantage and the rest will be forced to follow regardless of what they feel.
The same thing happened during the industrial revolution or even the neolithic revolution. Superior technology will diffuse no matter what barring an overwhelming geographical barrier.
I think the case of Luddites shows just how strong anti-worker and anti-union propaganda is - making it a mock word was very effective at preventing uneducated people from understanding what they fought for.
Doesn't that just say "a pair of pants"? Or literally one line pants.
You wouldn't be surprised how often chinese learners say 一双裤子 (one pair of pants) rather than 一条裤子 (one long slender flowing pants).
All learners of Chinese, or just the ones whose native language is English?
I figure the most natural mistake to make when counting pants is to use 件, since that's the measure word that applies to "clothes", but you'd have to already know at least some Chinese in order to make that mistake.
Ah, the ones coming from English of course. I don't have much experience in how Koreans or Russians learn Chinese (although I had plenty of non-English classmates when I took language lessons at PKU).
I'm thinking more Kouyu than writing. If I didn't think hard about it, I might say yi shuang kuzi or even, for some reason, yi jian kuzi.
条 is the appropriate measure word for pants; they're plural in English, but uncountable (like other nouns) in Chinese.
You could translate it as "a pair of pants", and that's the appropriate way to put it in English, but really it says "one pants".
I guess the measure word works equivalently to a unit of measurement, then. One bottle of beer, one sack of sugar, one 条 of pant.
Correct. In Chinese all nouns require them.
(It's not the case, however, that all measure words require nouns. 天 ("day") and 年 ("year") are measure words that are almost always used on their own. There might be an implicit notion of "one day of time" or similar.)
>I think a lot of the problem with the current discourse is how black-and-white it is.
There is too much money involved for any rational debate.
Only on the pro-AI side. The "it is bad" side is diverse on the reasons why, but being overwhelmed by bad content isn't a monetary concern.
Some "ai is bad" arguments come from the threat of losing one's livelihood, which is a monetary concern.
Which is why I phrased it the way I did.
Black and white thinking is not *limited to* monetary concerns.
> There is too much money involved for any rational debate.
For the Sam Altmans of this world, sure, but how much money is the average AI booster commenting on HN actually standing to make?
I get an extra slice of pizza on fridays for being a good little engineer
If you are just invested in an index fund, a lot. If you are an HN commenter who is more likely invested heavily in tech stocks, much more than a lot.
The other side is the stability of your job or job prospects, and we are adversely affected by that instead.
> If you are an HN commenter who is more likely invested heavily in tech stocks, much more than a lot.
The current state of the stock market is not exactly inspiring confidence about stability over the next few years. Number goes up over sufficient timescales, but if we get a Dotcom-level bust when AI investment slows, there may be a ways to climb back to current levels...
You are still financially affected if AI is a bubble set to burst, just as you are if the boom is set to continue.
Exactly. Also, keep in mind that CEOs have both a personal monetary incentive and a fiduciary duty to do what's in the best interest of their company.
It is batshit insane to me that these days we put a microphone to CEOs, ask them to predict the future, and believe it. The only correct answer that they can say into that microphone is whatever is beneficial to their company, nothing less or more.
The article specifically calls out agents:
> the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history
I don’t use agents, myself. I use a simple chat interface, and a running dialogue, to build software at a function-building level. The resulting workflow is quite “chimerical,” and benefits greatly from my own experience and expertise. The LLM simply lubricates the process.
In my case, it seems to be working well. I would not want to go back.
I agree with this. I've tried using agents over the last 2 months, and I feel they are just... bad. I spend more time trying to correct their inexplicable decisions than it takes to just go through step by step in a dialogue.
I think that's geohot's point as well. They're advocating against being fully "ai pilled". Saying we should be using AI as a tool, not for being a luddite.
>In most cases, LLMs can get you 80-95% of the way
On a tangent, this often gets misinterpreted as "LLMs reduce the time it takes to do the thing by 80-95%". That's not what it means.
> But it seems everyone is arguing about whether LLMs can be perfect software engineers
That's just those of us with longer memory holding the AI companies to the standards they declared themselves. Nobody forced Sam Altman to blab about a team of pocket PhDs, did they? I don't want the crap that does it correct 60℅ of the time - where is the god damn nation of PhDs in a datacemter already? Where is the AI doing all the SWE work "in 3-6 months"?
You want the AI that's doing all the SWE work in 3-6 months? Somehow I doubt that.
No buddy, not me. Dario Amodei however, keeps announcing it every about 6 months, on the dot. Last time i January this year. So I just want them held acccountable to their own statements. Otherwise if they would be untrue, that is at best incompetence, and at worst investor fraud. Both should draw serious consequences, given the ungodly sums which are burnt into these pipe dreams.
> In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more.
That's my experience too, but it's 60-95% solutions in my case[1], with about 120-140% of lines of code required. I wish there was a harness that would let me mask code it should/n't change, because prompt-based refactors fail from the same over-eagerness.
1. I try faster, smaller models first.
We had the same issue until we created a review skill that we run after a LLM is done implementing a feature. We give it a list of things to check that is based on the problems we have observed previously, like writing too verbose code, and ask it to report on issues and suggest improvements. The developer can then give feedback and let the LLM fix the issues, or just address them manually. It’s still early but I’ve been much happier now with the results. It makes it much easier as well for humans to review since there’s a report about what the change is about, why, things to keep an eye on etc. This is something you can do with any harness you may be using and there’s nothing to buy, just a suggestion from someone trying to make the best use of this insane technology.
I completely agree with your sentiment of “black or white.” I believe it comes from social media with primarily “radical” perspectives being the ones in the spotlight. Just not an environment that promotes nuance or friendly discussion
"Either you're a luddite or "ai pilled"."
The Luddites were (violent) activists. They were more than just "non-believers"
Generally, those being labeled "Luddites" in today's "discourse" are people who dare to question the "AI" hype. GGenerally, these people are not activists
Semantic weakening is common in all languages, just as literally doesn't literally mean literally.
LLMs can get you 80-95% of the way
But the big question is "where will '80-95% of the way' get you?"
Do you grind-out the last 5-20% in a period that's disappointingly long compared to the initial step? Or do you another 80% complete thing on top, and another and another until the whole structure collapses?
The post is talking about what groups might go what directions, which seems fair.
Yes, exactly, it's 'us' not the AI, which is great.
Why on earth would we ever remotely compare a 'tool' to 'a software engineer' ?
The 'great delusion' is not that 'AI can't code' - because obviously it can, and very well.
The problem is the 'anthropomorphism' and all this AGI nonsense.
If we called it 'Stochastic Mechanisms' and did not 'personalize' our prompts, refer to them as 'chat' or give them 'personalities' but remained in the domain of 'Stochastic Language CLI' ... then our metaphors would pbably not cloud our judgments.
Let the philosophers argue about AGI.
Because the alternative would've been telling everyone "here's the Stochastic Machine. The more what you write looks like the sources we blatantly stol- I mean, trained from, the closer it gets to output working code". And we're not ready to admit we don't want to know how the sausage is made.
By anthropomorphizing it, we give it some sort of authorship, which clears our collective conscience from what's really happening.
The saner philosophers don’t need to argue about AGI because we’re absolutely nowhere near it.
Please give a well defined and agreed upon definition of AGI.
For all I know you're the same guy that says we don't need to talk about nuclear weapons in 1937 because we're nowhere near them.
You are a tool. You're a human resource, from the perspective of the organization. That pushes buttons on bunch of other tools. That's why you compare it.
Edit: I don't mean tool as a perjoritive.
There are innumerable other software and process technologies that we use - and never before have we compared them to 'Engineers'.
The 'tool of the system' analogy is not an unreasonable point of discussion but it does not help us in this scenario.
I think the irony is that the perception of being called a tool as an insult, is exactly your meaning of it.
I don't know why you are getting downvoted. Perhaps because people don't like the sentiment. But its true, people are hired as tools to write programs.
Both people and AI make mistakes. Perhaps the AI makes more, a lot more, but its so fast, and works around the clock, and has no ego, there is a chance that the benefits outweigh the costs.
With the level of ability that AI is at right now, I've found it useful personally to think of it something like a very good search over existing knowledge. Another step up in searchability in the lineage of reference books, stack overflow, GitHub etc.
Programmers are rewriting and reinventing the same techniques more often than any other vocation I can think of, and so we were primed for a really good search over prior art. The fact that AI can also adapt that prior art to your particular use case makes it even more powerful.
Much like how great success never came from cobbling together various bits of copy-pasted code from Stack Overflow though, current AI can't really build your whole project.
> Programmers are rewriting and reinventing the same techniques more often than any other vocation I can think of
And the answer to that is clearly a tool that makes rewriting/reinventing cheaper than actually packaging nice reusable libraries
And on other hand I really do not understand how basic project boilerplate templating wasn't already a fully solved issue. Surely it should have been doable...
I guess the nice thing about AI is it points to the things that we really need to figure out how to abstract instead of rewriting from scratch all the time.
Nice reusable libraries are still a core part of most AI projects, but honestly I think it's not a terrible approach with all the updating dependency malware issues with stuff like NPM.
> I think it's not a terrible approach with all the updating dependency malware issues with stuff like NPM
I think in this instance, the only thing worse than a zero day in your dependency tree, is a zero day you don't know your LLM vendored directly into your codebase...
Personally I feel a vulnerability in local code (unshared ai slop) is much less likely to be exploited, than for say a npm package update that will pwn you as soon as it loads up.
I'm all for packaging nice reusable libraries, but someone has to actually do it. A lot of them just don't exist (yet).
I sort of hold the opposite view: pretty much anything I want to do, there are about ten competing libraries/frameworks/languages, and the lack of commonality across them means I'm often wedged into weird ecosystem choices
Yes, I don't have anything important to say other than I 100% agree with this comment. AI in its current state is akin to Stack Overflow and Google on steroids, but from my experience, it doesn't do well building out full-scale applications other than perhaps some initial scaffolding.
If I were to use it against a legacy, rather poorly written codebase, where the code may be hard to understand without some in-depth analysis. I could certainly ask an AI agent to read the code (How does application X do Y, for example), but I wouldn't have it start hammering out features or have it do any type of refactoring. That would cause far too many commits and confusion amongst the development team, leading to even more slop than whatever we'd already be dealing with.
Just leaving this comment here so I can come back to your comment. I've been getting a bit discouraged by AI lately, but this sums up my experience with it well enough.
> Yes, I don't have anything important to say other than I 100% agree with this comment. AI in its current state is akin to Stack Overflow and Google on steroids, but from my experience, it doesn't do well building out full-scale applications other than perhaps some initial scaffolding.
We're currently using it to build out a full-scale application. It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve.
>It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve.
That is also my experience.
When starting a project I observe how the agent fails, I add new rules to the harness to prevent it from falling and repeat the process until I am happy with the output.
I'm unfamiliar with harness engineering. Is there any good documentation about the subject you could point me to?
https://openai.com/index/harness-engineering/
https://www.anthropic.com/engineering/harness-design-long-ru...
https://www.anthropic.com/engineering/effective-harnesses-fo...
These were some of the first major articles on it. It's becoming a popular topic, so there's more content on it all the time.
I can't point you to a good complete documentation, because the field is changing very fast as people make new discoveries.
I learned by reading articles, success stories failure stories and mostly by doing, trying stuff, see how it works and adjusting it and burning a lot of tokens along the way.
What I would do in your shoes, I would ask an AI chat to find new articles on the matter (including on HN), explain how Codex, Claude, Pi are managing agents.
My compressed view is: you need to have a great specification both business and architecture wise that doesn't leave anything important for the model to guess because chances are it will make the wrong choices. That comprehensive spec should not be in one huge chunk. Have your plan divided in phases that each fit in a context window and have the spec for each phase. Use TDD, strive for 100% coverage. Force the model to behave: if it doesn't do what is supposed to, give it feedback and ask it to retry and don't allow it to progress to the next stage unless everything is perfect. I also like to write comprehensive integration tests before building anything. The agents are not allowed to touch or read the integration tests, only run them and they will get feedback where the tests fail. I like to build the integration tests in a different language than the software I am building, to make sure there isn't something platform specific that the tests rely on. I use C#, Go, Rust and Zig for development and Python for the integration tests.
For now, to get good results, I can't just copy and paste the setup from a project to another, I have to work a lot to tailor the process for each new codebase.
And that's why I am working on an agent harness to try to force the agents to do the right things in most common development scenarios without wasting much tokens. By common development scenarios I mean that is a large goal, right now I am working towards backend web development and microservices.
Sounds like bag pipes to me LOL
In my experience, you’ll eventually hit a context window issue and it will just start spouting gibberish/doing wrong things, and nothing will significantly improve it. But hey, maybe it’s improved.
Well, auto-compaction is a thing in Claude Code now. Plus we have /goal command and some automated review stuff, so you can kinda just get it to loop until the automated reviews are satisfied and CI is passing. Does most of the heavy lifting.
"The fact that AI can also adapt that prior art to your particular use case makes it even more powerful."
Well that's what everyone is claiming anyway
So currently there are people who are buying grey market peptides[1], marked "not for human consumption" and injecting themselves with them based on dubious anecdotes and vibes, to make their skin clearer, build muscle mass, and so on.
Are they are all suddenly turning into zombies? No. Do they have any real idea what that is going to do to their body a few years down the line? Also no. Could it be catastrophic? Maybe!
I think about this when I think about how violently much of the industry has pivoted into AI being the primary generator of code in the last 6ish months. AI is the peptide, your codebase[2] is the body. Literally no one knows how maintainable this approach is, because there simply hasn't been enough time to find out. It could be fine. It could be a complete mess, with your entire engineering team falling asleep at the wheel, lulled into thinking they understand what is being built when they don't, completely impotent to fix or maintain it once the LLM is no longer able to.
[1] https://www.bbc.co.uk/news/articles/cdr268m5pxro
[2] Well, _their_ codebase. I've stopped doing it with my own personal codebases, unless I genuinely don't care about maintainability or longevity
I think smart developers will be building isolated modules, so if your AI generated module keeps failing, you can amputate it and make a fresh one.
I've been thinking the same: the smaller the codebase, the better AI performs. So a way to scale AI is to modularize your architecture to maximize the number of leaf nodes in the dependency tree, and split out separate libraries where it makes sense.
It is huge for token usage also, Claude grepping the codebase for context it doesn't have is the main consumer of input tokens from what I can see.
The whole AI powered utopia is a bet on two possible outcomes:
- humans and companies somehow stop being greedy, selfish and cutting corners to optimize for revenue and time-to-market
- LLMs are the path to artificial superintelligences that will be able to deal with the exponential increase in tech debt from throwing AI slop at the wall (vibecoding) because no one has time to do things “the proper way”
The former is impossible. The latter is extremely unlikely and an existential threat to humanity.
The so called Luddites are the only ones to have even engaged at all with these concerns. Everybody else is just focused on the selfish game (see bet #1) of staying afloat in a rapidly changing ecosystem.
Where do you find the smart developers in 2026. Half are rage quitting and the other half have full-blown psychosis.
And interactive feedback rather than having to specify everything up-front.
One under-discussed phenomenon here, I think:
The hardest thing in software engineering is solving the right problem. The ability to identify the right problem to solve, is IMO, what distinguishes the top senior engineers. And we could have endless discussions about what constitutes the right problem, but for the sake of this discussion, let's reduce it to: the problem whose resolution adds the most value to the product for the amount of complexity and afferent costs that it incurs.
Once upon a time, long ago, I worked on a Web product whose original junior designer had figured it would be neat to be able to manage the backend with LDAP tools. So the database schema and structure that the product used mimicked that of OpenLDAP, with compound CN keys, and the entire codebase had to deal with that structure whenever reading from or writing to the DB. LDAP compatibility was not the right problem to solve when designing the DB schema.
But software that solves the right problems can be hard to identify because, quite often, how it does things seems so obvious that it's not readily apparent what other designs might have been chosen.
Now, the thing that usually keeps the blast radius of wrong-problem designs limited over time, is the very friction that they introduce. Development slows down, including the development of more wrong-problem designs. It's a self-limiting phenomenon.
And that's one major thing which worries me about LLM coding agents:
They paper over this friction. They don't repair it; they just make it so its cost is deferred.
So you gradually end up with codebases that grow unboundedly complex for the value they provide, with no controlling mechanisms.
You end up with juniors who never face the feedback loop from which they'd develop the engineering instincts and the taste for what makes a problem the right problem to solve in a given design.
At scale, as a field, you might end up forgetting there ever was such a thing as solving the right problem.
And I don't know what to do about that. Plan for an early retirement, maybe.
Part of my job is working on trying to make these models productive for the large corporation I work for. It's a lot of throwing tomatoes at a wall and to a degree I see the issue he is talking about output seemingly having a certain ceiling.
At the same time in no part of his post is any code snippet or anything to latch on to of "the model performed poorly here when it should have done this" - this style of criticism seems to be a pattern of most of these "the LLMs will never work" style posts on blogs and twitter.
They obviously can perform better than autocomplete and in my own day to day development build out huge portions of a codebase that I would have expected a junior or midlevel engineer to perform at.
How are we really supposed to grasp their actual capabilities when no one will actually cite specifically what mistakes they are making.
> How are we really supposed to grasp their actual capabilities when no one will actually cite specifically what mistakes they are making.
The mistakes they make are pretty subtle. Coding with LLMs can be like that scene in Whiplash – <excellent drumming >, not quite my tempo, <excellent drumming >, downbeat on 18, <excellent drumming>, you’re rushing, <excellent drumming>, dragging, …
Like yeah it produces working code almost always and the code usually does what you asked. And yet it makes you want to throw a chair because it’s not quite right in frustrating ways and it doesn’t even have the taste to know how it’s wrong.
Yeah. It does this. Pretty consistently and replicably depending on the issue, in fact! Yet I can point exactly where it fails.
Why are we not showing the bad choices? On my computer I have hundreds of diffs stored by my agent code review tool that point to style/architecture failures (and in the end, the result of that iteration on the AI output)
I'm not quite sure how people are generating unsalvageable outputs. I'd never ship the result of a first AI pass, either. I review all the code and the architecture, within reason (eg: in Rust I don't preoccupy myself anymore with precisely scoping pub, or whatever, unless I'm making a library crate). I sent a "changes requested" prompt+json to my agent, and it interactively fixes everything (even style, even comments with manual patches with my in-review-tool editor)
Well again that is just a "vibes" explanation with nothing concrete.
I feel like with LLMs, it's like a situation where you are close to some feature or project and have a pretty good idea in your head already of how you'd implement it yourself "I'd do this and have an API with that and a database table foo for storing bar with index on baz" and you're keen to get started on it ...but then someone else gets assigned to work on it not you.
They do it a totally different way than you would have thought of doing it, and the code feels alien and weird because it doesn't follow your "design" and decisions you already had in your head before they started work on it. Is it "bad" or just not how you'd have done it?
I think that is ok. So long as the code works and meets all stated requirements and is secure and performant and uses good abstractions and is not full of hacks, then it's ok to let go. Sure maybe you'd have done it a different way but ultimately that doesn't matter.
> So long as the code works and meets all stated requirements and is secure and performant and uses good abstractions and is not full of hacks, then it's ok to let go
That is the problem. The code often is full of hacks and bad abstractions. LLMs write code like a junior or mid-level engineer – perfectly overfitted to today’s request. Oh you need to work on this code tomorrow and there’s a laundry list of future requirements? Throw away and rewrite, I guess.
You can most easily see this when you ask LLMs to write tests. They have a tendency to write convoluted tests that absolutely definitely pass. Even when you know the code has a bug, they’ll write the test in a way that fits the code as written and passes. Because they know tests should pass.
Getting an LLM to write a failing test against a currently working function because you know the business requirements have changed is like pulling teeth.
You don’t see writing about this stuff because it doesn’t neatly fit in an article or video (I’ve tried). Plus it goes against the zeitgeist so you’d never get traction (even if people write these posts, we don’t see them)
Last week I helped a co-worker with some flaky tests, where the code and tests were generated from one of the models. While looking at how one of the tests work, I'd spotted a place in the code where a boolean condition was backwards in a way a human would never have written (and on top if it, there was a confidently-incorrect block comment above it so it was easy to assume it was correct) - so even if he'd fixed the test's flakiness, it would end up always failing instead of sometimes failing. He'd spent hours trying to figure out what was going on.
When people write blog posts about how LLMs failed for some particular task, the responses from boosters invariably fall along the lines of "just use this other model/just tweak your prompt like so/you're just not skilled enough—you can't make fundamental arguments about AI by citing specific examples."
So we can't make arguments by citing specific examples, and also can't make arguments by not citing specific examples. Whelp, I guess that's the ball game.
(yes yes, I'm committing a group attribution error, but still)
This is an excellent point, and as a novice using LLMs for projects I could never previously dream of doing I find myself looking for the same, examples or citations of what exactly agents are writing incorrectly and how would the human do it better. I'm sure they're out there, maybe someone can refer some good content showing such examples.
I have no doubt the top nth percent of coders could write circles around Claude or Codex, but how much worse are they than your average schnook?
Reality: the top nth percent of coders are seeing absurd, dramatic gains in productivity using LLMs. See: antirez, Simon Willison, Steve Yegge.
The more experience you bring to the table, the more value you get from these tools.
Look, about 12 years ago articles about how if you're not pair programming you're doing it wrong were on HN's home page every day. Doing well prompted plan -> agent -> debug cycles is like pair programming with someone that knows every SDK and API intuitively and doesn't have to pick up their kids from daycare at 4pm.
antirez is famous for creating Redis, which took a dump in quality and everyone switched to a fork called Valkey.
Rubbish. The license change was the reason for the fork of community, and people switching. Quality was never cited as the issue.
and Steve Yegge is currently just burning mountains of money with Gas Town or whatever came after that
While I don't actually disagree - to me, Gas Town sounds literally insane - I suspect that if you reframe his work to compare it against the cost of developing a new medication or chip fabrication technique, you can make a strong argument that he's putting his money where his mouth is to see how far he can take a new technology. He's doing science! And I think that's admirable, even if nothing comes of it.
When I think of how much money gets wasted on gambling apps and how much human potential gets wasted watching reality television and compare that to Steve going full Alexander Shulgin with LLMs, the comparison really falls flat.
The problem is what they do to large existing systems: subtle misunderstandings mean subtle bugs are constantly being introduced, and very few shops have adequate systems in place to receive reports of subtle issues at the rates they occurred 10 years ago, let alone today. And don't even get me started on llm-assisted support that some might suggest as a solution.
This article goes into quite a lot of detailed examples that include code snippets that demonstrate poor architecture: https://blog.k10s.dev/im-going-back-to-writing-code-by-hand/
I'm in the "haven't written any code in a while" boat ATM. I'd love to see examples of issues that are so big that they warrant reverting to manual coding.
My main issue has been the inconsistent quality across between model releases and the tendency to insert older APIs or documentation, especially with command line tools.
I can understand if the model struggles with a million line monolithic codebase with a decade of cruft but can't think of why it'd be too much of a pain with new codebases.
> I'm in the "haven't written any code in a while" boat ATM
How long do you think it will be before you can't write any code because you're out of practice?
One of the dangers of engineering management is that it can turn you into a person that can no longer do the thing.
Does that even matter?
How long will it be until you can't spin the thread and fabricate your clothes stitch by stitch because you're out of practice?
That's fair. In all honesty I'm already feeling challenged but given how much time I save I can set aside some time to keep myself sharp. I can learn more languages. Additionally, as pointed out by others, I'm trading coding effort for design and and strategy, which generally control business outcomes a lot more.
Having said that, I won't use AI for production system if I don't understand the programming constructs in enough detail.
> In all honesty I'm already feeling challenged but given how much time I save
And how much is that?
Easily 99 percent on most tasks. As an example, for a Python project with a dozen modules and ~50 files,a simple instruction like "Design a config file backed by Pydantic to store the project's settings. Keep the models modular" sets up nested Pydantic models, moves the settings to sensibly named JSON fields and updates the code to use Pydantic classes everywhere. Takes a few minutes maybe. Manually done the same task would take me half a few hours in the best case and a day in the worst case.
AI: I would urge you to reconsider, this is a multi week project.
Me: Do it anyway
10 minutes later
AI: Perfect!
How many more languages have you learnt, and how much time have you spent keeping yourself sharp? 99% of your work time, right?
The ability to read code doesn’t decay at nearly the same rate. Neither does your experience.
I read plenty of books, but I'd struggle to write one because that's a different skill that I don't have.
I’m not entirely convinced that’s true. Is there evidence that someone well-read would also be a bad writer?
I review every diff the clanker makes.
After a few hours of this I still look at the codebase and think "wtf is this?".
I think writing the code is a very important part of understanding it. LLM driven development is like doing maintenance programming from day one.
I've used that as well, it's like starting with a legacy app every time.
When every prompt produces a thousand line PR, you’re not very far from another million line monolith.
I’m a little more hopeful than the author though. I feel like it’s possible to manage the process so that does not happen.
It's not difficult to avoid the 1000 lines per PR thing: Depending on what kind of thing I am adding, the plan might also receive as instructions to value making as small a code change as possible. It still requires judgement, as on something big, the smallest possible code base is not necessarily the most readable, but this is the kind of thing one can decide with some experience and little work.
I've also managed to use LLMs to cut a lot of manual duplication in code where we typically didn't do enough investment: "Claude, evaluate code duplication in the functional test suite" will have no problem finding things like insufficient helpers, or tests that are testing simpler things as prerequisites, so they can rely on each other. So I am not seeing my codebases growing all that much. There's some risks of functional changes that before would be rejected due to cost which now are not, but I am not all that sure of how much that is controllable without being relatively antagonistic with management.
Monolith is an ordinary form of software btw - big ball of mud is the problem.
> manage the process so that does not happen
This is the gold, right here.
It doesn't engineer. It writes code. Enthusiastically. Usually without thinking about the bigger picture, the design, the architecture, the trade-offs, etc.
It's up to us to manage that process.
It's why senior engineers are finding LLMs a really useful tool - because we've learned to think about all that other stuff before opening the text editor. Writing the actual code was always the easy (and least valuable) bit.
Here's one that hit the frontpage recently:
https://blog.k10s.dev/im-going-back-to-writing-code-by-hand/
That's indeed a good example.
However I find their claim "I've lead teams of really competent engineers and I can leave them without supervision for months and come back and not feel like throwing away the entire code base." dubious. We all know how much effort it is to keep the quality of even small patches consistent.
Design, architecture, style and refactoring still require significant involvement. Providing only a description and a criteria will likely produce hopelessly messy code, which is also what you get with most corporate dev teams.
It isn't "so big." Things can also be "so small" that it's not worth it.
Coding isn't very hard, so it's often easier to just code than read and write English. I write Haskell exclusively though so this might bias me.
What type of projects you work on, in particular how rich it is in novelty, non-googlable data points and non-trivial project-specific deviations from industry standards?
Even if agents failed at that I'd wager that's a very small percentage of software projects anyway.
It's basically every game targetting consoles. Good luck finding any real infos about the Nintendo SDK API on Google.
> I'd love to see examples of issues that are so big that they warrant reverting to manual coding
Ah I see your org hasnt yet had an outage caused by a bad LLM code push.
This shouldn't actually change virtually anything. We had this happen recently, and were able to rollback within minutes. Devs hand-coding stuff breaks things too. If you already have good observability, fast rollback processes, and feature flag new changes plus do % based rollouts to limit the blast-radius, then it's more or less the same.
sounds like bad deployment practices - canaries, guardrails, fast rollbacks, ring based promotions, cell based architecture, blah blah etc... humans write bad code too, there should be systems in place to protect it from releasing
I think people spend way too much time trying to say that LLMs are bad / shouldn't be used / etc because the LLM can't get it right the first time and/or makes mistakes. I think this is because we all hope that software/computers work like this in an ideal world, and LLMs are software.
This is the wrong mental model.
The way to think about an LLM is like a human: prone to following bad examples if it sees them, needs guardrails to catch mistakes, needs code review. It also needs access to what "correct" looks like: architectural design documents, skills that explain each type of change, etc. It needs prompting/skills telling it to follow a safe workflow, telling it to consider how a safe rollout would work, what a safe rollback would look like, what the performance implications are - just like a human.
The nice thing is that you now have a very knowledgable assistant that can help write additional guardrails that would have always ended at the bottom of your backlog. Perhaps it used to take many hours to research and understand how to write a custom linter to catch a specific coding pattern. Today, ask Claude to do it and an hour later you'll have a custom linter rule for your language of choice, guaranteeing the same mistake can't happen again because CI will block it.
"Ah I see your org hasnt yet had an outage caused by a bad LLM code push"
"We went back to shovelling by hand because someone ran over the pole with the front-loader, even though he had no experience driving it."
This is definitely user error; obviously it's a hard tool to wrangle but it's entirely possible to use it safely.
An electronic medical record system. It's got too many moving parts and they all have to unify.
I've had a lot of success in using LLMs in smaller tasks that I peal off myself, but it's largely due to having an existing architecture that makes sense. The times in which I've tried to let the agent loose to make architectural decisions, it tends to wildly-overcomplicate stuff (which I'm only able to recognize because it's a problem I suffer from too).
That's not to say those smaller tasks aren't useful or time-consuming. They're important, but I try to remove the critical assumptions that might need to be made.
As long as you don't make stuff that others have to rely on, you can live as dangerously as you want.
Seems like AI isn't really solving complex bugs and issues (that it itself created) in my MAUI project over the last 18 months.
Seems like it is completely hopeless at doing anything netcode consistency and performance related in game dev. Seems like unique game mechanics it doesn't do well either.
Seems like asking it specific UI stylistic changes is basically like throwing darts at a board and hoping it sticks.
Even with relatively simple things, frontier models get me about 90% of the way - and this is without evaluating how good that 90% actually is. It's the last 10% that the model fucking sucks at. And it's often the simplest things. It takes a lot of tokens and a lot of time to cajole the AI to get that last 10% working. And even then, I've just given up and had to go read the slop and fix the bug myself because it become so frustrating.
Agent harnesses have barely been available for a year, and reasonably reliable for only half, and there's already fatigue. I think this says less about whether LLM's will actually be able to program and more about how mentally exhausting AI-assisted programming can be, which involves a higher frequency of decision making and reading an astronomical amount of both code and prose if you actually want to stay on top of what the agent is doing to your codebase. This personal/psychological exhaustion and negative sentiment is now being inaccurately transferred into a pessimistic prognosis for the advancement of the technology itself.
https://evilmartians.com/chronicles/ai-assisted-engineers-ar...
The technology doesn't have an existence outside how people use it.
If everyone who uses it correctly finds it frustrating, and if the only people who love it produce a mountain of unmantainable slop, we will quickly abandon it to the dustbin of history.
A lot of things have "potential" but never amount to anything.
We're going to keep using LLMs but the utility of agentic coding has already peaked in my opinion.
My guess is the models just continue to get better and better
When I got into agentic coding a year or two ago I was sure it was only good at autocomplete. Something happened earlier this year where the models hit a new level of capability.
Everyone I know now just does agentic coding, and it’s really amazing. I think we should just try pushing this as far as we can possibly go, it really feels like the acceleration of the human race is upon us.
We're already hitting some logistical limits. Even if transformers don't have an inherent capability plateau, we only have so many GPUs and so much power to improve them, and we're finding it very difficult to expand that infrastructure. Something like 6 GW of new DCs have been announced over the past 2 years of which less than 1 GW has actually been turned on and started serving, and the deliverable dates for the rest just keep slipping. (Not to mention that the DCs are all talking as if the chips in them will last 6 years, which is turning out to be a stretch.)
Sounds like we just need a Dyson sphere.
Besides, I have been hearing "this is the limit" since the doomers of "this is just a markov chain and can't be useful".
Yet the limits keep being broken.
Markov chains are very useful and imo so are LLMs, the issue I believe stems from calling it "inteligence" which creates some expectation which is often not met
Most of your comment history is defending LLM psychosis
Most of your comment history is less than a month old
what if we're accelerating to a brick wall?
More like neo-feudalism by way of breadlines.
It's hard to care much about how software is written when superpowers are invading their neighbors, democracy itself is under attack, the food supply chain is failing, and the world is hotter than it's ever been.
software plays a big role in enabling 2 (arguably 3) out of those. Least we could do is not make it worse. But it seems most of us are still just children. They give us something shiny that makes pretty lights and we happily burn the world to keep it.
Acceleration of the human race is the biggest cope I’ve read all year.
>... I was sure it was only good at autocomplete. Something happened earlier this year where the models hit a new level of capability.
Yes, something happened, it got better at autocomplete. What else could be? The underlying model hasn't changed.
>acceleration of the human race
Please just stop with this bullshit. Nobody's curing cancer, climate change, inequality or whatever important real problem there is with LLMs. Nobody.
If this tech is good enough to make you more productive is just because you're not working in anything new or cutting edge or innovative. The only reason a LLM knows how to do your job is because that code has been literally written before enough times to appear in the training data. Try to use llms to write C++26, some HDL or in any niche stack and you'll get a nice reality check about LLMs.
I’m sure I would be just as useless as an LLM in the “niche stack” examples that you cited.
Why do you think that is actually a good argument against? Most “business” problems have already been solved in some way and the times I had to write really novel code in my career have been very very few.
Also sure LLMs haven’t solved cancer or unequality in the few years they exist - but humans also failed here in the last couple thousand
AFL didn't find more vulnerabilities than LLMs. AFL and skilled practitioners found vulnerabilities. AFL triggers faults, many (most?) of which aren't exploitable, and humans (or, now, agents) have to triage and evaluate them. And they did so in a pre-AFL corpus of memory-unsafe software. The heyday of AFL was a decade ago. Every target is harder now.
For context: the author is George "geohot" Hotz, who has a long list of exploits, likely the best known of which is basically vibe coding (I mean that in the nicest possible way) comma.ai for autonomous cars on a shoestring budget long before actual AI vibe coding was a thing.
https://en.wikipedia.org/wiki/George_Hotz
A pattern i'm seeing is that people working with low level stuff (USB <-> PCIe chip reverse engineering) are much more often refusing the pill. My career is essentially cooking an unholy amount of bog standard nodejs CRUD. And i think i definitely represent a bigger segment than low level gurus, and for me AI is irreplaceable.
ai agents can program, in fact, in our current time with current models, i'd say they program better than most people in the industry (an industry where people were literally copying and pasting from stack overflow for years prior)
being able to program is not the only skill required to be a successful software engineer, so no ai agents cannot be software engineers
very important distinction - i personally like the radiologist example - looking at scans is a part of a radiologist job, AI can do it better than most of them, but looking at scans is a small part of the job, most of it communicating with doctors to help their patients
Not reviewing outputs, which is my main issue, is one-way to subpar experience. No amount of "make it right" will fix that.
I hope that professionalism still matters as these new ways of doing things strikes me as unprofessional as f...
Yeah, the next macOS will be worse... time to place bet on prediction market
I agree that I can write better code than an agent.
But it can write working code much faster than I can.
And in a lot of cases, unfortunately, faster beats better.
> Unfortunately, faster beats better.
I think you have just written the epitaph for corporate software.
I suspect is very hard to get a consultant that doesn't hand over a repo of AI generated nonsense right now. For the reason above, they are (usually) under time pressure to deliver, and they can wash their hands of the results at the end of the contract.
I am at 2 for 2 at the moment in the "infrastructure as code" arena (I wasn't involved with choosing to use a consultant, just dealing with the output). Which is an area that AI was supposed to eat for lunch. And it seems like it should, DSLs with a narrower scope seem perfect for an LLM, but I'm not convinced.
I think the issue is, infrastructure DSLs like Terraform or Azure Bicep are distilling down an architecture that has complex interactions and often needs a lot of "inside baseball" knowledge from outside of the code to create a congruent result. Unless you feed a bible of markdown files to the LLM to guide it in the right direction the output goes off the rails fast. The time spend creating the bible might as well be spent creating the code.
Of course there are areas where an LLM will definitely help, like re-factoring, stamping out boiler-plate or even building on a solid base. But attempt to create even a semi-complex architecture from scratch using a few paragraphs of prompt and you are asking for trouble.
The trouble with the consultants I have interacted with is they don't write the bible first, as far as i can tell they just iterate on slop and you end up with multiple 800+ line PowerShell scripts in IaC pipelines and other craziness that is almost impossible to unpick after they have gone.
I agree it can write faster in a language I don’t know very well
At a granular level, it's almost guaranteed that you cannot write better code than an agent.
Agents now are writing extremely consistent, normalized canonical code, that usually compiles the first time.
Right out of the 'textbook'.
For what it's trying to do - it writes nearly perfect code.
The only thing you could nominally disagree with are some of the conventions and idioms.
It 'writes a perfect novel, in perfect prose'.
What it will not do however, is 'write the novel that's in your head'.
And that's the crux of it.
It's not even your job to 'write code' at this point, but rather to be the storyteller - and a very good editor who has enough taste and grasp of gammar to be able to know when it's going awry.
It will make mostly what you tell it too, the quality of the output is the quality of your guidance, but at the lowest levels it's generating extremely high quality syntactic prose.
I don't think LLMs inherently do anything perfectly. They can make sure it compiles and passes tests and they can be trained to do an enormous array of tasks, but the code it generates isn't perfect, it's selecting one of many possible outputs based off of some numbers it came up with after a few matrix multiplications and ReLU activations.
Those matrix multiplications aren't a divine perfect thing. They suffer from floating point precision issues and training data issues and there's still debate if adversarial examples are just an unsolveable property of our linear-algebra based neural network architecture.
Can they do things way faster than a human? No doubt. Can they do very complex tasks? Yes. Do they do things with perfection? Not by our human definition of perfect.
"Those matrix multiplications aren't a divine perfect thing. They suffer from floating point precision issues " - this is not the right intuition.
"Not by our human definition of perfect."?
'Human definition' has nothing to do with it.
Your job is to define what you want, to the extent you can do that, the AI does really well at a certain scale, at the 'functional' scale, nearly perfectly.
Syntax is the least of my concerns.
Modelling a problem is what I'm concerned about. And I'm currently better than any AI agent at doing that, given enough time.
It can’t even write a passable novel… if you know of a great LLM-authored novel I should read please point me to it and I’ll take your recommendation!
AI can already easily emulate human creative output [1][2] and there is zero change that you or anyone else can reliably detect AI output with any degree of consistency.
Even with the few 'tell tale' patterns it's been leaving ... that threshold is being moved past quite quickly. Within not even 6 months, works will be identical for all bus some specific activities.
[1] https://www.sciencedirect.com/science/article/pii/S147738802... [2] https://link.springer.com/article/10.1186/s41077-025-00396-6...
> the quality of the output is the quality of your guidance
I wish you just started with the copout.
How is it a copout? These exercises are making an assumption that developers have been given even more context.
It's the LLM version of "You're holding it wrong."
I don't see how you think that's a cop out. Instructions are never laid out perfectly:
https://www.commitstrip.com/en/2016/08/25/a-very-comprehensi...
Holding LLMs to the standards of human contextual understanding without communicating sufficient context can be dispensed with, as fantasy.
Using LLMs you quickly learn how much can be inferred from existing, which is dynamic and particular to each model today. There will always be a gap in how much instruction is needed due to mismatches of existing versus intent. Current state, you don't need much to get a lot out that can be verified prior to merge.
-> You Are Holding It Wrong.
LLMs + Harnesses are incredibly effective, as evidenced by the literal millions of people who are paying quite a lot to use them, who speak glowingly of them and would 'never go back'.
Whatever 'shape they take' - they are obviously useful - ergo - 'you're doing something wrong' if you can't make use of them for most tasks.
Mileage varies, there are downsides, but it's the same with anything.
> as evidenced by the literal millions of people who are paying quite a lot to use them, who speak glowingly of them and would 'never go back'.
Anecdotes from sunk cost users aren't evidence.
I think geohot is somewhat of a clown but I think he is speaking reason here and I'm happy to see voices address this. Most seniors I work with agree.
He did admit that he writes his posts in a slightly baity manner. They are his views, but he likes to make waves for the sake of it. It's a smart tactic; it helps his companies if he stays well-known. It's also just fun to read his inflammatory musings. I'm sure he emjoys writing them to.
>But each time I suspected I could have done it better and faster manually.
I've heard this said so many times, but my experience has just been so dramatically the opposite that it rings false. But geohot seems to be a pretty productive and smart guy, so it's hard to just dismiss what he's saying.
I get the sense that he's truly one of the 10x engineers. And maybe he can do it faster and better manually. But for those of us who aren't 10x, I think it lets us bridge that gap. Now we're getting back to "status anxiety": is this an attack on his ego, if the average becomes 10x?
Anecdote: Over 2 weeks of spare time, I used AI tooling to build a fairly sophisticated debian package caching proxy server (~72KLOC, 27K implementation, 45K tests). This would have easily taken me 6 months of focused time to implement by hand. I literally couldn't have done it because I can't take that much time off work and I have other weekend/evening obligations.
Exactly my thoughts, I believe that the 10x engineers are basically living in a bubble and completely oblivious to the average developer/engineer (I consider myself average). Yes the LLMs cannot bring the level of sophistication that geohot would have but they totally could satisfy the needs of an average developer and their average job. More than 95% engineers are not having the problems that geohot is solving and most of our works is straightforward that any LLM can do, thus enabling us to do more of the same and possibly focus on slightly abstract problems if we have the time. Someone said couple of months ago that manual coding will be a privilege and I see that now.
Every C program I've had codex write ended up costing me more time than had I just done it from the start myself. Whereas almost every Python program it's written for me saved me time, even including the time I spent cleaning it up.
I chalk this up to primary two reasons. First, I cared a lot more about the implementation details of the C program than I did the Python one, and second, it's just better at simple stand-alone python programs than it is at C programs.
The criteria I know use is "do I care about the implementation details of this?". If I do (because for example it's going to be long-term code that I need to maintain) then the agent likely isn't worth it. But if I don't, there are huge efficiency gains to be had using the agent.
Data from six months of production from one SaaS codebase provides a more limited response. Maintainability doesn't depend on the level of AI usage. Maintainability depends on the discipline during diff reviews. Good sessions: One topic per session; scope defined prior to the agent starting; all diffs read prior to committing. Poor sessions: Broad scope; undefined constraints; rubber-stamped results.
The quality of the codebase decays precisely at the rate you stop reading the results. This is not an issue of AI writing the code. This is an issue of unreviewed code. geohot's issue is entirely valid. This problem does exist. But this isn't dependent on the generation phase.
Wonder if LLMs in autoreasearch loops would be able to complete tasks geohot has in mind in say 100x average token budget.
If the answer is yes, the argument doesn’t matter: you just run the loop and wait for llm analog of moore’s law to get costs down.
Loops make the code even worse. The more local the changes, the better LLMs at it.
When I started coding with AI I would copy / paste code into GPT-3.5 and ask it to update the code, it was a massive productivity boost, minor changes, fully reviewed. Then VSCode allowed tabbing, it was okayish, but I had my finger on the pulse and knew exactly what was changed and had an opinion on the suggestions. Then cursor allowed you to see and approve changes, after a few changes it started making bigger changes but had an review and approve process, things were starting to feel more magic and required more discipline to be on top of changes. Then YOLO mode hit and you could make massive changes, slowly it became easier and easier to just let the AI build code and you just guide it.
The issue is people mix up complexity, novelty, repeatability and scale.
Well documented complex problems can easily be solved by LLMs.
Doing the same thing over and over again is easy for an LLM.
Novelty and scale is very hard for an LLM.
Even small novel problems confuse LLMs.
When you start a new code base the LLM smashes through the boilerplate work. Then when it gets to scale it struggles with context rot plus novelty.
While I was reading this post, Anthropic sent me an email with the subject line "Your account has been suspended". What a coincidence :D
This post hits the nail at a bit of an angle.
The AI agents are great, and any expert can prompt them correctly to get good code. LLMs occasionally pick wrong patterns and start digging a hole, but this is why an expert is required. The code itself is just not worth writing when a detailed prompt can get you the same code typing 20x less text.
Where I agree with the post is:
The adoption of AI agents into software engineering is a problem. Solo projects are great, but our teams have not adjusted to the speed-of-change to a mental model of a project. So I see orgs making a choice to either: slow down or forgo the shared mental model.
Anybody choosing to forgo the mental model is building crooked legacy slop at scale. You can and should save the mental model to an AGENTS.md, but devs need it in their brain to prevent the digging a hole behavior.
To be fair the digging a hole behavior is something humans do just as well. But in teams you'd communicate enough to catch it - hopefully^1. It's the combination of higher speeds and teams that's creating a bit of a disaster.
I'm not sure what a good solution is either. There is a case for solo devs running for 2-month sprints with much more freedom. Perhaps we'll have an "AI Agile manifesto" within a year.
[1] Though you should not underestimate the amount of poor code being created before LLMs. There are enough teams for whom LLMs are practically all upsides. Stay very far away from those.
Thanks for the message, is exactly what I have in my brain and how I'm seeing it unfold.
I'm lucky to be part of different codebases, +200 engineers codebase in a 10 years old company and code, +5 engineers on fresh code. My personal projects, that are beyond POC's, real users, hundreds of commits.
The LLM agent sweet spot is the last one, they are perfect, as I can contain most of the knowledge in my brain of how it works in/out. Speed is insane as a solo developer.
Then the 5 engineers codebase, is also really good, but here you already start to see the problems, thanks to agents you don't even need to care how it works, I have been working on it for +6 months, it uses TRPC and I don't even know (I don't care) how TRPC works. You feel that no one in the team really knows how stuff works at 100% (fresh codebase, we have build this ourselves!!).
Then there is the old codebase with +200 engineers, this is the worst of all of them, you described it perfectly, a bottomless pit of tech debt. This codebase before agents was an old non-typescript one, it was not perfect, but you could build a mental model and understand it perfectly after a few weeks working on it. Now, is a hot-mess of code duplication and the quality is degrading faster and faster as the code gets worse and the Claude Code adoption increases within the engineering team.
Not sure what will be the outcome of all this, but I wouldn't be surprised if some company wakes up in 2027 with a codebase that maintenance and development has increased by x100 fold thanks to Agents.
‘If it hurts, stop’ vs ‘if it hurts, do more of it’. Organizations have a choice, some slow down to avoid, some speed up in hope to make issues… non-issues. If the go-fast orgs find workflows that actually truly speed things up without loss of quality, it’s like hitting the jackpot - you’ve found a way to run away from competition without them even realizing it’s possible (for a while anyway, until they notice they’re grossly outpaced).
>When people see an artifact, they make assumptions about the process that was used to create it. Without even thinking about it, they assume the creator had a basically human state of mind. This assumption is no longer true. Things can be broken in ways that weren’t previously possible, and old proxies of underlying quality like syntax and grammar are useless. AI produced artifacts are not produced by the same process as human ones, and this difference, while extremely subtle in statistics, makes itself obvious when you try to interact with and build on the artifact in human ways.
Once Humans just had oral language, and we could us words to pass ideas from one human mind to another. Then with writing ideas could pass to minds that weren't immediately close together in space or time.. and with this we made complext global spanning civilization. When words just become noise, that one has to be suspect of each one as to whither they'er coming from another human mind, or just a statistical process, can this civilization even survive?
When digital cameras replaced traditional ones, we thought it would make photography more democratic: each of us would be Helmut Newton for 15 minutes. But it didn't give us the beautiful portraits and inspired lanscapes we expected, only millions of pictures of food.
How much will it take for AI agents to pass from distilling decades of collective wisdom to copying each other's worst mistakes?
>But it didn't give us the beautiful portraits and inspired lanscapes we expected, only millions of pictures of food.
Here's a sample of my work using digital cameras, not a food picture in sight.
https://flickr.com/photos/---mike---/albums/7217772029640662...
The thing about having the ability to take effectively free photographs is that it really lets you experiment and learn the edges of what's possible.
I was inspired by Stanford's camera array, and wound up doing virtual focus synthetic aperture photography. I'm hoping to build a rig to do it on near real time, instead of the manual process I used to do on my train rides to and from work.
Sure, the removal of cost lead to a flood of the mundane, but it also means we can capture our lives in ways that even kings couldn't afford in the past. I have thousands of good photos, and even some video, of friends and family.
The comparison between AI and digital photography is a great one IMHO.
I used to be deep into photography in the 00s, until I came to the conclusion that I was spending lots of money, spending hours trying to take the perfect picture and then hours reworking them on the computer... for what?
Sure, I had fun while doing all of this, but to me art is about sharing. And given the sheer amount of pictures that were published daily on Flickr at the time, I basically had to spend more time in trying to reach out to people to share with them, than in creating something, which was my initial goal.
Or I could strong-harm my relatives, but I myself hate people that are forcing me to watch at their kids/holidays pictures, so...
And looking at the figures on your pictures, taking them at absolute value (not considering what personal value you can give them), it seems that your art is not seen by many. Contrary to thousands of random pictures of take-away than can reach thousands of eyes in a mater of hours.
Which is, to me, depressing. You put your soul in something and it has almost no reach, while random snapshots got millions of views.
And I see how the same thing will apply to software: flood of vibecoded trash that can be used by many, while deeply thought after software will be used by a few.
The good news for me here is that I never thought software as art but only a tool. I can vibecode the hell out of an issue, and the code will live on my private repos. Don't care. Sometimes I put stuff on Github but I don't care if someone (or something) uses it. I'm focussed on solving my issues.
>it seems that your art is not seen by many
Oh, there's a story there about assumptions and bad UI. I had fairly large numbers, and tried to whittle down the thousands of photos I had posted to the ones that were favorited, and in the process erased everyone elses favorite tags, leading to rage quitting Flickr for a while. It's all now a mere hint of what it was.
When a blog like this goes completely black or white on a topic I get skeptical. Nothing in life is 0 or 1. So is AI. Has some good to it and some obvious issues. All not that big of a deal. Ppl try to position themselves on the edges bc that’s what’s polarizes and engages conversation…
I do think that between luddite or "ai pilled" ai usage should be much more in favor of "ai pilled".
However, this isn't a plug to be using AI for coding everything, but a more general plug that AI should be integrated to a lot more things outside of the mainstay of chatbots.
There is a lot of merit to using AI to establish a new abstraction layer.
I don't think Geohot has a good idea about LeCun and Hutter's views on the limitations of LLMs. I think that on abstract, textual domains, LLMs perform superbly, and they would agree. I am not too well-informed about LeCun and Hutter's views either, but I think that:
LeCun thinks that LLMs are a bad fit for AI that understands the physical, dynamical systems that we inhabit, and that understanding this is necessary for AGI/ASI.
I don't know that Hutter is bearish on LLMs, but Hutter is interested in AI that can reason exceptionally well given infinite compute, and approximations of such a reasoning AI. I think he is open to the idea that LLMs can be such an approximation.
From the article:
> Without fully endorsing all their ideas, I’m now in the LeCun/Marcus camp on LLMs.
I'm pretty sure he means "Yann LeCun and Gary Marcus" not "Yann LeCun and Marcus Hutter".
I think the ai-as-an-exoskeleton analogy is quite apt: 1. It still requires your intent to move in the right direction rather than being fully autonomous, and 2. It's too big and bulky to use for delicate tasks.
Yes. I think of it as a car or tractor (which is also a kind of exoskeleton). You still need intricate knowledge; it's really an amplifier. Steer it wrong and you'll have a 1000 (very hard to detect upfront) bugs in the blink of an eye. Indeed it's hard to wield. At a minimum you need to understand your harness at the character level - the exact shape of the context should be roughly known when operating a harness.
I'm very interested in APIs that allow client-side context construction rather than relying on opaque APIs concatenating strings from your JSON messages and injecting tool prompts. I found that generally, you can craft the entire context as a unicode string and just stuff it in the system message. This works best with models where the chat template is published.
Very much agree. All these currently hyped workflows removing the human from the loop attribute a modularity to those agents that just does not hold up. They will always be leaky abstractions given their stochastic nature. That being said, they can be a great tool for getting past the "blank page" and just start or getting unstuck in general.
If nothing else, Eternal Sloptember is a term that seems obvious once you have it. I can’t believe this is the first time I’m seeing it.
If you were on Usenet before '93 the words still haunt you
Joined Usenet in 1990 and for a few years it was great.
I always wonder whether HN suffers from periodic influxes of newbies who don't get it yet and rile up the regulars.
There was the Reddit Exodus of 2023 and there seems to be a fair few new accounts posting recently, which feels like an Attempted Agentic Takeover.
It's not periodic so much as continuous. But yes, HN is constantly where Reddit was about 3 years ago and correspondingly declining. E.g. see the rise in "underrated comment" type comments lately.
I phrased it poorly, I haven’t heard the term sloptember.
Eternal September is the kind of trivia that let me know if someone is a grey beard or not.
Its one of my favorite parts of online trivia.
I was very much unborn in '93, could you share the folk history for the record?
https://en.wikipedia.org/wiki/Eternal_September
Eternal LLMber?
"Things can be broken in ways that weren’t previously possible" and also "Things can work in ways that weren’t previously possible". It all depends on what the use the tool for, if you're a carpenter you're going to do a bad job regardless if you have a fancy hammer or a basic one. If you're an expert, give them a basic hammer and they'll do the work, give them a fancy hammer and they'll do the same, perhaps a little faster (or not).
Yet the Eternal September is what made the modern AI possible. I asked Cowork for share of the corpus before / after the event, and the corpus before it is <1%, which fits my hunch.
I don't think you can go completely hands-off for quality products but you can relax and let the agent do as much as possible. It does enable things that probably wouldn't have happened otherwise.
If you are already comfortable with letting other devs work on features then it's easier, because it's similar (arguably you have more control with AI, because what you say goes regardless of hierarchy).
"When people see an artifact, they make assumptions about the process that was used to create it. Without even thinking about it, they assume the creator had a basically human state of mind. This assumption is no longer true."
I've been running into this experience with non-code artifacts, like slideshows and documents.
Didn't expect this to come from him. Seeing some of his recent YouTube streams and previous blog posts, he seemed like he has unconditionally bought into the idea of vibecoding, even as he had Opus 4.5 (latest at the time) stuck failing to enumerate a serial device for solid hours. What a turn.
> It’s definitely a better Google for most searches
I can't agree with this. You tend to get one point of view, often without any actual resources and references so you have to go look it up yourself, on [insert search engine]. Plus, what does it say when we consider an AI the one stop for our data intakes.
I find that it's typically better than Google search has been for a while, but not better than it's ever been.
More so tells me just how bad Google search has become and just how bad content in general has become.
how do you measure if google’s engineering org is more productive than meta’s? What about comparing 2 startups/small teams?
I think the discussion about methods (coding agents included) depends on answering those questions. Seems pointless to claim these agents [dont] make you more productive.
Although, at a first glance, the productivity increase does seem like nothing I’ve seen before. Even more than the transition of making webapps in plain js -> jquery -> frameworks or going from something like Flask to using Rails.
Problem is this is not evidence based. I just feel prototyping has speed up 100x. So the number of iterations/attempts has gone up. Transforming specs into a test suite takes a fraction of the time. Dunno, feels weird not to be able to be overall more productive (do more with less time) if you have these new tools.
Smart guy but whoever eventually actually fixes X search will probably use AI coding assistance to do it.
It all depends on the code quality bar. If it's high, a lot of tasks will not be completed much faster. The main speed comes from trusting LLMs output. When you review each change and reprompt LLMs to make the code look like you want. Suddenly, things become much slower and reviews/reprompts are very mentally exhausting.
I have started to think of engineers who have entirely replaced their critical thought processes with AI agents, AI proxies.
AI labs should put some incentives in their RL to make their models write shorter code so it's easier to check
The issue then becomes it writes insane one liners that are impossible to read. Would be difficult to set up an RL environment where the goal is correctness and readability.
you could have an AI judge look for correct and readable code
I wish these posts that talk about non-human mistakes that agents make would post some examples. They would be interesting to see.
One nasty set of bugs Claude recently introduced; it was doing a large refactor which involved changing call sites to conform to a changed API. Tedious, but straight forward. It helpfully added about 50 if(!something) continue; statements, this would make the code silently absorb issues that should have thrown. Had I accepted this, the results would have made the program run like shit but not crash, making debugging much harder than it needs to be. Really effing annoying! Thanks Claude!
Eh but statistical models are obviously useful, because statistically 99% of your codebase wont involve new idea invention. Tools that write all the boilerplate code used to have names and job titles.
I hate how both the for and against case for LLMs are just so bloody terrible at addressing these things.
This is a good take. The most effective combination of AI and skilled practitioner is using AI to amplify the abilities of the skilled practitioner. And in particular, max benefit comes from exploiting comparative advantage. AIs are really good at boilerplate -- in many cases better than humans because humans will optimize the process by doing copy/paste and often inject errors in the process -- whereas humans are better at abstract and critical reasoning. There's a very real and valuable use case for AI, but it's not replacing humans, it's taking the things that humans don't like doing (and that a computer can do well already) off the human's plate, so humans can focus more exclusively on the things that they do better than the AI. And at least with the current architecture of AI models, there will _always_ be higher-level reasoning that humans do better than the machine.
This. A ~staff software engineer designing big changes at one level above the raw implementation details using Opus 4.7 + superpowers today can genuinely ship multiple times more at the same quality level than pre-AI. The level of what a whole team could ship before.
You have to use something like superpowers, the key is that the humans need to make the important decisions.
You have to review the code - just like you had to review the code humans wrote. There will be iterations.
You have to give the LLM skills and patterns to follow, access to architectural documents, etc, just like humans needed to be onboarded at a company and do the same.
If you get all of these right with today's LLMs, you will never write code at all because it is so obviously not the best use of your time. If you feel that you are still better at writing the code manually, you have not done the above right, fix your workflow and try again.
I don't know why but I feel happy and relieved reading a piece of this written by Geohot himself.
It really feels like a mass psychosis. I'm not an AI sceptic insofar as I fully expect to get replaced by some future AI system. But what we have now isn't it.
To use a Geohot-inspired analogy, what we have now is like the Google self-driving car of 2010. It works most of the time, yet sometimes fails in unpredictable ways. So you need a safety driver behind the wheel to constantly watch what it's doing (the code review).
A real AI agent would not need a safety driver. We don't have that but many people are basically saying "fuck it, I'm just going to set this car off on its own and see what happens". And sure if you're prototyping it's not dangerous. But for production systems that is dangerous.
That is a fair analogy.
There is some very cool tech it just needs continued refinement, there is a path forwards even if it isn't always the clearest. This is happening but it is taking years and a lot of work to get done.
> The bottom performers won’t have that self check. They are the ones producing 10x output with the agents. What do you think is happening to the average output of that organization?
Nailed it!
At my last place this was encouraged (by non-technical leadership driving the AI adoption policies, as well as setting salaries) and seen as a huge win.
The "step change in number of created PR's" was celebrated (cult-style), and by one of the (co) CEO's praised as a paradigm shift of the same magnitude as the personal computer. Meanwhile, I was stuck finding insta-reject level bugs in pull requests from people one-shotting 6000 line PR's "finally solving" long-standing issues from the backlog. Needless to say I left.
Another problem with perception of AI tools, for coding and other things, is that people often adopt a one-size-fits-all view. If Claude/Codex whatever can fix a bug in my tiny hobby project then it's going to revolutionize all software engineering. If it can write a haiku, then it the great American novel will be dead in a few years and the novelists will starve.
There aren't many truly general purpose tools so viewing things this way seems like either a fantasy or an over-reaction. And if nothing else the processes we use will have to change along with the tools.
It's the early days so we still have a lot to figure out but one of the most significant is which tools are appropriate for what sort of tasks. I've had good luck refactoring a small code base, building some small hobby projects and building features for our company's product. But, I've also dodged bullets doing greenfield development on some features where Claude (my default) has made what seemed like sound choices early on, and which I approved of, only to build something fragile or with unforseen consequences. I haven't quite figured out what distinguished those situations from the successful ones but I'm trying. But it's complicated by the fact that things are evolving quickly and yesterday's failure mode isn't the same as today's and, for that matter, yesterday's successes aren't guaranted to be repeatable today.
On Saturday I thought I had vibe coded myself into a mess. I had implemented a new block type in my structured editor for Practal Zero (or rather let Codex do it), and suddenly the syntax highlighting broke in the whole document. Asking Codex to fix it didn't work. I was contemplating to restart the whole project on a basis that I actually fully understand, but that would set me back so much when the first reasonable prototype seemed so close. Instead, I took a walk.
See, the project actually has a well thought out structure that I design carefully, but more and more of it gets filled out by Codex. Codex is not smart enough to remember all the high-level design considerations, some of which had not been documented because I was just implicitly assuming them. So the fix was to use Codex to isolate the error, think about in terms of the high-level design, and fix the problem, which was partially an implementation problem, and partially a problem of the high-level design.
I fixed the high-level design with discussions with Codex, and documenting this, and then let Codex implement the fixes. The discussion took me more than an hour, the implementation was done in a few minutes.
This working style is similar to doing math: You have a high-level idea of what you are doing, and let that guide you, and Codex assumes the role of something that fills out all of the details you take for granted. Often it turns out your high-level idea had flaws, and this shows up in your code not working as expected. So you revise your high-level idea, refactor the code to reflect the modified high-level design, rinse and repeat.
Working this way is still really hard, but it allows me to do things I could not have done before. Getting your ideas validated (or refuted) in minutes instead of days is huge, and makes it possible to march through stuff that would have turned into a deadly swamp before, at least for me.
Now. Do I think that most corporate programmers will use Codex or CC in this way? I don't know, but I think probably not. So what will stop them going into the swamp until it swallows them, instead of backing up in time and marching around it?
To add, what also often happens in these discussions is that Codex suggests a design that makes no real sense at all, or that it brings up two or three design alternatives, and recommends exactly the wrong one.
"It’s definitely a better Google for most searches"
This is dangerously incorrect. AI summaries of search results consistently return incorrect information and grossly oversimplified and thus misleading summaries, neither of which are detectable unless one either has prior domain knowledge or spends time drilling into search results to validate the AI output.
My experience with ChatGPT as a search engine - it is totally paranoid about checking and re-checking its answers by referencing them in multiple places (I usually read its thinking output). I have not seen an outright hallucination for at least a year. (It is of course a different situation with Google's "AI summary" which is wrong half of the time.)
Ironically I quit using ChatGPT a while back. I decided to run it through it's paces and asked it some rather detailed questions about a range of topics that I have significant domain knowledge on. Without exception the responses I got back where glibly superficial to the point the responses were almost totally devoid of meaningful information. The AI summary on Google search results is so bad it represents an assault on reason.
> and it’s taking longer and longer to realize that they can’t
For something to take "longer and longer" to realise, doesn't they imply that it's been realised at least once before or that there was an expected deadline for the realisation?
Okay, that's a nitpick.
I read it as "agents can't program, and with each new generation of agents it's taking longer and longer to realize that that specific iteration can't". Maybe taking the Principle of Charity too far, I dunno.
Nah, that's fair, I was just being a bit tongue-in-cheek.
Wake me up.... when sloptember ends.
It won‘t..
It's just a tool. Use it well or use it badly - just the same as any. If you are generating slop using the tool, we'll then that is your own problem.
For me, the AI is essentially "faster hands" that can type what I am thinking way faster than I can do it. I tell it what I want, I give it the broad architecture and design patterns/types to use, and any specific test conditions, and let it write all of that usually by the time I have responded to a single email or chat message or two. Custom instructions etc build overtime to address model blind spots or my own personal taste so I don't have to repeat myself in every prompt for cross-cutting things.
Does it "one shot it"? Almost never - we go around the cycle a few times, treating it like pair programming a junior or intern by keeping a close eye on the broad direction and making sure it is acceptable - course-correcting where it matters, but cutting some slack where it doesn't. Sometimes I ask it why it picked a particular approach (that I wouldn't have necessarily) and it gives me a cogent explanation and we go with it, so I actually sometimes learn new things from it too which is great.
The other use case is just it's sheer capacity to research a codebase and hold everything in it's attention at once. It can comprehend unfamiliar code way faster and way more in-depth than I can. So if you are in an unfamiliar code base or a language or framework you are not that familiar with, it absolutely shines because it can just absorb all that info in seconds, and then you can just drill it with questions and what-abouts and how does it do this and what technique is used for that and that, what are the existing patterns and norms in this codebase when it comes to foo or bar? Etc etc
What I am not doing is deferring everything off to the AI unless it really doesn't matter (e.g. disposable one-off or prototype code). Same that I would not expect a junior or intern to make big architectural decisions when implementing something - you keep them on a fairly close leash and watch what they are up to.
Precisely. It is faster than I am at the easy part of writing the code; but the decisions about what to write (at least at a high level) are still my own.
Why sloptember when it's may
It's a reference to Eternal September, the name applied to the cultural shift of the internet as the general public started gaining access to it en masse in the 90s.
Sloptember is clearly a reference to this - the similarity being that masses of AI generated content, from social media posts to open source contributions are replacing the human internet. In a way this is related to the "dead internet theory", an idea I previously found hard to believe, but these days could easily be true.
If the history of the internet interests you, both these are worth looking up.
https://en.wikipedia.org/wiki/Eternal_September
Prediction: the author will wish the Internet Archive didn't exist in a few years.
It depends
> And whenever you need a quick prototype and don’t care about polish, it is absurdly fast. But is it a software engineer? Not close to the bar at any company I have worked at.
This line which he wrote, will override any quality gaps, because the cost to produce that shitty software will be lower than the cost to produce good software.
My point is, before LLM's 90% of the code was already human made slop, now its just going toward computer generated slop instead.
The more specific your work is, the more these LLM’s seem to struggle.
If your work was previously googling stack overflow, it can be incredibly useful at working through that. Which let’s face it, that’s what a lot of us do.
People misunderstand how AI is used in coding in normal work environments. New feature requirement comes - maybe you need a new service or some new classes. You need to do some research first.
You guide the AI with some prompts and give it some guidance on how to scenario-test it. It makes some classes, test methods. Maybe ~2000 lines and you do a quick verification, check if the overall idea looks okay. Ask it to fix a few design things and then merge it.
Its much easier than doing it yourself with all the boilerplate and understanding each esoteric language specific thing. Which library do I use for UDP communication in golang? The agent might have made a good assumption. These kind of things is where it speeds it up.
If you don’t know what library to use in your specific language, do you think you know enough to have an LLM generate most of it?
I just joined a new team and have been using copilot with opus models.
We have our core code in a weird dialect of C and rust. C I know well, but not rust. Our tests are in Python. The pipeline descriptions are in Yaml.
Outside of the core code there are so many arcana to learn. Writing syntactically and semantically correct yaml/Python test code would be a nightmare. The Agents have flaws, but they provide a huge leg up in improving the tests.
And they are great at providing a first pass review of the core code before bothering a human reviewer. Lastly we run some of our test failures through AI triage, which often enough finds the root cause or rules out simple failures.
This shows up in a higher checkin rate. I'm curious to see whether this will lead to quality end product since we have more support for the more manually written and reviewed core product code.
YES. This line of thought is exactly why people are still skeptical of LLM's.
LLM's are directionally right and if their answer "fits" then I take it at face value.
I wrote a blog detailing the computational difference between "generation" and "verification" and why it matters for LLM's: https://simianwords.bearblog.dev/the-generation-vs-verificat...
As an example: I asked the LLM "synonym for "provides" that also means "places" on you" and it gave me 5 answers and I immediately knew the right one was "confers". How? It just fits. Just like most things.
People are skeptical of LLMs because the experiences they’ve had with LLMs. You can’t blog your way out those experiences.
I’m skeptical because I’ve seen this exact situation and I’ve seen the result be something that anyone experienced wouldn’t do.
Sometimes I think folks having ‘experiences’ should play more poker.
The point is, it’s a game of chance and yet good players beat bad players in the long run. Your job in the new era of software engineering is to design the process so LLMs doing your code monkeying avoid the losses (including discarding bad changes) and take the wins. Win often enough and you’ll come out ahead.
Hmm you have written it in a way that gives me a new line of thinking.
I think what you are saying is that people should learn and appreciate working in high variance environments and still exploit small gains. This is clearly not something that is easily digestible to people so they end up rejecting LLMs.
that's a very dangerous analogy, because you would be considered the domain expert and you are just asking for synonyms for something you already know but may not remember off-hand.
now, what if you asked for the synonym for "provides" in a language that has gender differences (e.g. spanish/portuguese) as well as societal nuances (e.g. japanese) and it gives you "confers", how would you now know that's correct?
ah, so you say you tell it to take into consideration gender differences, as well as societal nuances. What are those, if you were not already familiar with the language?
You are pushing the limits of my framework and it’s a good thing.
The extent to which LLMs help is determined by how well acquainted you are with the domain. But it will always push you directionally in the right direction.
In your case, you used a language example and this is one where LLMs have natural strength in. I don’t need to be an expert in Spanish to trust it because I know that LLMs are specifically good at catching these problems.
But again there are limits and good to understand it.
Yes, just throw 2kloc over the wall for some feature. Your coworkers must love you.
My coworkers don’t care, they’re doing the exact same thing.
I also review 2kloc using Claude and approve quickly. Claude is directionally right so this sort of works out.
We all remember cryptocurrency. Everyone in tech proclaimed fiat was dead, every office buzzed with talk of every possible way that cryptocurrency could be used, billions of dollars flooded in to projects losing money hand over fist. The cynics reacted to the froth with outright rejection of the idea. And today… cryptocurrency exists, it has some use, but it didn’t take over the world, it didn’t kill fiat, it was useful in some areas and worthless in others. AI will be the same. The noisiest proponents will be over exaggerating. The most cynical cynics will be underestimating. The result will be somewhere in the middle. Success will not be predicated on adoption of the technology. We, nerds, are bad at predicting the impact of technology.
Almost nobody used crypto, everyone is using AI, every day for doing productive work and almost nobody would give it up.
I don't understand how it's remotely reasonable to try to make the comparison.
I would throw it away for everyone in an instant if it was possible.
I wish people would stop comparing AI with cryptocurrency. The hype/perception was the only thing that was similar between them. The fundamental usefulness of the respective technologies are not comparable.
Two other similarities: they both rely on burning huge amounts of electricity, and have driven up costs of GPUs around the world.
The cognitive dissonance around this is astounding.
First of all define productive. Would someone using AI to build software at a startup which is likely to fail be considered productive? What if there is already similar software available that solves the same problems? What about the broad use of LLMs to draft emails or make silly memes?
It’s funny how everyone’s concerns around climate change just disappeared when they realised AI was useful to them.
One of the reasons it gets compared with crypto is because the same people who were all in on crypto schemes are now all in on AI schemes.
Yeah, that's the hype part. Those people are creatures of hype.
That's the fallacy. You think technology usefulness dictates the outcomes. There are billions of people living in poverty the world over, starving every day, we have the technology and resources to solve that right now, we have for decades, but we don't because we don't want to.
Could AI technology change the world? Sure. Will it? That depends on so much more than what the technology can do. Why are we all still working 40 hours a week? Why are people still hungry? We could have radically changed our world with the technology we have had for decades. Yet, we have not, we have continued, nothing has really changed.
The internet is a great example. What is the most impactful part of the internet today? Social media. Social media has radically changed our culture. What is social media? A database, a few endpoints and an app? The technology is the least consequential part, the consequence comes from how we use it.
Nerds focus on what is possible with the technology, not what society is likely to do with it. What evidence is there that AI is going to change the world? What change is going to come from... being able to generate plausible sounding text? From being able to instruct agents? How many companies are using garbage software from 20 years ago despite dozens of revolutionarily better equivalents being available out there today that could have drastically reshaped their workforce? What are agents if not better macros? How many businesses have hundreds of employees doing the same tasks over and over again that could have been replaced by a few macros? How much of the code that you and I have written in our careers has already been written before?
The fundamental usefulness is the least important part of a technology when discussing how that technology will impact the world.
What "society is likely to do" with a technology drops to zero for things that technology cannot do. That's a technical property. On the other hand, people quite reliably find unexpected uses for technology, beyond the initial hype, again, due to inherent technical properties. You're vastly overcorrecting against "nerds".
It’s a matter of timeframes too.
Cryptocurrency is more popular and intertwined with the financial system than it ever was so while the claim isn’t currently true it doesn’t mean it won’t be on a long enough timeframe.
If you are old enough then you would be aware that similar claims were made about email but only one country that I know of (the Netherlands) no longer processes mail. Still if we had to guess I would say that we are still early and email will replace the worlds postal systems.
You are predicting the weather by saying tomorrow will be just like today.
Which has a good track record of being right, most of the time.
I agree!
preach it
The horse is better than a car!
The car is better than the hyperloop!
Good News! The horse and the car can coexist
Bad news! The horse population declined by 85% after the widespread introduction of the car.
Good news! Buggy/carriage mechanics became auto mechanics and the number of mechanics jobs increased overall.
Good News! I learned something today.
I wouldn’t get too excited, I got that figure from ChatGPT. Who knows if it’s correct.
> It is a golden era for buckets and buckets of slop, and a dark age for gems of quality.
I mean, this has been the trend for decades really, before LLMs were a thing. The incentive is skewed toward quantity rather than quality. The new tools just add more fuel to the fire.
Code quality is also really lacking in much of the industry. The truth is, these LLM models, as limited as they are, program at a level above that of the median junior programmer.
Bro claims to write good code. He got fired <4 weeks from twitter. AI is hyped but code isnt that bad.
spill the tea, fired or quit?
Wasn't enough room at Twitter for those two egos.
"I don’t think models like this will ever be able to program,"
I don't get how anybody who has used the SOTA models in the last 3-4 months can write a sentence like this?
They most certainly can program. And usually better than 90% of my coworkers.
The question is really.. Can they engineer? By which I mean handle the duties of a software engineer working in a team, managing a large complex system, making reviewable pieces, forward progress in incremental steps, etc.
No, that part I'm definitely more skeptical about. That requires slave driving by the person in front of the prompt.
But this is a useful distinction to make. Because making overly pessimistic claims about the coding capabilities of the models makes me question the author's experiences with them.
I think agentic tools are toxic to team programming culture and engineering that produces reliable stable results. But I wouldn't for the life of me question their ability to write programs.
There's a time and a place for assembly language programming. Of course, I knew someone who would say there's a time and a place for machine language programming (improved it by reprogramming a device by flipping the 17 switches on the front panel)
On the other hand we see success stories such as antirez using agents to work on Redis and Deepseek v4 flash inference.
To me this sounds like an old cobbler complaining that machines aren't producing good shoes if left unsupervised and that the old process of making shoes completely by hand is far superior.
So what he is telling us? That agents are not infaillable and they are not capable to one shot complex software and they do not produce perfect code?
We know what and the solution is to use agents for what they are good at and work around their limitations and we have a human in the loop.
>not some RLVR shit that comments out the failing test and tells you all the tests are now passing
That's what harnesses should be about: detect when the agent is misbehaving and force it to take the right approach.
This example in particular should be easy to solve if we generated the tests before coding and we have a workflow or state machine that doesn't allow the agent to disable tests and doesn't allow it to reach the next stage unless all tests are passing.
LLM proponents always use some language like "these old, stuck up dinosaurs with their manual labor vs us cool smart kids with automated labor", but they forget one thing - with automated labor the performance and cost difference was easily measurable in favor of the automation. With LLMs it's neither measurable nor visible (no better software, no faster delivery overall in the industry), and the costs are pretty bad. Besides personal anecdotes of someone toiling away at yet another AI harness project on GitHub.
Right now, to get some good results from AI and save time, you will have to spend a lot of tokens and money. Maybe in the future, the things will get better, I don't know.
Saving money is the wrong reason to use AI now. AI is expensive if you want good results.
But what AI is good for, is it allows you to build fast.
Also, I don't see everything being automated. To get good results you have to drive the AI.
The factories still have workers supervising the process and doing some high value manual processes even if most of the production is done by machines.
Modern shoes are made by a mix of machine and hand. There is still quite a bit of manual labor to produce shoes: https://www.youtube.com/watch?v=bK8pcAYapXQ
This is effectively what's happening to software. We are getting some forms of automation but I believe there's plenty of manual work and coordination left for humans to do.
Coders underestimate the utility of AI in so many boring day to day tasks. If you freelance, that’s where the money is at, not in creating a startup that fills holes in AI offerings or in creating generic slop while hoping for ad money.
The amount of domain specific apps that will be created will likely make excel look like yesterday’s news.
I heard that a year ago, I guess we still need to wait a bit more. Thought agents were fast!
People have been trying to replace Excel for the last 40 years...
Nah this person is dead wrong. Lets come back in 2 years and check on it. I'm willing to make a reasonable bet on these terms: companies will go even more AI native, will use even more tokens and spend even more money.
EDIT: To people downvoting me, please come up with a reasonable bet and lets try to work it out.
EDIT 2: $500 bet paid to your account on whether LLM's are going to still be used productively or not. No one?
EDIT 3: Any bet that would express the author's argument in a way that can be disproven in the future
The author is not arguing against your bet. I think the author wouldn't be surprised if you won that bet. But that doesn't change the argument he does make.
What is a bet that most charitably represents what the author would make? If there’s no such bet then it’s just whining. If there is then I’d like to know.
The author, George Hotz, owns an AI company: https://comma.ai/
He is just making a moment-in-time assessment of how he feels about current state of LLM coding. I don't think it's a super-strongly held opinion, I'm sure he would be willing to change his opinion if the next two years of LLM development produce exponentially better results.
> The _Eternal_ Sloptember
> I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history
> Agents _will_ end up hurting large organizations more than high performing individuals or small orgs
> Do you think macOS will get better or worse in the next 2 years?
> I don’t think models like this will _ever_ be able to program, I think the process matters.
From this quote especially how can you say he only thinks about the "current" state of LLM coding?
You are wrong. I don't think this is a point in time assessment. I don't think it is not supper strongly held. I do think he will change his mind though.
That's why I asking for terms of the bet.
You could both be right.
Right the original article says "Do you think macOS will get better or worse in the next 2 years?" (rhetorically implying "worse").
That could easily be true and Apple "will use even more tokens and spend even more money".
Sure but what would be a bet that can disprove the author? If there’s nothing then the post is useless.
Something that would disprove the author would be an increase in industry code quality. A reduction in bug rates might be a reasonable proxy measure.
No, I don't think people will be spending even more money on AI if it is not becoming productive. 2 years is a long time to get used to it.
You can be completely right in a bad way, and it's what I suspect is going to happen.
Three years ago I mentioned to co-workers I was most concerned about juniors not being able to build skills to become mid-level or senior. In the past year others have started talking about the same. But also I had thought people who were already mid-level or senior could resist and control themselves enough to use it well - but in the past six months two different co-workers have independently said they've noticed their own skills atrophying. And with those skills atrophied, they'll have less and less input/direction for the AI tools.
My suspicion is that for everyone who has gone all-in, a few years from now, they will still see a productivity increase from their own baseline - but their baseline will have dropped from where it is now as they "get used to it".
Geohot's next venture will be writing a book titled "Fear & Trembling".
> They are a highly sophisticated statistical model designed to mimic the distribution of programming
Are we really still doing this?
Author here. I have never said that phrase before this blog post and certainly understand the absurdity of it. I certainly don't mean that you need something biological or whatever consciousness might or might not be.
However there's still a distinction. Unless I'm responding to an LLM, you had a childhood. You learned about the world and space and agency before you ever learned how to program. And you didn't learn it from billions of examples, you learned from a few examples, some self directed experiments, some feedback from teachers, etc...
I'm saying that's what matters. The process matters. You didn't learn to mimic a distribution, you learned to program. Of course in the perfect mathematical limit it's the same, but in practice it's not.
This description falls apart for two reasons
1. It only accurately describes pre-training 2. It ignores the existence of generalization
Next token prediction is just a training task, not "what the model does internally" in any meaningful sense
For a lot (most) of what we do with programming, the process actually doesn't matter. I understand you are a real ass dude who is in this shit for the love of the game. I respect that. You are a true artisan and exist in a kind of rarified space. There will always be a place for people like you and in some senses you are correct - you are not replaceable by any AI as they currently function today.
However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
LLMs are even more useful to experts who know the limitations. However the process matters even more if you want to build robust and scalable secure systems that generate millions of dollars and can explain that accurately to high value clients.
I do not want a $10M - $100M dollar issue (lawsuits) because I admitted that I don't understand why a breach happened after using a coding agent. Responsiblity and reputation can't be vibe-coded.
So:
> However, 99.9999% of coding is not like that. Non-coders don't care about the code at all. They just care about outcomes. People don't care if it's "slop" if it works. Similar to bug prevalence, the optimal level of slop is not zero and will be decided by the market, not by coders.
There's a vast difference between code that works as a prototype vs how it works in production. I don't think you would trust anyone with no experience to fly a commercial plane with them vibe-coding a flight simulator without knowing the process of becoming a pilot.
But since "it works", it is ok right?
Yes, until everyone gets it through their head.
I'm confused what else you think they are?
Its fundamentally how LLMs work.
"Magic"
Well, since the fundamental underlying structure is still the same, yes.
It's not exactly what it is; they now model an incredibly complex markov process, and harnesses that control how that thinking is done.
Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern.
The LLM in this use case is perfect because almost everything involved is text based, and the model is able to take in all the expressive that is language.
> Is this any different than how a PM gets a programmer to work on a project?
Yes, it's very different. You seem to be suggesting that the current frontier LLMs, when tied to their tools and harnesses, have emergent properties that are similar to human consciousness. If you truly believe that, I'm not sure how to have a productive discussion here.
I think they have the capabilities to execute a well defined plan. If you truly don't believe that, then you I suspect any work you do as a programmer will not survive the coming changes.
It's not just that, but the core is just that, even with reasoning models. Harness can only get you closer to the good result, but can't save you from every pitfall. As for PM analogy - don't forget that models don't learn and keep doing same stupid stuff they were doing a month ago.
I would suggest you examine current harness memory persistence. Any reprimand you give your model will be remembered, in the same way a puppy that has a bad social experience will become more shy.
They will not save you from every pitfall, but that isn't the point; engineers walk into pitfalls all the time. This can get you in, and out, much much quicker.
Agents are perfectly capable of learning. Why would the model need to learn? The harness and tooling are all that matter.
They aren't, it's just populating context window until "memory" is pushed out.
But its not useful because even humans are like that - a bunch of neurons slapped together. Overall a tired analogy that is more suited to stay in 2024 where it belongs. Right now it is clear that it is _much_ more than a statistical model semantically. It is misleading to claim it is _just_ that just like a human is _just_ a statistical model.
Neurons? Go lower. Just atoms. Dumb, senseless atoms.
That's a very good way to describe what they do (better than 'AI') but ironically, it really well explains the mechanism, and how they are in fact able to 'code so well' which is contrary to the authors own premise.
Agents code extremely well.
They're not particularly good at 'architecture' and I think that's where his specific concerns about 'not being able to see the problems' arise - the issues are are almost never in the syntax, because the AI writes perfect code. The issue is that it's not doing exactly what you intended.
Instead of 'missing the target' ... it's 'hit the wrong target perfectly'.
Any senior developer working with AI daily should be able to have a baseline intuition for all of this, and would therefore reject the hyperbole of the premise 'it can't code!'.
Of course it's producing gargantuan amounts of slop - that's not because 'it can't code', that's something else entirely.
> Of course it's producing gargantuan amounts of slop - that's not because 'it can't code'
That is precisely because it can't code! Or rather, it's because it can't reason, or understand things, which in turn means it can't code. The output of LLMs is sloppy because they have no understanding of what they are doing.
"Airplanes don't fly, humans do!"
People are hallucinating on this thread.
That people will say 'Code Me An App' and expect some kind of magical results, will be more common than not, but it's no way evidence that the AI can't code.
Given a sufficiently detailed prompt, the AI will produce almost whatever you ask it within a certain scale.
As sure as the sky is blue.
And it will make perfectly compilable code usually on the first prompt.
Obviously, it can code.
Obviously, it can 'synthetically reason' about the code.
You can point it an arbitrary code base and it will give a better overall assessment than most humans.
Is it fallible? Obviously. Is it limited in scope? Obviously.
Yes, the beatings will continue until morale improves!
It's the mantra of the AI skeptics. Sounds so clever because it's technically true. Just like humans are just piles of oxygen, hydrogen, nitrogen and carbon atoms, along with maybe a dozen or so other trace elements, none of which have any intelligence or will or desire to do anything - hence humans cannot possibly be intelligent or have any free will.
I don't think LeCun is saying they won't be able to program. I think he says we won't hit AGI. Programming does not require AGI; it's a pretty specific skill!
-- I think this article is COPE, if I'm being quite honest. I thought of putting cute analogies, like the C programmers saying the Python and Javascript programmers are not "hardcore" enough... but the truth should be obvious to anyone using LLMs effectively.
-- Current AI is a much better programmer than 100% of people and when directed by someone in that top 10%, it's a force majeur.
> it's a force majeur
I assume you meant something like 'force multiplier"? Force Majeure is an uncontrollable event that prevents a party from fulfilling a contract. Which some may argue is what AI will also deliver. :)
This is my experience. Though ice been writing LLM harnesses, agents, tooling, etc for 5 years now and believe it requires several hundred hours of experience before understanding how to consistently outperform at scale.
A problem that's impossible to detect is indistinguishable from it working. Hence it works. Hence it's not a problem.
" Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming"
In other words - they can program, and probably better than you.
I don't like being too critical but this is a really superficial post - as if either 'AI is a Software Engineer - or - It must be Fraud'
It's an extremely powerful tool that is very 'pattern oriented' and with guidance can absolutely write great code - and even across modules given the right basis.
It's also great at so many other tasks - finding bugs in big code bases, doing migrations etc.
It's not going to make very goo architectural decisions for you, and if you're doing anything novel you have to read most of the code ... but that's too be expected.
It’s not like the author is a noob.
https://en.wikipedia.org/wiki/George_Hotz
In fact, he’s done several things that are truly hard, and has a well-deserved engineering reputation.
Well he doesn't write CRUD apps, which plenty of us do, and with a decent harness, agents can do decent enough work on them.
The author is absurdly wrong.
It's ridiculous to suggest that 'AI can't code' - when the entire development world has moved into agentic coding, including all of the best developers in the world, and it's yielding positive results in most scenarios.
It's a callow 'bad twitter take' the length of an article.
He's not wrong to suggest that IA is a 'stochastic mechanism' over all the code that's ever been written, but that's evidence of the mechanism, and frankly, describe how it is able to code.
And yes - organizations will misappropriate AI at scale as they do with everything.
His premise is so far out of proportion and misguided, it's tantamount to 'fake moon landing' conspiracy theory.
> when the entire development world has moved into agentic coding
Careful, your bubble is showing.
“By 2005 or so, it will become clear that the Internet's impact on the economy has been no greater than the fax machine.”