Yes, and: the most important time to use your brain isn't in the loop.
That _is_ valuable, and will remain so, but the temptation to turn it off is there because the loop is good and getting better at what it does. Once you're in the loop, with the rare high-value exception of catching total mistakes nad redirecting, you're mostly choosing between similar-yet-reasonable options. This isn't so much right-vs-wrong as relativistic optimization. Letting the loop do the work will get you a mediocre result quickly, and that's usually fine.
The most important time to use your brain is _before the prompt_. Once you engage with your LLM and agent, you start biasing yourself, and its reasonable suggestions constrain your visibility into other options, other worlds.
I don't agree. I'm recently a huge believer in Programming as Theory Building. The software creation itself is the part that makes you understand what you are building. It's impossible to understand things before you start building them, initial prompt can help you explore, but the idea that after that you can lean back is ridiculous. Most or maybe even all successful software projects are still heavily relying on the best software engineers. That's why anthropic and others are still hiring SDEs.
I somewhat agree with your point, but also I find that the major agent harnesses work really hard at tricking you into leaning back and not letting a proper understanding of what you are building to develop.
This isn't a new problem. I had it when working with outsource developers, too. In both cases, it seemed that just sitting back and describing what I wanted in plain English prevented me from properly wrapping my head around the problem. Which then meant I failed to develop the insight needed to find the really good improvements. Those usually consisted of identifying and removing the bits that were making things worse.[1] It's incredibly difficult for me to think of anything other than just adding more stuff on top of what's already there if I'm not getting my own hands dirty.
So I've been moving away from plan mode and letting the spin away on building large chunks, and back toward working in small increments and typing out the important code with my own hands. That seems to be the sweet spot. I'm still getting a productivity boost because there's more than enough boilerplate and trivial function definitions to farm out. (Easy prompt, too: "Implement `someFunctionStub`.") But I'm also not letting Plan Mode drag me back into the waterfall development pit of insanity.
Good designs tend to have simple mental models. Some of these mental models are generic and apply across many projects. Or are even written into standards. I wouldn’t be too dogmatic about impossibility of understanding before building. Sometimes knowledge beats experience, and having an intuition for when can save a lot of time.
In broad strokes, I agree. You still have to make decisions and apply taste. It's _extremely_ unlikely that you'll have the full spec and shape and architecture of something large fully settled in your head before you start.
It's also unlikely that you're going to make those realizations, expressions, and decisions when your harness is asking you to choose between what _it_ identifies as being important decisions. If you're in the loop you're getting dumped on with lots of stuff that barely differentiates and you're biased away from catching things that matter.
If you're in the loop, use the loop. It's not a terrible construct. It will give you a mediocre outcome. And if mediocre is sufficient, you're done.
And here's the part that didn't make it into my prior comment -- if mediocre isn't sufficient, _stop the loop_. Play with what you've got. See how it behaves. Explore the edge cases yourself. Sketch how it works on paper, and how you want it to work, and think about how that should be constructed. Use your brain.
Then write a really long prompt (maybe referencing your files and notes and assets) to get back into the loop again.
I guess what I'm ultimately getting at is that the loop is useful _and_ a mind-killer, and using your mind is _also_ high value, so do both those things, but don't do them at the same time.
That's great, but you and your family will go hungry as a result. If not this year, next year.
Your stance isn't new, if it's any comfort. I heard (and probably said) exactly the same thing when C compilers started getting good enough to eliminate the need for most assembly coding.
So it is your opinion that in the future llms will take extremely vague specs and turn them into fully functional software that actually works the way the prompter wanted it to?
Does the prompter sit there with the llm all day answering the 1000s of questions necessary to make what he wanted, and the prompter then spent days testing it to make sure what he initially thought he wanted is actually what he wanted?
Then for every additional feature and integration the prompter will do this again?
Are we going to make a new job title for that? Should we call that job a programmer?
Tools are tools. Better tools let us do more faster, but they are still tools.
That temptation is akin to giving in to maladaptive (day)dreaming. The problem is the dreaming, the noticing, the getting annoyed, and the feeling drained - all of these are happening at one place - being done by one entity - your brain. So how to train it to not to do that? Well, again, that same brain has to do it. What I am saying here is - our brains are extremely good at getting "used to". Because it "feels good" to the brain - at least in that instant. So any work that's happening on its own, feels good for most people. More the time passed in that "got/getting used to" state unchecked, more difficult it is to try and bring it back to not getting used to. Or control it, in an attempt to strike a fine balance.
It doesn't help that this is being forced from above. I agree you can't turn off your brain, but any time I argue that AI isn't the right tool for something management gets angry. There is a constant push to fully automate things, regardless of what those things are.
Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.
Suppose I do something myself. I reap all the benefits, but it takes a lot of
time, I may not have the expertise to do a good job, and I get all the blame if
something goes wrong.
Suppose I go to the park and find one of several guys who would do it for free.
I get all the of benefits, he might do an excellent job, but, because there's no
contract, I still take all the blame.
Suppose I actually hire a professional to do it. I receive fewer of the
benefits. However, he's much more likely to do a good job, and the legal system
provides a variety of ways to hold him accountable for the work.
Suppose I hire a professional, but then insist that he use legions of guys from
the park to do the actual work, since that's going to be some much faster and
cheaper per quantum of work. I also tell him that if he's unwilling to do that,
I'll find someone who will. I also tell him that, since he has so much more
unoccupied time now that the original work is in the hands of all those other dudes,
he gets to supervise still more work done by other swarms of guys from the park.
At this point it should be clear that I'm not interested in quality work, but in
having someone poised to absorb blame. I want a patsy.
Now replace "guys at the park" with LLMs, and replace "free" with "nearly free,
but with no warranty."
Good on you. I absolutely hate reading the output from Opus. It's disrespectful to make people read it, and to share AI output without reading and understanding it first.
> Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.
Some would tell you that you need to find a different job with a better boss, but I'd say that this is quickly becoming something very hard to escape from. Investors want line to go up, and they all talk to each other about this "AI" thing that lets them cut staff and keep productivity, so naturally we're all getting it crammed down our throats.
Finding a better boss is like picking up the good cop. There are none. The whole industry has gone completely bonkers, and bosses are the ones doing it. The problem is systemic and if there are any good bosses out there, then there are systems in place to ensure that their ideals and behavior are not allowed to stand.
Yup. This is coming from the top, and it's very much because the AI sales pitch that executives bought into is, "Automation can help you reduce head count."
That means that they really don't care if, right there in the moment, just doing the job is faster and better. What they care about is that the actual work is being done by humans, and that's out of line with their grand vision of getting rid of humans.
Same.
My team lead sits near me.
He keeps checking if I'm using claude, or when we talk he pushes me to use claude.
It's have gotten to the point that I wanted to work on something manually, so I booked a room in the office. Just to not be at my desk, because I'm afraid my claude usage will be reflected in my performance review.
I spend a lot of time encouraging my manager and coworkers to ... write some actual english. To not just point an AI at a PR and call it a day. To actually collaborate before we jump into something - leaning on the knowledge we've built instead of being a cowboy and assuming you know it all (you don't, not yet).
In many ways it feels like we're back in some awful version of 90's based software dev. All these amazing tools have got us stuck deep in the shit swamp again.
Even pre-LLM: I've found a lot of colleagues (mystifyingly to me, both seniors and juniors) who really want to be able to turn their brain off when doing investigations into complicated cross-system behavior (feedback loops abound), and they don't like hearing that it can't be collapsed into a runbook. If a drinking bird (https://en.wikipedia.org/wiki/Drinking_bird) on an Enter key could do it, I could rig something up so I wouldn't need you!
To be fair, that is often a reasonable tradeoff when there’s higher priority things the business needs to focus on.
I sometimes suggest timeboxing investigations for this reason. Spend 4h on this - if root cause is not found, just throw more hardware at it. Although I expect a writeup of what was investigated and ruled out, so that it can be used when someone returns to it.
Yes, one of the primary dysfunctions I see in business environments is a culture that doesn’t create space for timeboxed and clear discussion of tradeoffs. Everyone must have both the _appetite for_ and _implicit or explicit permission to have_ a reasonable discussion of option A vs option B and the long vs short term costs and benefits of each. There must be enough discipline to not rabbit hole or scope creep and decisions must be captured somewhere in a concise way so that everyone who is impacted is informed, and everyone who will inherit the downstream consequences can understand and articulate the decision chain that led to current state.
This is rarely if ever done in a sensible way. Instead you find Managers, Engineers or PMs avoiding troublesome stakeholders, hallway convos or Slack DMs that are never properly written into public record in a searchable/readable way, last minute overrides from higher ups based on limited context of the problem space or their own unaligned career goals, poor recordkeeping, panic mode bandaid fixes when systems break, etc...
IMO a lot of the “meat proxy” and “loop obsession” behavior we’re seeing is simply people using the sense of short term productivity gains as self-soothing against anxiety from corporate dysfunction that they have no control over.
I've seen it too, and it seems to be getting worse as companies move more and more communication onto Slack, ticketing systems, Confluence, etc. Modern channels are a huge high velocity dumping ground. The only way anyone can even pretend to keep up with all the - frankly - spam is arbitrary sampling.
Oldschool periodic reports and memoranda weren't perfect, but they did encourage taking time to curate one's message. And gave others somewhere to go for a properly curated message.
I think a lot of people are not able to reason to this point and then adjust behaviour accordingly. They probably reason to the point when pressed for it, but cannot resist the low effort and low resistance route of asking AI, and then probably make up reasons why in their case it's fine (typical behaviour to address cognitive dissonance).
Then again, for many people their job security doesn't depend on what work they do, it depends on how well they are embedded in the organisation. Before AI you also already had plenty of people doing their job at a questionable level, yet never seem to get fired.
Work in support for most of your life and you come to the conclusion that there are a whole lot of meat proxies out there. In fact everyone does the meat proxy somewhere in their life.
> I started seeing people turn off their brain as they use LLMs
I mean, this was true for computers. "Computer says no"[0] was thing long, long before LLMs. I've had well educated, seemingly very intelligent people break when they couldn't log in steadfastly ignoring the [caps lock key is on] error message. This becomes even more "fun" and "interesting" when a person is being a meat proxy for some horrifically dangerous industrial process involving high pressures, excited chemicals, and enough rotational inertia to send god flying.
I don't doubt using an LLM is deskilling. The question is whether it's the kind of deskilling where you can't farm if you don't know how crops work or the kind of deskilling where you can drive without knowing how to maintain your own internal combustion engine.
(In my lifetime, I've even watched mass cellular communication move "replacing your own tire" from skilled to deskilled, since AAA is always one phone call away for most drivers).
I'm starting to thing AI is the baseline of cognition and it matches the worst parts of humanity. Its just, we trained on the better parts so it's not so obvious.
I think you missed the point entierly. The problem is not people turning their brains off, the problem is brains wony be needed anymore in the near future, not even meat proxy brains
You are a photographer from the olden age. Taking a photo back then took forever. You are standing on a hill trying to get the perfect picture of a sunset. You measure and sort and calculate angles and light. A tourist stands next to you with an high-speed iphone auto-settings camera. They spin around and get thousands of photos from different angles while you are thinking and measuring your one shot. That’s what AI does. It gives you many shots at the same problem. The thinking guy with the one approach will likely lose a photo competition to one of the accidentally better shots of the person who tried a lot more. Make and filter now may be a better approach than overthink and try only once.
I don't buy this analogy and I think it takes a depressingly reductive view of creativity. In both cases, the photographers had to take the time to go to a place, see a view they thought was worth capturing, operated their equipment to get a shot, and then had enough discernment to recognize which of their final products were worth something. Both the traditional photographer and the modern photographer can create images of value in different ways.
If you want to shoehorn AI into this analogy at all then no one is even bothering going hiking at this point, let alone seeing anything worth capturing. It would just vomit up endless permutations of statistically average images that look like the kind of pictures people take of landscapes.
Well yeah and then you can be liberated by the fact that maybe getting to the landscape and experiencing it and not taking the photo was the goal all along.
You can be creative and take the photo for your sake but at the same time accept that you can generate 1000s of them and many of the generated ones will be more liked by the general public than your creative one.
However, for writing software with AI the analogy still works. Just as the photographer needs to pick a shot worth capturing, you have to figure out software worth building. That often also involves some leg work: you need to understand the problem and user needs. Then you need discernment to figure out where the software works well, and which parts need improvements
There used to be no such thing as a woodworker. It used to be a number of different professions, such as the jointer, the cabinet maker, the planer, etc. now with modern tools it’s merged into one job
In the past few could afford to be photographers due to time and money. Many were not the most artistic or talented minds in the universe on the subject (you can’t find those until you give everyone a camera) but just people who enjoyed it and could make money off of it. Those who gained experience with the old tech were essentially AI models trained on statistically good probabilities of getting good shot. You had smaller selection and the model / brain quality determined what % of
the best that photo was between 0 and 100% for any sunset. In modern days more people get a camera - more people with bad taste and more people with good taste. You also get more attempts at the shot - more photos taken, more tries. More tries means you are now preserving more of that moment from more angles and extending the time your taste people have to pick the best shot of all possible.
Your plethora of statistically average photos happen if you don’t have anyone filtering the ginormously increased output. If the problem is really valuable, someone will filter it. Possibly a model.
The real filter is value, not capability anymore. Do you have the empathy and taste to know what matters? The AI tokens/cost lets everyone make and become interested in making of someone hasn’t solved their problem. And then if search isn’t improved - everyone will just repeat the making of the same thing over and over. So if you don’t want repeated slop. fix search.
Or do you copy everything and everyone into making more of nothing.
The credulous LLM users Dan Luu is complaining about don't take the equivalent of thousands of photos of the same sunset. They take a couple photos at most, say "LGTM", and move on to taking a photo of something else, even if their couple of photos of the sunset aren't great.
As someone who will often compare different things, I think doing that usually takes time. I think LLMs can help reduce the time (especially if it's programming, not necessarily so much in other areas), but I'm not seeing many people use LLMs to try a large number of independent approaches and compare them.
The tourist has another advantage over your photographer: experience. They have thousands of photographs to look at and thus get a better scene of what makes things good. That is if you took both to the same spot and gave them one photograph (the phone's memory is almost full) the tourist is likely to get the better shot because they have a better idea of when to pull the trigger having seen all the possibilities.
Really? The one who canonically doesn't consider lighting, depth of field, composition, etc... I can only speak from my own experience, but on average a typical person who doesn't consider themselves a photographer isn't going through the old photos on their phone and analyzing the quality and technique for future endeavors. They just want to send a fun picture to their friends and family. Nothing wrong with that, but it won't cultivate skill or lead to award-winning photography.
They don't think of those terms. However they generally are looking at their photos and so get a sense of what "looks nice" - which is what those terms are all getting at.
Now if you know and analyze your photos in those terms you can get good faster. There is no shortcut to the value of quantity.
What we have never done here is define how much effort the "professional" has put into getting better. I've known a lot over the years that know the terms and have the "best" equipment, but they just take a measurement and put it into the tools without understanding the whole and so take really bad pictures. They consider composition as a checklist - which works okay (only okay) in a studio when it means put the chair here - but they can't handle a sunset where they can't control the scene.
Award winning photography means get those details perfect. If you have the details right a phone is often more than good enough to take the right shot - and if it can take more photos faster than is worth a lot more than the best equipment limited to one shot (or a few) and hope that you get hit the shutter at the right moment. If you can get the right moment the better equipment is better, but that is one of the least important parts of the shot.
I am sure the tourist can take hundreds or thousands of photos, and have every single one of them turn out wrong.
It's not as if he was covering every combination possible of exposition, composition, depth of field, shutter speed etc. The tourist doesn't even know the existence of those settings.
I mean there are some obvious examples of a photo just being all black or all white. There are things so blurry that you can't identify the subject matter. But in between these extremes exist matters of preference and taste and everything goes subjective as hell. There are very few rights and wrongs here.
> someone clueless about photography just shooting hundreds of photos?
That doesn't exist. The very act of taking hundreds of photos will give anybody a clue. They won't be able to pass a written test - they will have no clue what terms like "depth of field" mean - but they will start to apply the concepts in their photos without knowing the terms.
I suppose there is someone who takes hundreds of photos without ever looking at them and thus never starts to develop any sense of what is good. Likewise some people just have more taste than others and so will get better. However in the real world people are taking lots of photos and they are on the way becoming reasonably good.
Training alone will not make you a good photographer. Someone already skilled at this can help you look at your photos and understand why it "looks bad". However it takes hundreds - perhaps thousands of photos before you become an award winning photographer in the best case. While someone without training will need thousands of even tens of thousands - but these days a photo on an iphone is cheap enough that getting a thousand photos is not hard.
What is more interesting to me, the most beautiful photo of a sunset taken by an iPhone ever, or a mundane quality photo of my daughter taking her first steps?
I really don't think so. A pro that knows all the ins and outs of working their camera for the perfect shot in the given conditions will do better than a one-off hope that the camera auto settings nail it. A professional literally spends their life framing shots, a tourist spams the shutter hoping one turns out and crops/filters/AI enhances their way from the best of the bunch to whatever will get likes from their family. No contest for a single exposure: the pro knows what they're doing, the tourist doesn't.
> The thinking guy with the one approach will likely lose a photo competition to one of the accidentally better shots of the person who tried a lot more.
If that were the case then photo competitions would have been upended by now by tourists who accidentally took masterpieces with their phones.
I can take as many thousands of pictures as I want but, without taking the time to critically review them to see what works and what doesn't, I will rarely produce a prize-worthy picture. And then there's the risk that I'll discard it anyway because I don't have the eye to separate the prize-winning picture from the Instagram shot all my contacts are publishing.
I think this analogy works with respect to stuff like coding, engineering, etc, where the AI is capable of validating the success of its own efforts and where the result just has to "work", regardless of its elegance or aesthetic quality.
I also think the analogy works if we're talking about average photographs, or even very good photographs, that are conventionally well-composed, balanced, etc, but lack the special spark that we associate with true art.
But if the photographer on the hill is a real artist, it's hard for me to imagine a current frontier LLM winning the contest. I suppose it may stumble upon a winning photo sooner than a roomful of monkeys would, but I believe the number of attempts would still be so vast that it would be more efficient for the human to make one single attempt at taking a photo, rather than dealing with prompts and sorting through the millions of AI-generated options.
EDIT: I also think it's very easy to underestimate the size of the nearly limitless set of possible "solutions" to the photograph, and also easy to overlook the constant feedback loop in the artist's mind as he considers each option and looks through the viewfinder, while the LLM is shooting blind the entire time.
Even in the late 19th century "bracketing" was already a photographic technique where a photographer would take several shots of the same subject from various angles and with various settings. Digital cameras may make this faster and cheaper, but the one who employs thoughtful practices in their photography is far more likely to win that competition than the one who just (to paraphrase your example) spins to win.
Maybe so, but for the good shots by the thinking guy, I suspect he would derive longer term emotional gratification from how well they came out. Effort and investment adds to the experience.
Making things easy is good as long as it is not too easy, then it's Spotify muzak.
Except it sometimes shows you a photo of something completely different than what you're looking at for or alters the landscape to "improve" it. Maybe that's not a good analogy.
Who is this "we're" you're talking about, because I see shitloads of powerseeking people that would build a god in a heartbeat if they could stand on its right-hand side.
This ignores the reality that many people are, unfortunately, taking that one iPhone picture, sending it to everyone in their contacts list as if it’s art, and then moving on to the next slop shoot.
Once upon a time, in Switzerland, there was a "chronometry contest" putting the best watchmakers of Switzerland against each other to make the most accurate mechanical watch and have it certified by the observatory of Neuchâtel (observatories were at the time the entities in charge of giving the most accurate time, based on reference star observations).
It worked for a time, and manufactures employed experienced, legendary even, watchmakers and specialized "watch setters" to create and tune to the best possible extent a few experimental watch movements. They would be sold as collection pieces, the firm would gain boasting rights in its advertising, and the experience informed the design and tuning of more mainline movements.
By the end of the history of (the revival* of) this concours, the winner was always Tissot, a second-rate company. What hides behind Tissot is ETA, a huge watch movement manufacturer for all of the Swatch group (dozens of brands, from Omega to the worst no-name stuff you can find).
What they were doing to win was just to pick the best watch movements out of the millions of basic, bland, uninteresting "workhorse" ones they were producing each year. Just by chance, there was always one combination of the hundreds of components, each with their manufacturing tolerances and variations, that would fit all together to make an exceptionally accurate movement.
In the end, yes, Tissot won repeatedly. But the concours is dead. And Tissot is still a second-rate brand making second rate watches. Nobody is interested by their watch movements, no collector, no historian, no customer.
And how do you get a person able to nail "that one shot" when the moment is fleeting? Your "tourist" is unable to do it, because he never put any deliberate practice in photography.
Incidentally, analog film photography is having a huge revival, and Instax is still Fujifilm's best seller. Maybe there is something missing from "I did an award-winning photo but I have have no clue how, lol".
Is there anybody here who wants to have their portrait taken by the "tourist"? Is there anyone here who would prompt an AI to generate their portrait, apart from Ghibli-style remixes (which are funny five minutes, and stale a month later)?
Have even more fun with the instax UP!™ app.
With the introduction of AI, you can now scan an instax™ photo regardless of the background or angle, or even in your hand.
Incidentally, the memory price increase is going to hit hard the "hybrid" cameras - the ones who take a digital picture and then print it out. So the pure analog one may increase its predominance over the "hybrid" one.
Well now the phones actually take a video over a period of time so you can pick the best shot.
But a better analogy is how we hired a guy to take photos of us in front of the Eiffel tower on Christmas day. He came with a DSLR, directed the different poses, then sent us the photos later that day. Whole thing took less than 20 minutes.
It's the fact that the floor has been lowered so substantially.
Of course every modernized, automated craft ends up having its hobbyist revival. Whether it's analog photography, hand tool woodworking, knitting, assembly programming, or whatever. And that's lovely, but it doesn't mean much for economic trends.
That's an interesting analogy, because there's still a spectrum. Photography's big advancement was the move to digital. Prior to that, you had to know what an F stop is, different developer chemicals, manual focus. These days, yes, an iPhone is actually quite good at taking pictures. Apple invested billions for that to happen, it wasn't an accident. SLR cameras are still there though, you still have wedding photographers. You don't have the mall photographer for once a year photoshoots so much anymore, however. Everyone just takes pictures everywhere as they need. Software is the same now. I need a program to do a thing, I'll just have AI whip one up instead of looking for software that does something that but has 100 more features I don't need. Gone are the days where customers would come to the mall to get photos done and buy software. (Though that world's been gone for a while.) Software is now only a means to an end, so better find the customers that have needs and embed with them instead of having them look for Photoshop.
Airline pilots and train conductors are basically meat proxies 95% of the time and they are still employed. Still necessary for when things really go wrong.
I'm interpreting this as justification for sysadmins making a comeback, which I would absolutely love. We lost so much as a society when they were taken behind the shed.
Train conductors should be automated, we have better things to spend government money on than make work jobs. LIRR pays conductors 200k. It’s theft at that point
Train conductors are valuable IF they have first aid and CPR training; maybe riot training. There are probably a few other things along those lines as well - but none of them have anything to do with running a train: all the train related tasks can be better done automated.
There are still some things in an airplane that an autopilot cannot do. There is nothing on a train in normal operation (as opposed to maintenance) that a computer cannot do - as evidenced by multiple trains around the world running without humans operators since the 1990s!
The long island economy is split between well paid public servants, NYC commuters, and the circular economy of driveway contractors and tile contractors in an arms race for the nicest walking surface.
There are no other jobs on the island that pay decently. Cutting edge engineering work will make you half of what it will in either the DC or Boston areas.
The conductors are highly paid in part because it's the local custom, and part because a third of the economy would collapse if that train stops running.
There are much worse things the government spends money on (or doesn't tax) than well paying jobs that play an important role in society. What do you think happens to that $200k? Do you think it just gets sent to the void? That money is spent on things like food, local businesses, rec sports and trips for family... and all that money gets spent again by those local businesses.
Meanwhile Boeing did enough stock buybacks from 2014 to 2019 to give every single boeing employee $200k... and when the 737 max crashes happened, the stock crashed back to pre buyback levels. THAT is truly sending money to the void to be lit on fire, money that could of been used to change the lives of thousands of employees and factory workers.
If screwing up gets me killed I want the person doing that job to be paid well and I want them to be held up to a high performance standard. I don't see how cutting pay for conductors is going to make anyone's lives better.
Or if we replace that well-compensated train conductor with automation I sure hope it's high quality stuff and not lowest-bidder slop.
> I don't see how cutting pay for conductors is going to make anyone's lives better
It's going to make passengers' lives better by making their train rides more affordable, allowing them to improve their lives by leveraging those funds.
Or, it's going to make shareholders' lives better by increasing their profits.
Either way, some people will benefit from it. The question is -- what is the real, practical tradeoff, and is the tradeoff worth it to the participants in the system.
Money, as a limited resource, means that money not spent on the conductor's wages can be spent elsewhere. In this hypothetical, fully automated trains can be run more frequently, so the money not spent on conductor wages means that the train comes every 5 minutes instead of every 15, and more stations.
It's going to make shareholder's lives better. And you better believe we're replacing conductors with lowest-bidder slop, that's the whole point. That's shareholder capitalism's playbook, it hasn't changed for the last 50 years.
What, you don't want to save a few bucks on a train ticket? Come on, you'll love it!
An AI system cannot replace professional people. It doesn’t replicate or do the work that they do. That’s not what it is for and it is incapable of performing these tasks.
If you are using these systems you still need to know what you want and how to specify it. You can use agents to automate much of the work but it can’t tell you that your ideas are bad, that there’s a better way to accomplish your goals; it can’t tell you what you want. And if you can’t understand what it is outputting you’re failing in your primary task as a developer: understanding the problem.
I agree with the title. I just don’t think these systems will ever be capable of replacing humans. As much as product managers and executives would love it and not have to deal with us; they’ll just be shooting themselves in the foot if they try to replace professional developers with AI harnesses and fire all the people with the institutional knowledge and skill.
I think you may need to update your priors a bit. AI is already replacing professionals. Developers are already losing jobs. LLMs and agents are already running in the hot loops of our technological civilization, unsupervised, with root. Institutional knowledge and skills are already being lost.
AI isn’t replacing people. Managers and executives have been sold a product that they believe they can use to replace people. Some of them probably still hold some enthusiasm for these products. But there seem to be some that are getting wise to the fact that they’ve been duped.
Yeah this is the closest I've found to explaining my own discomfort. Why I've been almost entirely an individual contributor my whole career and why I may opt-out and retire.
I have limited tolerance to take demeaning workload that I know "the boss" could do himself. I think my pride and satisfaction is in my intellectual contribution, not my ability to relay messages nor do repetitive slop work.
But the real revulsion is being a scapegoat. I will not rubber stamp and take responsibility for things not under my control. I have limited capacity to review and sign off work I've delegated to someone else. It is usually easier for me to do it myself and know it inside-out than to delegate and then review enough to have the requisite confidence in the result.
When the new plan is to increase throughput via delegation to these generators, I can't possibly review to my standards. So, I refuse to accept any of it as my work.
It also freaks me out a bit to think of the collective mess that is developing across society, with so many people apparently ready and willing to shovel and rubber-stamp. I think the "move fast and break everything" cult is going too far.
But, primarily, I just need to protect my own sanity and live to my own standards. And hope that I can back away far enough to avoid some of the splash damage when entropy catches up...
Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee
I would argue that there's still an important role for human employees as compliance ablative armor.
The human's purpose is to keep an eye on the LLM, occasionally intervene/interfere with its work, and most importantly, incur social/legal liability for anything that the LLM does wrong.
Kind of like the backup drivers used by self-driving cars.
Is being compliance ablative armor uplifting, fulfilling work? Is there any way in which your personal competence and effort impacts whether the AI screws up or not? Do you have any way to build job security as ablative armor?
Fulfilling work disappeared a long time ago, the book bringing this up was from 2018 before chatbots were really a thing. We are just experiencing the acceleration to all jobs being bullshit jobs.
The version of "meat proxy" I've heard is the person who forwards a machine-answer to someone else who didn't ask for a machine answer, and could just as easily made one themselves.
Calling someone a "meat proxy" for being the human in the loop is wild.
I don't think that's quite the problem the OP is pointing out. A corollary of "meat proxy" is "cognitive surrender." [1] In both cases, blindly pasting AI output and being an "Enter key presser", you have surrendered your cognition. In the latter case, you're in the loop but there's not a lot of reason for you to be.
Your underlying supposition is right, though. If you are truly thinking and in the loop, you are by definition not a meat proxy. I think those are the people who are successful with AI coding tools, but it takes discipline and it's still hard to find the joy and reward when you're not the one solving the problems.
I think it might be appropriate if what we're talking about is a human connecting an LLM to something it can't natively access. Like something in the physical world - a piece of hardware, a different human, software the LLM can't access. As the human in the loop your role might be to mostly just shuffle information back and forth from this external resource. But depending on the situation there could still be quite a lot of interpretation and analysis performed by the human in the loop.
The problem is, that digital work, the new frontier of employment for a lot of epeople, got and will continue to get A LOT cheaper, faster and more accessable.
The PO who always complains about someone? They will just vibe stuff. It might even be shit but it will work good enough until LLMs/AI/AGI/ASI is good enough or better anyway than an avg developer.
Man i have seen so so much shitty code even in software companies.
You know when your HR software takes seconds to load? Yeah an LLM can do that today too
I've noticed some tasks (80-90%) can be accomplished by quickly typing out a feature/bug fix, accepting plans with ~10% brain power, and moving on. But there are other tasks where the LLMs struggle, and it requires significantly more time and mental effort to think through the architecture and requirements before it's to a point where the LLMs can process it.
What's difficult, is I now spend most of my time on these high-throughput tasks, where my brain isn't exactly off, but focused on quantity over quality. It's often difficult to identify when a task requires "real" brain-power, and when it does, it adds a lot of friction to dig into the code and orient myself. So it's hard to know when it's the right time to do that.
Turning off your brain with LLMs is how you get fired.
Even with small things, you need to be at the wheel. The other day I had to stop copilot from literally deleting the file I was working on. Why did it choose to go down that route? I don't know, but I nixed and reported the chat once it did that.
LLMs are dangerous and if for some reason you are stuck with using them, do NOT ever turn your brain off.
Turning your brain off is usually a defense mechanism to protect yourself against something bad. dissociate during trauma etc. lighter dissociation points to the interaction trending toward badness, not goodness. Seems like the mind and body is telling us something that we shouldn't ignore
No, it's usually an instinctive power-saving mechanism, and it happens incredibly frequently.
For example, the average driving commuter is not "zoning out" because of some bad experience with roads or cars, but because the route is extremely familiar.
I assume you're not saying it is but I just wanna call out that zoning out behind the wheel isn't a good thing. Drivers should be attentive at all times regardless of whether the route is familiar. So maybe something is going wrong there, even if it's not trauma?
One can still drive attentively but be on a route so familiar that unless something unusual happens, the brain does not bother recording any specific memory of it.
Even at work, I can spend a week on developing something, implement it, and once it's done will have almost no memory of it a month later because it was just one more feature in a long series of features that are all pretty much the same.
The human brain is a filter that seeks novelty. All the repetitious junk gets compressed down to something like key frames. This is why things like alert fatigue are so dangerous. Any signal that only very occasionally means danger will eventually be ignored.
Yeah, I experience this a lot. I call it “zombie mode.” It’s like pulling a slot machine, though. Sometimes AI can go a few turns just with encouragement. But when I don’t maintain conceptual engagement, drift is inevitable.
My solution is saying “gnite” and coming back in a new terminal window with a clear restatement of my purpose. Or “what are you working on and why” can help.
Terminal windows are not great for keeping context; waiting for better design. It is possible.
The more I see posts like this, the more I realize that the problems people have with AI are no different than the problems we had with outsourcing (development to low-cost countries) 20 years ago. The results and challenges are the same; except now the feedback loop is within minutes instead of overnight.
I think this is the dissonance that has been making me feel insecure / depressed around the state of AI coding. On the one hand I am way more productive. But on the other hand it is almost requiring more of my brain than before, because I am having to determine what is good output and what is slop. But that's not the part of my brain that I particularly like engaging — I like understanding something deeply. But in this case, that is just not possible. I like to think that as the human-in-the-loop, I am supplying the requisite _taste_ to the design and overall structure of the program. But at what point does my taste get taken over by the LLM as models improve?
Another problem is that when something breaks then I am very limited in my ability to reason through the breakage. It's necessary to go back to the LLM and ask it why it broke. At which point, I'm able to follow the reasoning — but not in any durable manner where I'll be able to remember it. To me, it's sort of like the difference between memorizing a passage and having a ready reference at hand at all time. We had this same problem with information before — now it's come for our reasoning.
> What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee.
Company is pleased! Cost savings!
Alas, the joy doesn't last long as pretty soon after the company realizes that after an LLM replaces almost all their employees, the much easier task of replacing the entire company begins, because the company itself has just become a much thinner meat proxy for getting an LLM to do whatever it is the customer relies on the company for.
Customer is pleased! Cost savings!
Alas, the joy doesn't last long as whatever savings they have from replacing the company with their own LLM loop is offset by whatever it is they do to earn income being similarly impacted.
All of the moats degrade together though they may fully give out at slightly different speeds.
> Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee.
If this is true, turning off the brain... is completely rational. Why burn out the brain for something happening either way? It's not like being fully engaged is going to result in anything that much better for the employee. In a world where code is this disposable, memorizing a codebase that won't recognizably exist in 3 years is also stupid.
I’m not allowed to enter legit chess tournaments where if I’m going be using my phone’s chess engine app to tell me the best move in every single position. Because that’s considered cheating by the chess players and tournament organizers and rightly so. But, there doesn’t seem to be an incentive to not do something similar in the world of work unless your LLM output is so bad or obviously inaccurate.
The challenge here is that the business world doesn't really recognize the concept of cheating.
In particular, tech is very open to people cheating and if billions of dollars could be had by somehow communicating the chess engine's work to a human at chess, the world of business would cheat at it.
Tech has a number of large startups that basically started with the assumption of "what if we assumed the law had no teeth?"
I’ve pondered this too. I was in a group meeting with a consultant who is upfront with clients that he has started to use Claude heavily to speed up his output. There are probably a lot of clients who would still pay for his services because maybe they still appreciate his input or the relationship and don’t want to go through the pain of figuring out how to replace him with an LLM. It has been my experience that some teams would rather just subscribe to some software or platform even if there’s a free alternative.
> Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee.
Why do you think management is so gung-ho about LLMs? Employees are expensive, and laying them off is therefore extremely appealing.
The entire, explicitly stated goal of the AI movement is to produce an ASI, which is better across every metric than a human. Even if you refuse to turn off your brain, the economics of paying you to keep the lights on upstairs aren't going to work out.
This was never going to work out well for the human.
I'm not sure what else to say. You listened to these people speak, what the heck did you expect?
There are a lot of programmers who are working with chatbots and watching YouTube simultaneously for the first time, simply matching the rest of the economy for the nature of work.
If you can do your job while watching TV your brain is turned off.
Maybe, but the parts of my brain I'm still using are a lot easier to use.
I was dreaming to be promoted to a cushy management role where I could talk bs for a living.
I managed to attain 'hands-on team lead' and later an ill-fated 'bootstrapped cryptocurrency CTO' role... Nobody wanted to pay me to talk as a career. Now they do!
So I'm very happy about that.
I feel contempt for all the people who had it easy and 'chose' to be engineers voluntarily.
he was writing in reaction to the McCarthy era which, arguably, is very similar to kind of bombastic nativism we're seeing now from our political elites. his argument essentially being that there are cultural and institutionalized forces that are driving people to, well, turn their brains off - specifically to no longer be skeptical of what they consume, to not want to refine and improve their toolbox of analysis, to not want to understand how things work for the sake of knowing
and that these acts were deliberate - a populace is so much easier to control when it's complacent and unknowing, who believes, unskeptically, that Oceania has always been at war with Eastasia, unquestioning about who was writing the PR missives that feed the news cycle (see also: https://www.commondreams.org/opinion/what-is-copaganda), how verbatim they are, and what the real intent of the authors were
like, I get that you don't want to put in effort for a job you don't like. but that effort also makes you better at your work. it makes it easier on you, it lets you create less incidents, it empowers you to be more agent in your own life, to, who knows, have the guts to start a consulting gig because your skill-sets actually are better than the average
there's so many benefits from just being more diligent, longterm, proactive benefits. but that's not everybody's extrinsic motivator and it's so much easier to just be Skinner-boxed by social media companies into doomscrolling, I suppose
Here's the thing I'm finding - LLMs allow me to do more coding when my brain isn't firing on all six cylinders. It means I get more code written because I don't have to be "in the zone" to exhaustively hunt down every piece of API and dependency and have my flow continuously broken up by missing information. The code written in that state is lower quality, but I can patch up five-cylinder-brain code faster than I can synthesize code from a blank page outside of my flow state.
When throughput is an issue, you have to think judiciously. I'm definitely not saying you shouldn't think but you have to apply human oversight effectively. For me that means front-loading through dialectic planning, and maximizing guardrails.
Turning my brain off actually works really well when I'm doing something more creative, and while I know this isn't exactly what he's talking about, I wonder if we are just offloading things we shouldn't have to think about to have more space to think about other things.
Of course, this becomes a lot more difficult when you're unemployed.
There's some software that really doesn't require my brain. I gave Astra an ssh key for root on a little Pi Zero with a microphone attached and asked it to make recordings of a siren going off in our neighbourhood, based on a couple of sirens. This would have been a great deal of work a couple of years ago, now it's a few tokens, and honestly, it doesn't really matter if it's shitty code, I just want the timestamps.
Then there's the software that I write for a living, and almost my entire day is now spent making hard decisions. /grilling gives me difficult technical design decisions endlessly and I am forever doing product management on features from agents. My brain is back in CTO mode, which isn't what I'd planned for myself after I'd left that particular cursed path, but fine. I am doing the opposite of turning my brain off, because I no longer can just chill out and mindlessly write code.
Both these states can exist simultaneously. But my brain work has shifted into something different, while getting no less intense.
Developers like to "turn their brain off" during "super hard debuging" sessions. Mental flow state is a well documented as pleasurable experience. It is like a drug!
Auditing and reviewing autogenerated commits, is hard gruelling mental labour, compared to that. You do not even get a credit as an author! You can not get into groove to get your fix!
”Nowhere is this clearer than in entry-level software engineering—though the picture depends on whom you ask. Brynjolfsson discovered that employment among developers aged 22 to 25 has fallen nearly 20% from its late-2022 peak. The online job site Indeed also paints a stark picture: software development job postings have fallen 53% from the same starting point.”
They aren’t firing the meat proxies to that degree yet, but they absolutely have stopped hiring the meat proxies. It’s going to be a very dirty year on Wallstreet when you’ll see massive sneaky layoffs of the existing meat proxies.
You can turn your brain off with these models. It is what it is.
I think there's just too many confounding variables to say. ZIRP policy ended in 2022, with interest rates going up 5% between 2022 and mid-2023[1]. This both means that money was historically cheap when these early-career developers were being hired in the peak, and it was relatively (in terms of the past couple of decades; eg, the modern tech industry) expensive after.
We've seen layoffs as a result of this, and nobody's hiring a cohort of juniors when they're trying to afford the employees they already have.
On top of that, companies are firing the money cannon at AI initiatives and experiments (some of which will be successful and others which won't), leaving less money for humans.
Anyway, it's just a single occurrence of a hiring drop in a single economic cycle. Unfortunately, it just happens to perfectly line up with the rise of GenAI. We may never be able to tease apart cause and effect in this period, and we certainly won't be able to do so until we have at least a handful more years of data to see how junior hiring does or does not rebound.
”Unfortunately, it just happens to perfectly line up with the rise of GenAI.”
Right. That’s just something I think we should keep our pulse on and keep looking at this data every six months (seriously, this needs to be tracked by devs because this data may start forming into that proverbial “writing on the wall”).
No one should be planning on being a software engineer in about 3-5 years, all need to plan for alternative income sources or early retirement. I think, at least.
>No one should be planning on being a software engineer in about 3-5 years, all need to plan for alternative income sources or early retirement. I think, at least.
Interesting. I recall Geoffrey Hinton saying something similar about Radiologists.
“People should stop training radiologists now” "it is just completely obvious that within 5 years deep learning will do better than radiologists”
Yes, and: the most important time to use your brain isn't in the loop.
That _is_ valuable, and will remain so, but the temptation to turn it off is there because the loop is good and getting better at what it does. Once you're in the loop, with the rare high-value exception of catching total mistakes nad redirecting, you're mostly choosing between similar-yet-reasonable options. This isn't so much right-vs-wrong as relativistic optimization. Letting the loop do the work will get you a mediocre result quickly, and that's usually fine.
The most important time to use your brain is _before the prompt_. Once you engage with your LLM and agent, you start biasing yourself, and its reasonable suggestions constrain your visibility into other options, other worlds.
First, use your brain. Then write the prompt.
I don't agree. I'm recently a huge believer in Programming as Theory Building. The software creation itself is the part that makes you understand what you are building. It's impossible to understand things before you start building them, initial prompt can help you explore, but the idea that after that you can lean back is ridiculous. Most or maybe even all successful software projects are still heavily relying on the best software engineers. That's why anthropic and others are still hiring SDEs.
I somewhat agree with your point, but also I find that the major agent harnesses work really hard at tricking you into leaning back and not letting a proper understanding of what you are building to develop.
This isn't a new problem. I had it when working with outsource developers, too. In both cases, it seemed that just sitting back and describing what I wanted in plain English prevented me from properly wrapping my head around the problem. Which then meant I failed to develop the insight needed to find the really good improvements. Those usually consisted of identifying and removing the bits that were making things worse.[1] It's incredibly difficult for me to think of anything other than just adding more stuff on top of what's already there if I'm not getting my own hands dirty.
So I've been moving away from plan mode and letting the spin away on building large chunks, and back toward working in small increments and typing out the important code with my own hands. That seems to be the sweet spot. I'm still getting a productivity boost because there's more than enough boilerplate and trivial function definitions to farm out. (Easy prompt, too: "Implement `someFunctionStub`.") But I'm also not letting Plan Mode drag me back into the waterfall development pit of insanity.
1: https://en.wikipedia.org/wiki/No_Silver_Bullet
Good designs tend to have simple mental models. Some of these mental models are generic and apply across many projects. Or are even written into standards. I wouldn’t be too dogmatic about impossibility of understanding before building. Sometimes knowledge beats experience, and having an intuition for when can save a lot of time.
In broad strokes, I agree. You still have to make decisions and apply taste. It's _extremely_ unlikely that you'll have the full spec and shape and architecture of something large fully settled in your head before you start.
It's also unlikely that you're going to make those realizations, expressions, and decisions when your harness is asking you to choose between what _it_ identifies as being important decisions. If you're in the loop you're getting dumped on with lots of stuff that barely differentiates and you're biased away from catching things that matter.
If you're in the loop, use the loop. It's not a terrible construct. It will give you a mediocre outcome. And if mediocre is sufficient, you're done.
And here's the part that didn't make it into my prior comment -- if mediocre isn't sufficient, _stop the loop_. Play with what you've got. See how it behaves. Explore the edge cases yourself. Sketch how it works on paper, and how you want it to work, and think about how that should be constructed. Use your brain.
Then write a really long prompt (maybe referencing your files and notes and assets) to get back into the loop again.
I guess what I'm ultimately getting at is that the loop is useful _and_ a mind-killer, and using your mind is _also_ high value, so do both those things, but don't do them at the same time.
There are certain parts of my job that can be mediocre and I'm happy to offload them to models. Those are, however, not the parts that count.
That's great, but you and your family will go hungry as a result. If not this year, next year.
Your stance isn't new, if it's any comfort. I heard (and probably said) exactly the same thing when C compilers started getting good enough to eliminate the need for most assembly coding.
I will take my chances. Software is part of my identity and I won't abandon it that quickly.
Understanding assembly coding makes you a vastly better C coder.
Nah, not really. Not anymore. Nice skill to have when debugging weird stuff, but nothing you can't get by without.
So it is your opinion that in the future llms will take extremely vague specs and turn them into fully functional software that actually works the way the prompter wanted it to?
Does the prompter sit there with the llm all day answering the 1000s of questions necessary to make what he wanted, and the prompter then spent days testing it to make sure what he initially thought he wanted is actually what he wanted?
Then for every additional feature and integration the prompter will do this again?
Are we going to make a new job title for that? Should we call that job a programmer?
Tools are tools. Better tools let us do more faster, but they are still tools.
Are we going to make a new job title for that? Should we call that job a programmer?
Works for me! The most popular programming languages in 2030 will be English and Mandarin. You don't have to like it, you just have to deal with it.
That temptation is akin to giving in to maladaptive (day)dreaming. The problem is the dreaming, the noticing, the getting annoyed, and the feeling drained - all of these are happening at one place - being done by one entity - your brain. So how to train it to not to do that? Well, again, that same brain has to do it. What I am saying here is - our brains are extremely good at getting "used to". Because it "feels good" to the brain - at least in that instant. So any work that's happening on its own, feels good for most people. More the time passed in that "got/getting used to" state unchecked, more difficult it is to try and bring it back to not getting used to. Or control it, in an attempt to strike a fine balance.
It doesn't help that this is being forced from above. I agree you can't turn off your brain, but any time I argue that AI isn't the right tool for something management gets angry. There is a constant push to fully automate things, regardless of what those things are.
Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.
Suppose I do something myself. I reap all the benefits, but it takes a lot of time, I may not have the expertise to do a good job, and I get all the blame if something goes wrong. Suppose I go to the park and find one of several guys who would do it for free. I get all the of benefits, he might do an excellent job, but, because there's no contract, I still take all the blame.
Suppose I actually hire a professional to do it. I receive fewer of the benefits. However, he's much more likely to do a good job, and the legal system provides a variety of ways to hold him accountable for the work.
Suppose I hire a professional, but then insist that he use legions of guys from the park to do the actual work, since that's going to be some much faster and cheaper per quantum of work. I also tell him that if he's unwilling to do that, I'll find someone who will. I also tell him that, since he has so much more unoccupied time now that the original work is in the hands of all those other dudes, he gets to supervise still more work done by other swarms of guys from the park.
At this point it should be clear that I'm not interested in quality work, but in having someone poised to absorb blame. I want a patsy.
Now replace "guys at the park" with LLMs, and replace "free" with "nearly free, but with no warranty."
A “moral crumple zone”: https://www.researchgate.net/publication/351054898_Moral_Cru...
They're getting mad because they want you to build your replacement
My company is doing this, too. It’s the reason I’m submitting my resignation notice next week.
I just did. Good luck my brother/sister.
Good on you. I absolutely hate reading the output from Opus. It's disrespectful to make people read it, and to share AI output without reading and understanding it first.
Opus in particular seems to somehow combine pretentious intellectualism with sheer idiocy.
> Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.
Some would tell you that you need to find a different job with a better boss, but I'd say that this is quickly becoming something very hard to escape from. Investors want line to go up, and they all talk to each other about this "AI" thing that lets them cut staff and keep productivity, so naturally we're all getting it crammed down our throats.
Finding a better boss is like picking up the good cop. There are none. The whole industry has gone completely bonkers, and bosses are the ones doing it. The problem is systemic and if there are any good bosses out there, then there are systems in place to ensure that their ideals and behavior are not allowed to stand.
Yup. This is coming from the top, and it's very much because the AI sales pitch that executives bought into is, "Automation can help you reduce head count."
That means that they really don't care if, right there in the moment, just doing the job is faster and better. What they care about is that the actual work is being done by humans, and that's out of line with their grand vision of getting rid of humans.
Same. My team lead sits near me. He keeps checking if I'm using claude, or when we talk he pushes me to use claude.
It's have gotten to the point that I wanted to work on something manually, so I booked a room in the office. Just to not be at my desk, because I'm afraid my claude usage will be reflected in my performance review.
This is BONKERS.
I spend a lot of time encouraging my manager and coworkers to ... write some actual english. To not just point an AI at a PR and call it a day. To actually collaborate before we jump into something - leaning on the knowledge we've built instead of being a cowboy and assuming you know it all (you don't, not yet).
In many ways it feels like we're back in some awful version of 90's based software dev. All these amazing tools have got us stuck deep in the shit swamp again.
Also: I would hate working with your team lead.
Not a big fan of him,but he was a dev before and it was a nightmare working with him as a dev.
If the only metrics your managers have on your Claude usage is visual inspection/conversational, I think you'll be ok...
I don't know how they eval me. Maybe they eval usage and maybe they don't. I just know I feel uncomfortable to sit next to him sometimes
Even pre-LLM: I've found a lot of colleagues (mystifyingly to me, both seniors and juniors) who really want to be able to turn their brain off when doing investigations into complicated cross-system behavior (feedback loops abound), and they don't like hearing that it can't be collapsed into a runbook. If a drinking bird (https://en.wikipedia.org/wiki/Drinking_bird) on an Enter key could do it, I could rig something up so I wouldn't need you!
Or not trying to actually finding a root cause. Look i increased the storage for customer, problem solved.
No? You solved the symptom not the issue?
To be fair, that is often a reasonable tradeoff when there’s higher priority things the business needs to focus on.
I sometimes suggest timeboxing investigations for this reason. Spend 4h on this - if root cause is not found, just throw more hardware at it. Although I expect a writeup of what was investigated and ruled out, so that it can be used when someone returns to it.
Yes, one of the primary dysfunctions I see in business environments is a culture that doesn’t create space for timeboxed and clear discussion of tradeoffs. Everyone must have both the _appetite for_ and _implicit or explicit permission to have_ a reasonable discussion of option A vs option B and the long vs short term costs and benefits of each. There must be enough discipline to not rabbit hole or scope creep and decisions must be captured somewhere in a concise way so that everyone who is impacted is informed, and everyone who will inherit the downstream consequences can understand and articulate the decision chain that led to current state.
This is rarely if ever done in a sensible way. Instead you find Managers, Engineers or PMs avoiding troublesome stakeholders, hallway convos or Slack DMs that are never properly written into public record in a searchable/readable way, last minute overrides from higher ups based on limited context of the problem space or their own unaligned career goals, poor recordkeeping, panic mode bandaid fixes when systems break, etc...
IMO a lot of the “meat proxy” and “loop obsession” behavior we’re seeing is simply people using the sense of short term productivity gains as self-soothing against anxiety from corporate dysfunction that they have no control over.
I've seen it too, and it seems to be getting worse as companies move more and more communication onto Slack, ticketing systems, Confluence, etc. Modern channels are a huge high velocity dumping ground. The only way anyone can even pretend to keep up with all the - frankly - spam is arbitrary sampling.
Oldschool periodic reports and memoranda weren't perfect, but they did encourage taking time to curate one's message. And gave others somewhere to go for a properly curated message.
I think a lot of people are not able to reason to this point and then adjust behaviour accordingly. They probably reason to the point when pressed for it, but cannot resist the low effort and low resistance route of asking AI, and then probably make up reasons why in their case it's fine (typical behaviour to address cognitive dissonance).
Then again, for many people their job security doesn't depend on what work they do, it depends on how well they are embedded in the organisation. Before AI you also already had plenty of people doing their job at a questionable level, yet never seem to get fired.
Combining your thoughts, one could almost think that being a meat proxy works for quite a long time, before institutional inertia catches up.
Sounds like a quiet few years and then effort once demanded. So I guess... being a meat proxy does work, after all?
Work in support for most of your life and you come to the conclusion that there are a whole lot of meat proxies out there. In fact everyone does the meat proxy somewhere in their life.
> I started seeing people turn off their brain as they use LLMs
I mean, this was true for computers. "Computer says no"[0] was thing long, long before LLMs. I've had well educated, seemingly very intelligent people break when they couldn't log in steadfastly ignoring the [caps lock key is on] error message. This becomes even more "fun" and "interesting" when a person is being a meat proxy for some horrifically dangerous industrial process involving high pressures, excited chemicals, and enough rotational inertia to send god flying.
[0] https://en.wikipedia.org/wiki/Computer_says_no
I don't doubt using an LLM is deskilling. The question is whether it's the kind of deskilling where you can't farm if you don't know how crops work or the kind of deskilling where you can drive without knowing how to maintain your own internal combustion engine.
(In my lifetime, I've even watched mass cellular communication move "replacing your own tire" from skilled to deskilled, since AAA is always one phone call away for most drivers).
You can farm without knowing how crops work and drive without knowing how engines work
I'm starting to thing AI is the baseline of cognition and it matches the worst parts of humanity. Its just, we trained on the better parts so it's not so obvious.
I think you missed the point entierly. The problem is not people turning their brains off, the problem is brains wony be needed anymore in the near future, not even meat proxy brains
You are a photographer from the olden age. Taking a photo back then took forever. You are standing on a hill trying to get the perfect picture of a sunset. You measure and sort and calculate angles and light. A tourist stands next to you with an high-speed iphone auto-settings camera. They spin around and get thousands of photos from different angles while you are thinking and measuring your one shot. That’s what AI does. It gives you many shots at the same problem. The thinking guy with the one approach will likely lose a photo competition to one of the accidentally better shots of the person who tried a lot more. Make and filter now may be a better approach than overthink and try only once.
I don't buy this analogy and I think it takes a depressingly reductive view of creativity. In both cases, the photographers had to take the time to go to a place, see a view they thought was worth capturing, operated their equipment to get a shot, and then had enough discernment to recognize which of their final products were worth something. Both the traditional photographer and the modern photographer can create images of value in different ways.
If you want to shoehorn AI into this analogy at all then no one is even bothering going hiking at this point, let alone seeing anything worth capturing. It would just vomit up endless permutations of statistically average images that look like the kind of pictures people take of landscapes.
Well yeah and then you can be liberated by the fact that maybe getting to the landscape and experiencing it and not taking the photo was the goal all along.
You can be creative and take the photo for your sake but at the same time accept that you can generate 1000s of them and many of the generated ones will be more liked by the general public than your creative one.
However, for writing software with AI the analogy still works. Just as the photographer needs to pick a shot worth capturing, you have to figure out software worth building. That often also involves some leg work: you need to understand the problem and user needs. Then you need discernment to figure out where the software works well, and which parts need improvements
There used to be no such thing as a woodworker. It used to be a number of different professions, such as the jointer, the cabinet maker, the planer, etc. now with modern tools it’s merged into one job
But surely the reasons for that change are not "the invention of a free army of unpaid minterns."
Ex: Much-more furniture being sold are duplicates from a factory process; There are better tools—which do not substitute for executive function.
Here is why it works in more than one way:
In the past few could afford to be photographers due to time and money. Many were not the most artistic or talented minds in the universe on the subject (you can’t find those until you give everyone a camera) but just people who enjoyed it and could make money off of it. Those who gained experience with the old tech were essentially AI models trained on statistically good probabilities of getting good shot. You had smaller selection and the model / brain quality determined what % of the best that photo was between 0 and 100% for any sunset. In modern days more people get a camera - more people with bad taste and more people with good taste. You also get more attempts at the shot - more photos taken, more tries. More tries means you are now preserving more of that moment from more angles and extending the time your taste people have to pick the best shot of all possible.
Your plethora of statistically average photos happen if you don’t have anyone filtering the ginormously increased output. If the problem is really valuable, someone will filter it. Possibly a model.
The real filter is value, not capability anymore. Do you have the empathy and taste to know what matters? The AI tokens/cost lets everyone make and become interested in making of someone hasn’t solved their problem. And then if search isn’t improved - everyone will just repeat the making of the same thing over and over. So if you don’t want repeated slop. fix search.
Or do you copy everything and everyone into making more of nothing.
"Those who gained experience with the old tech were essentially AI models trained on statistically good probabilities of getting good shot"
I am so glad I am not you. What a miserable way to live your life.
This analogy seems strained to me.
The credulous LLM users Dan Luu is complaining about don't take the equivalent of thousands of photos of the same sunset. They take a couple photos at most, say "LGTM", and move on to taking a photo of something else, even if their couple of photos of the sunset aren't great.
As someone who will often compare different things, I think doing that usually takes time. I think LLMs can help reduce the time (especially if it's programming, not necessarily so much in other areas), but I'm not seeing many people use LLMs to try a large number of independent approaches and compare them.
The tourist has another advantage over your photographer: experience. They have thousands of photographs to look at and thus get a better scene of what makes things good. That is if you took both to the same spot and gave them one photograph (the phone's memory is almost full) the tourist is likely to get the better shot because they have a better idea of when to pull the trigger having seen all the possibilities.
Really? The one who canonically doesn't consider lighting, depth of field, composition, etc... I can only speak from my own experience, but on average a typical person who doesn't consider themselves a photographer isn't going through the old photos on their phone and analyzing the quality and technique for future endeavors. They just want to send a fun picture to their friends and family. Nothing wrong with that, but it won't cultivate skill or lead to award-winning photography.
They don't think of those terms. However they generally are looking at their photos and so get a sense of what "looks nice" - which is what those terms are all getting at.
Now if you know and analyze your photos in those terms you can get good faster. There is no shortcut to the value of quantity.
What we have never done here is define how much effort the "professional" has put into getting better. I've known a lot over the years that know the terms and have the "best" equipment, but they just take a measurement and put it into the tools without understanding the whole and so take really bad pictures. They consider composition as a checklist - which works okay (only okay) in a studio when it means put the chair here - but they can't handle a sunset where they can't control the scene.
Award winning photography means get those details perfect. If you have the details right a phone is often more than good enough to take the right shot - and if it can take more photos faster than is worth a lot more than the best equipment limited to one shot (or a few) and hope that you get hit the shutter at the right moment. If you can get the right moment the better equipment is better, but that is one of the least important parts of the shot.
I am sure the tourist can take hundreds or thousands of photos, and have every single one of them turn out wrong.
It's not as if he was covering every combination possible of exposition, composition, depth of field, shutter speed etc. The tourist doesn't even know the existence of those settings.
>them turn out wrong.
Define wrong in this case.
I mean there are some obvious examples of a photo just being all black or all white. There are things so blurry that you can't identify the subject matter. But in between these extremes exist matters of preference and taste and everything goes subjective as hell. There are very few rights and wrongs here.
Look at the iPhone photography awards:
https://ippawards.com/winners/showcase
How many of these pictures do you think were just a lucky shot by someone clueless about photography just shooting hundreds of photos?
How many of these pictures could even be the result of just shooting at random thousands of photos?
> someone clueless about photography just shooting hundreds of photos?
That doesn't exist. The very act of taking hundreds of photos will give anybody a clue. They won't be able to pass a written test - they will have no clue what terms like "depth of field" mean - but they will start to apply the concepts in their photos without knowing the terms.
I suppose there is someone who takes hundreds of photos without ever looking at them and thus never starts to develop any sense of what is good. Likewise some people just have more taste than others and so will get better. However in the real world people are taking lots of photos and they are on the way becoming reasonably good.
Training alone will not make you a good photographer. Someone already skilled at this can help you look at your photos and understand why it "looks bad". However it takes hundreds - perhaps thousands of photos before you become an award winning photographer in the best case. While someone without training will need thousands of even tens of thousands - but these days a photo on an iphone is cheap enough that getting a thousand photos is not hard.
"look at this list of subjective things"
What is more interesting to me, the most beautiful photo of a sunset taken by an iPhone ever, or a mundane quality photo of my daughter taking her first steps?
This conversation is about who would take an award winning photo. It's not about how much you personally like photography.
Looking in the conversation chain I'm not seeing that point...
I really don't think so. A pro that knows all the ins and outs of working their camera for the perfect shot in the given conditions will do better than a one-off hope that the camera auto settings nail it. A professional literally spends their life framing shots, a tourist spams the shutter hoping one turns out and crops/filters/AI enhances their way from the best of the bunch to whatever will get likes from their family. No contest for a single exposure: the pro knows what they're doing, the tourist doesn't.
> The thinking guy with the one approach will likely lose a photo competition to one of the accidentally better shots of the person who tried a lot more.
If that were the case then photo competitions would have been upended by now by tourists who accidentally took masterpieces with their phones.
I can take as many thousands of pictures as I want but, without taking the time to critically review them to see what works and what doesn't, I will rarely produce a prize-worthy picture. And then there's the risk that I'll discard it anyway because I don't have the eye to separate the prize-winning picture from the Instagram shot all my contacts are publishing.
I think this analogy works with respect to stuff like coding, engineering, etc, where the AI is capable of validating the success of its own efforts and where the result just has to "work", regardless of its elegance or aesthetic quality.
I also think the analogy works if we're talking about average photographs, or even very good photographs, that are conventionally well-composed, balanced, etc, but lack the special spark that we associate with true art.
But if the photographer on the hill is a real artist, it's hard for me to imagine a current frontier LLM winning the contest. I suppose it may stumble upon a winning photo sooner than a roomful of monkeys would, but I believe the number of attempts would still be so vast that it would be more efficient for the human to make one single attempt at taking a photo, rather than dealing with prompts and sorting through the millions of AI-generated options.
EDIT: I also think it's very easy to underestimate the size of the nearly limitless set of possible "solutions" to the photograph, and also easy to overlook the constant feedback loop in the artist's mind as he considers each option and looks through the viewfinder, while the LLM is shooting blind the entire time.
Even in the late 19th century "bracketing" was already a photographic technique where a photographer would take several shots of the same subject from various angles and with various settings. Digital cameras may make this faster and cheaper, but the one who employs thoughtful practices in their photography is far more likely to win that competition than the one who just (to paraphrase your example) spins to win.
Maybe so, but for the good shots by the thinking guy, I suspect he would derive longer term emotional gratification from how well they came out. Effort and investment adds to the experience.
Making things easy is good as long as it is not too easy, then it's Spotify muzak.
The real question is who can sell their photos and for how much profit. Even muzak has its market of satisfied customers.
Of course "manual" programming can always be a hobby that provides emotional gratification, but that's not what dominates the economy.
>That’s what AI does.
Except it sometimes shows you a photo of something completely different than what you're looking at for or alters the landscape to "improve" it. Maybe that's not a good analogy.
Those are the equivalent of bad photos. The point is that you can easily look at it and just try again.
You are a god. You step on the puny human and create a thousand sunsets. Maybe the photographer wasn't needed in the first place.
Hell yea, I want this power.
Sorry, we're not working to make people better, we're busy trying to build a god to stand over us.
Who is this "we're" you're talking about, because I see shitloads of powerseeking people that would build a god in a heartbeat if they could stand on its right-hand side.
Humanity is a rather fragmented bunch.
This ignores the reality that many people are, unfortunately, taking that one iPhone picture, sending it to everyone in their contacts list as if it’s art, and then moving on to the next slop shoot.
And like, that's transformed human interaction? I like following my friends' stories and receiving quotidian snaps of my nephews.
That's the point of the analogy, not that people are passing it off "as if it's art."
Once upon a time, in Switzerland, there was a "chronometry contest" putting the best watchmakers of Switzerland against each other to make the most accurate mechanical watch and have it certified by the observatory of Neuchâtel (observatories were at the time the entities in charge of giving the most accurate time, based on reference star observations).
It worked for a time, and manufactures employed experienced, legendary even, watchmakers and specialized "watch setters" to create and tune to the best possible extent a few experimental watch movements. They would be sold as collection pieces, the firm would gain boasting rights in its advertising, and the experience informed the design and tuning of more mainline movements.
By the end of the history of (the revival* of) this concours, the winner was always Tissot, a second-rate company. What hides behind Tissot is ETA, a huge watch movement manufacturer for all of the Swatch group (dozens of brands, from Omega to the worst no-name stuff you can find).
What they were doing to win was just to pick the best watch movements out of the millions of basic, bland, uninteresting "workhorse" ones they were producing each year. Just by chance, there was always one combination of the hundreds of components, each with their manufacturing tolerances and variations, that would fit all together to make an exceptionally accurate movement.
In the end, yes, Tissot won repeatedly. But the concours is dead. And Tissot is still a second-rate brand making second rate watches. Nobody is interested by their watch movements, no collector, no historian, no customer.
And how do you get a person able to nail "that one shot" when the moment is fleeting? Your "tourist" is unable to do it, because he never put any deliberate practice in photography.
Incidentally, analog film photography is having a huge revival, and Instax is still Fujifilm's best seller. Maybe there is something missing from "I did an award-winning photo but I have have no clue how, lol".
Is there anybody here who wants to have their portrait taken by the "tourist"? Is there anyone here who would prompt an AI to generate their portrait, apart from Ghibli-style remixes (which are funny five minutes, and stale a month later)?
Fuji also sells an Instax printer so you can "print your smartphone images in seconds."
https://www.instax.com/printer/
Also the AI powered instax Up! app...
Have even more fun with the instax UP!™ app. With the introduction of AI, you can now scan an instax™ photo regardless of the background or angle, or even in your hand.
Yes, Fuji tries to cover all use cases and user demands. But the one getting increasingly successful is the pure analog one:
https://www.digitalcameraworld.com/cameras/instant-cameras/f...
https://www.digitalcameraworld.com/cameras/instant-cameras/6...
Incidentally, the memory price increase is going to hit hard the "hybrid" cameras - the ones who take a digital picture and then print it out. So the pure analog one may increase its predominance over the "hybrid" one.
https://www.digitalcameraworld.com/cameras/fujifilm-is-break...
Well now the phones actually take a video over a period of time so you can pick the best shot.
But a better analogy is how we hired a guy to take photos of us in front of the Eiffel tower on Christmas day. He came with a DSLR, directed the different poses, then sent us the photos later that day. Whole thing took less than 20 minutes.
It's the fact that the floor has been lowered so substantially.
Of course every modernized, automated craft ends up having its hobbyist revival. Whether it's analog photography, hand tool woodworking, knitting, assembly programming, or whatever. And that's lovely, but it doesn't mean much for economic trends.
That's an interesting analogy, because there's still a spectrum. Photography's big advancement was the move to digital. Prior to that, you had to know what an F stop is, different developer chemicals, manual focus. These days, yes, an iPhone is actually quite good at taking pictures. Apple invested billions for that to happen, it wasn't an accident. SLR cameras are still there though, you still have wedding photographers. You don't have the mall photographer for once a year photoshoots so much anymore, however. Everyone just takes pictures everywhere as they need. Software is the same now. I need a program to do a thing, I'll just have AI whip one up instead of looking for software that does something that but has 100 more features I don't need. Gone are the days where customers would come to the mall to get photos done and buy software. (Though that world's been gone for a while.) Software is now only a means to an end, so better find the customers that have needs and embed with them instead of having them look for Photoshop.
Airline pilots and train conductors are basically meat proxies 95% of the time and they are still employed. Still necessary for when things really go wrong.
I'm interpreting this as justification for sysadmins making a comeback, which I would absolutely love. We lost so much as a society when they were taken behind the shed.
Train conductors should be automated, we have better things to spend government money on than make work jobs. LIRR pays conductors 200k. It’s theft at that point
Train conductors are valuable IF they have first aid and CPR training; maybe riot training. There are probably a few other things along those lines as well - but none of them have anything to do with running a train: all the train related tasks can be better done automated.
There are still some things in an airplane that an autopilot cannot do. There is nothing on a train in normal operation (as opposed to maintenance) that a computer cannot do - as evidenced by multiple trains around the world running without humans operators since the 1990s!
The long island economy is split between well paid public servants, NYC commuters, and the circular economy of driveway contractors and tile contractors in an arms race for the nicest walking surface.
There are no other jobs on the island that pay decently. Cutting edge engineering work will make you half of what it will in either the DC or Boston areas.
The conductors are highly paid in part because it's the local custom, and part because a third of the economy would collapse if that train stops running.
Automating trains would let LIRR run trains more often because they aren’t having to pay 2-3x median income for someone who does little on each train
It would be better for society to pay these people to do nothing while we run more trains automatically.
All US passenger rail runs off of massive state subsidies. The absurd pay is the taxpayers money.
There are much worse things the government spends money on (or doesn't tax) than well paying jobs that play an important role in society. What do you think happens to that $200k? Do you think it just gets sent to the void? That money is spent on things like food, local businesses, rec sports and trips for family... and all that money gets spent again by those local businesses.
Meanwhile Boeing did enough stock buybacks from 2014 to 2019 to give every single boeing employee $200k... and when the 737 max crashes happened, the stock crashed back to pre buyback levels. THAT is truly sending money to the void to be lit on fire, money that could of been used to change the lives of thousands of employees and factory workers.
If screwing up gets me killed I want the person doing that job to be paid well and I want them to be held up to a high performance standard. I don't see how cutting pay for conductors is going to make anyone's lives better.
Or if we replace that well-compensated train conductor with automation I sure hope it's high quality stuff and not lowest-bidder slop.
> I don't see how cutting pay for conductors is going to make anyone's lives better
It's going to make passengers' lives better by making their train rides more affordable, allowing them to improve their lives by leveraging those funds.
Or, it's going to make shareholders' lives better by increasing their profits.
Either way, some people will benefit from it. The question is -- what is the real, practical tradeoff, and is the tradeoff worth it to the participants in the system.
Money, as a limited resource, means that money not spent on the conductor's wages can be spent elsewhere. In this hypothetical, fully automated trains can be run more frequently, so the money not spent on conductor wages means that the train comes every 5 minutes instead of every 15, and more stations.
It's going to make shareholder's lives better. And you better believe we're replacing conductors with lowest-bidder slop, that's the whole point. That's shareholder capitalism's playbook, it hasn't changed for the last 50 years.
What, you don't want to save a few bucks on a train ticket? Come on, you'll love it!
Automated trains have been around for nearly 40 years.
I want automate trains because they would make running trains more often possible
An AI system cannot replace professional people. It doesn’t replicate or do the work that they do. That’s not what it is for and it is incapable of performing these tasks.
If you are using these systems you still need to know what you want and how to specify it. You can use agents to automate much of the work but it can’t tell you that your ideas are bad, that there’s a better way to accomplish your goals; it can’t tell you what you want. And if you can’t understand what it is outputting you’re failing in your primary task as a developer: understanding the problem.
I agree with the title. I just don’t think these systems will ever be capable of replacing humans. As much as product managers and executives would love it and not have to deal with us; they’ll just be shooting themselves in the foot if they try to replace professional developers with AI harnesses and fire all the people with the institutional knowledge and skill.
I think you may need to update your priors a bit. AI is already replacing professionals. Developers are already losing jobs. LLMs and agents are already running in the hot loops of our technological civilization, unsupervised, with root. Institutional knowledge and skills are already being lost.
AI isn’t replacing people. Managers and executives have been sold a product that they believe they can use to replace people. Some of them probably still hold some enthusiasm for these products. But there seem to be some that are getting wise to the fact that they’ve been duped.
"What reason is there for the company to employ the meat proxy?" Maybe some form of liability, accountability...
New concept: the AI patsy
Moral Crumple Zones https://ferd.ca/notes/paper-moral-crumple-zones.html
Interesting thought. In a way this is already the case, to some extent.
But wouldn’t it be way more economical to have some sort of AI-insurance service? i.e. protection against AI fucking things up?
Yeah this is the closest I've found to explaining my own discomfort. Why I've been almost entirely an individual contributor my whole career and why I may opt-out and retire.
I have limited tolerance to take demeaning workload that I know "the boss" could do himself. I think my pride and satisfaction is in my intellectual contribution, not my ability to relay messages nor do repetitive slop work.
But the real revulsion is being a scapegoat. I will not rubber stamp and take responsibility for things not under my control. I have limited capacity to review and sign off work I've delegated to someone else. It is usually easier for me to do it myself and know it inside-out than to delegate and then review enough to have the requisite confidence in the result.
When the new plan is to increase throughput via delegation to these generators, I can't possibly review to my standards. So, I refuse to accept any of it as my work.
It also freaks me out a bit to think of the collective mess that is developing across society, with so many people apparently ready and willing to shovel and rubber-stamp. I think the "move fast and break everything" cult is going too far.
But, primarily, I just need to protect my own sanity and live to my own standards. And hope that I can back away far enough to avoid some of the splash damage when entropy catches up...
More commonly known as an "accountability sink". These existed before but it is so much worse with AI.
It pays 16 craploads of money per year.
Like so: https://www.smbc-comics.com/comic/blame
Or a different way around, Asimov's The Machine That Won The War, 1961: https://xpressenglish.com/our-stories/machine-that-won-the-w...
The human's purpose is to keep an eye on the LLM, occasionally intervene/interfere with its work, and most importantly, incur social/legal liability for anything that the LLM does wrong.
Kind of like the backup drivers used by self-driving cars.
Is being compliance ablative armor uplifting, fulfilling work? Is there any way in which your personal competence and effort impacts whether the AI screws up or not? Do you have any way to build job security as ablative armor?
That sounds miserable.
>That sounds miserable.
Welcome to our future that was already here years ago.
https://en.wikipedia.org/wiki/Bullshit_Jobs
Fulfilling work disappeared a long time ago, the book bringing this up was from 2018 before chatbots were really a thing. We are just experiencing the acceleration to all jobs being bullshit jobs.
The version of "meat proxy" I've heard is the person who forwards a machine-answer to someone else who didn't ask for a machine answer, and could just as easily made one themselves.
Calling someone a "meat proxy" for being the human in the loop is wild.
I don't think that's quite the problem the OP is pointing out. A corollary of "meat proxy" is "cognitive surrender." [1] In both cases, blindly pasting AI output and being an "Enter key presser", you have surrendered your cognition. In the latter case, you're in the loop but there's not a lot of reason for you to be.
Your underlying supposition is right, though. If you are truly thinking and in the loop, you are by definition not a meat proxy. I think those are the people who are successful with AI coding tools, but it takes discipline and it's still hard to find the joy and reward when you're not the one solving the problems.
[1] https://swiftrocks.com/using-ai-without-losing-critical-thin...
I think it might be appropriate if what we're talking about is a human connecting an LLM to something it can't natively access. Like something in the physical world - a piece of hardware, a different human, software the LLM can't access. As the human in the loop your role might be to mostly just shuffle information back and forth from this external resource. But depending on the situation there could still be quite a lot of interpretation and analysis performed by the human in the loop.
Thats not the problem of LLM/AI.
The problem is, that digital work, the new frontier of employment for a lot of epeople, got and will continue to get A LOT cheaper, faster and more accessable.
The PO who always complains about someone? They will just vibe stuff. It might even be shit but it will work good enough until LLMs/AI/AGI/ASI is good enough or better anyway than an avg developer.
Man i have seen so so much shitty code even in software companies.
You know when your HR software takes seconds to load? Yeah an LLM can do that today too
I've noticed some tasks (80-90%) can be accomplished by quickly typing out a feature/bug fix, accepting plans with ~10% brain power, and moving on. But there are other tasks where the LLMs struggle, and it requires significantly more time and mental effort to think through the architecture and requirements before it's to a point where the LLMs can process it.
What's difficult, is I now spend most of my time on these high-throughput tasks, where my brain isn't exactly off, but focused on quantity over quality. It's often difficult to identify when a task requires "real" brain-power, and when it does, it adds a lot of friction to dig into the code and orient myself. So it's hard to know when it's the right time to do that.
Turning off your brain with LLMs is how you get fired.
Even with small things, you need to be at the wheel. The other day I had to stop copilot from literally deleting the file I was working on. Why did it choose to go down that route? I don't know, but I nixed and reported the chat once it did that.
LLMs are dangerous and if for some reason you are stuck with using them, do NOT ever turn your brain off.
Ideally, you should not be using LLMs at all.
Turning your brain off is usually a defense mechanism to protect yourself against something bad. dissociate during trauma etc. lighter dissociation points to the interaction trending toward badness, not goodness. Seems like the mind and body is telling us something that we shouldn't ignore
No, it's usually an instinctive power-saving mechanism, and it happens incredibly frequently.
For example, the average driving commuter is not "zoning out" because of some bad experience with roads or cars, but because the route is extremely familiar.
I assume you're not saying it is but I just wanna call out that zoning out behind the wheel isn't a good thing. Drivers should be attentive at all times regardless of whether the route is familiar. So maybe something is going wrong there, even if it's not trauma?
One can still drive attentively but be on a route so familiar that unless something unusual happens, the brain does not bother recording any specific memory of it.
Even at work, I can spend a week on developing something, implement it, and once it's done will have almost no memory of it a month later because it was just one more feature in a long series of features that are all pretty much the same.
The human brain is a filter that seeks novelty. All the repetitious junk gets compressed down to something like key frames. This is why things like alert fatigue are so dangerous. Any signal that only very occasionally means danger will eventually be ignored.
I feel that AI's biggest threat is making us lose the intuition and judgment we need to correct it.
Yeah, I experience this a lot. I call it “zombie mode.” It’s like pulling a slot machine, though. Sometimes AI can go a few turns just with encouragement. But when I don’t maintain conceptual engagement, drift is inevitable.
My solution is saying “gnite” and coming back in a new terminal window with a clear restatement of my purpose. Or “what are you working on and why” can help.
Terminal windows are not great for keeping context; waiting for better design. It is possible.
That's the smallest dan luu post I've seen.
The more I see posts like this, the more I realize that the problems people have with AI are no different than the problems we had with outsourcing (development to low-cost countries) 20 years ago. The results and challenges are the same; except now the feedback loop is within minutes instead of overnight.
Agents are still a bit to eager to please, so they are still missing cases when the better answer is to push back, change course or refactor.
But when the human suggests changing the course, the LLMs will often agree the new direction is better and start heading there...
Heh, do you see how mad some people here on HN get when Claude tells them no. They must all be in management. It's a very delicate dance.
I think this is the dissonance that has been making me feel insecure / depressed around the state of AI coding. On the one hand I am way more productive. But on the other hand it is almost requiring more of my brain than before, because I am having to determine what is good output and what is slop. But that's not the part of my brain that I particularly like engaging — I like understanding something deeply. But in this case, that is just not possible. I like to think that as the human-in-the-loop, I am supplying the requisite _taste_ to the design and overall structure of the program. But at what point does my taste get taken over by the LLM as models improve?
Another problem is that when something breaks then I am very limited in my ability to reason through the breakage. It's necessary to go back to the LLM and ask it why it broke. At which point, I'm able to follow the reasoning — but not in any durable manner where I'll be able to remember it. To me, it's sort of like the difference between memorizing a passage and having a ready reference at hand at all time. We had this same problem with information before — now it's come for our reasoning.
> What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee.
Company is pleased! Cost savings!
Alas, the joy doesn't last long as pretty soon after the company realizes that after an LLM replaces almost all their employees, the much easier task of replacing the entire company begins, because the company itself has just become a much thinner meat proxy for getting an LLM to do whatever it is the customer relies on the company for.
Customer is pleased! Cost savings!
Alas, the joy doesn't last long as whatever savings they have from replacing the company with their own LLM loop is offset by whatever it is they do to earn income being similarly impacted.
All of the moats degrade together though they may fully give out at slightly different speeds.
> Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee.
If this is true, turning off the brain... is completely rational. Why burn out the brain for something happening either way? It's not like being fully engaged is going to result in anything that much better for the employee. In a world where code is this disposable, memorizing a codebase that won't recognizably exist in 3 years is also stupid.
I’m not allowed to enter legit chess tournaments where if I’m going be using my phone’s chess engine app to tell me the best move in every single position. Because that’s considered cheating by the chess players and tournament organizers and rightly so. But, there doesn’t seem to be an incentive to not do something similar in the world of work unless your LLM output is so bad or obviously inaccurate.
The challenge here is that the business world doesn't really recognize the concept of cheating.
In particular, tech is very open to people cheating and if billions of dollars could be had by somehow communicating the chess engine's work to a human at chess, the world of business would cheat at it.
Tech has a number of large startups that basically started with the assumption of "what if we assumed the law had no teeth?"
In the world of business and money, you lose if you don't cheat because everyone else is cheating.
And if it's that's true... What reason is there for the company's clients to keep sending money?
Clients will cut the middle man and run the LLM themselves.
I’ve pondered this too. I was in a group meeting with a consultant who is upfront with clients that he has started to use Claude heavily to speed up his output. There are probably a lot of clients who would still pay for his services because maybe they still appreciate his input or the relationship and don’t want to go through the pain of figuring out how to replace him with an LLM. It has been my experience that some teams would rather just subscribe to some software or platform even if there’s a free alternative.
Why do you assume that the fully engaged employee would be laid off at all?
You're telling me none of the 730,000 people laid off over the last four years were fully engaged employees?
This guy footnotes.
> Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee.
Why do you think management is so gung-ho about LLMs? Employees are expensive, and laying them off is therefore extremely appealing.
The entire, explicitly stated goal of the AI movement is to produce an ASI, which is better across every metric than a human. Even if you refuse to turn off your brain, the economics of paying you to keep the lights on upstairs aren't going to work out.
This was never going to work out well for the human.
I'm not sure what else to say. You listened to these people speak, what the heck did you expect?
There are a lot of programmers who are working with chatbots and watching YouTube simultaneously for the first time, simply matching the rest of the economy for the nature of work.
If you can do your job while watching TV your brain is turned off.
Maybe, but the parts of my brain I'm still using are a lot easier to use.
I was dreaming to be promoted to a cushy management role where I could talk bs for a living.
I managed to attain 'hands-on team lead' and later an ill-fated 'bootstrapped cryptocurrency CTO' role... Nobody wanted to pay me to talk as a career. Now they do!
So I'm very happy about that.
I feel contempt for all the people who had it easy and 'chose' to be engineers voluntarily.
trends like these remind me very much of how true some of the predictions in Douglas Hofstadter's "Anti-Intellectualism in American Life" were: https://ia802903.us.archive.org/6/items/richard-hofstadter-a...
he was writing in reaction to the McCarthy era which, arguably, is very similar to kind of bombastic nativism we're seeing now from our political elites. his argument essentially being that there are cultural and institutionalized forces that are driving people to, well, turn their brains off - specifically to no longer be skeptical of what they consume, to not want to refine and improve their toolbox of analysis, to not want to understand how things work for the sake of knowing
and that these acts were deliberate - a populace is so much easier to control when it's complacent and unknowing, who believes, unskeptically, that Oceania has always been at war with Eastasia, unquestioning about who was writing the PR missives that feed the news cycle (see also: https://www.commondreams.org/opinion/what-is-copaganda), how verbatim they are, and what the real intent of the authors were
like, I get that you don't want to put in effort for a job you don't like. but that effort also makes you better at your work. it makes it easier on you, it lets you create less incidents, it empowers you to be more agent in your own life, to, who knows, have the guts to start a consulting gig because your skill-sets actually are better than the average
there's so many benefits from just being more diligent, longterm, proactive benefits. but that's not everybody's extrinsic motivator and it's so much easier to just be Skinner-boxed by social media companies into doomscrolling, I suppose
It's an interesting idea. I'm not sure I agree.
Here's the thing I'm finding - LLMs allow me to do more coding when my brain isn't firing on all six cylinders. It means I get more code written because I don't have to be "in the zone" to exhaustively hunt down every piece of API and dependency and have my flow continuously broken up by missing information. The code written in that state is lower quality, but I can patch up five-cylinder-brain code faster than I can synthesize code from a blank page outside of my flow state.
When throughput is an issue, you have to think judiciously. I'm definitely not saying you shouldn't think but you have to apply human oversight effectively. For me that means front-loading through dialectic planning, and maximizing guardrails.
I still don't get who is this guy and why anyone should care.
Turning my brain off actually works really well when I'm doing something more creative, and while I know this isn't exactly what he's talking about, I wonder if we are just offloading things we shouldn't have to think about to have more space to think about other things.
Of course, this becomes a lot more difficult when you're unemployed.
The obligatory XKCD, panel 2: https://xkcd.com/568/
There's some software that really doesn't require my brain. I gave Astra an ssh key for root on a little Pi Zero with a microphone attached and asked it to make recordings of a siren going off in our neighbourhood, based on a couple of sirens. This would have been a great deal of work a couple of years ago, now it's a few tokens, and honestly, it doesn't really matter if it's shitty code, I just want the timestamps.
Then there's the software that I write for a living, and almost my entire day is now spent making hard decisions. /grilling gives me difficult technical design decisions endlessly and I am forever doing product management on features from agents. My brain is back in CTO mode, which isn't what I'd planned for myself after I'd left that particular cursed path, but fine. I am doing the opposite of turning my brain off, because I no longer can just chill out and mindlessly write code.
Both these states can exist simultaneously. But my brain work has shifted into something different, while getting no less intense.
I would like to argue opposite.
Developers like to "turn their brain off" during "super hard debuging" sessions. Mental flow state is a well documented as pleasurable experience. It is like a drug!
Auditing and reviewing autogenerated commits, is hard gruelling mental labour, compared to that. You do not even get a credit as an author! You can not get into groove to get your fix!
Um, some data:
”Nowhere is this clearer than in entry-level software engineering—though the picture depends on whom you ask. Brynjolfsson discovered that employment among developers aged 22 to 25 has fallen nearly 20% from its late-2022 peak. The online job site Indeed also paints a stark picture: software development job postings have fallen 53% from the same starting point.”
https://insights.som.yale.edu/insights/the-real-job-destruct...
They aren’t firing the meat proxies to that degree yet, but they absolutely have stopped hiring the meat proxies. It’s going to be a very dirty year on Wallstreet when you’ll see massive sneaky layoffs of the existing meat proxies.
You can turn your brain off with these models. It is what it is.
I think there's just too many confounding variables to say. ZIRP policy ended in 2022, with interest rates going up 5% between 2022 and mid-2023[1]. This both means that money was historically cheap when these early-career developers were being hired in the peak, and it was relatively (in terms of the past couple of decades; eg, the modern tech industry) expensive after.
We've seen layoffs as a result of this, and nobody's hiring a cohort of juniors when they're trying to afford the employees they already have.
On top of that, companies are firing the money cannon at AI initiatives and experiments (some of which will be successful and others which won't), leaving less money for humans.
Anyway, it's just a single occurrence of a hiring drop in a single economic cycle. Unfortunately, it just happens to perfectly line up with the rise of GenAI. We may never be able to tease apart cause and effect in this period, and we certainly won't be able to do so until we have at least a handful more years of data to see how junior hiring does or does not rebound.
[1]https://newsletter.pragmaticengineer.com/p/zirp
Everything you said is true, I agree with you.
”Unfortunately, it just happens to perfectly line up with the rise of GenAI.”
Right. That’s just something I think we should keep our pulse on and keep looking at this data every six months (seriously, this needs to be tracked by devs because this data may start forming into that proverbial “writing on the wall”).
No one should be planning on being a software engineer in about 3-5 years, all need to plan for alternative income sources or early retirement. I think, at least.
>No one should be planning on being a software engineer in about 3-5 years, all need to plan for alternative income sources or early retirement. I think, at least.
Interesting. I recall Geoffrey Hinton saying something similar about Radiologists.
“People should stop training radiologists now” "it is just completely obvious that within 5 years deep learning will do better than radiologists”
https://youtu.be/2HMPRXstSvQ?si=prlQ_uGkCrWAZwgU&t=22
All is possible with good blackbox testing.