Funny how the article starts with someone using AI — to develop more AI stuff.
This reminds me of web3, where almost all projects were just web3 infrastructure or services, to the point that the purpose of most start-ups was completely inscrutable to outsiders.
Great point about web3. People always talk about during gold rushes, selling shovels instead of being the ones digging for gold. And it might be a reasonable idea, but there needs to be actual people getting rich from the gold to be an actual gold rush, if everyone is just selling shovels, it is probably a sign of a bubble.
As an experienced dev who’s gotten his feet wet a little with AI, I feel like , and does anyone else feel this way, that I spend more time telling AI what to do than it would spend actually writing this all out myself?
I don't understand how it doesn't provide evidence for the first of these. Sure you can't necessarily extrapolate to all software developers but it's still evidence just not proof.
Yup, entirely gave up on AI as a result, and it didn't help that reviewing and fixing AI code made me incredibly bored to the point that if this somehow becomes the new norm of development, I'll have to find a new career, because I want to actually build things, not manage AI.
Not a dev but I’ve had similar experiences testing AI to help me craft various documents.
I spend ages crafting the perfect prompt only to cross my fingers and hope that the output is what I want it to be when I could have just spent all that time actually crafting the document exactly as I want it to be in the first place. Often I then need to spend more time editing and tweaking the output from the AI anyway.
I’m honestly starting to feel a little crazy because I’m the only one at my workplace that sees it this way.
I've found AI to be a poor fit for writing when the task is getting an idea out of my head and on to the page. Works well enough when the task is taking some collection of already written information and creating writing based on it though.
Absolutely, much harder to describe what I want the software to do in English than logical programming language, and then add to that the time taken to understand the code and make sure it does what was intended. Perhaps the worst part, takes the joy out of it all.
It is nice as a tool to help solve some hairy stuff sometimes.
I find it depends. I just "vibe coded" (I hate this phrase) a simple mobile app for something I needed, without writing a line or code, or indeed ant knowledge of any web/mobile technology. No way realistically I would spend a few days learning Flutter then a few more writing my app.
That’s the key, really. It’s excellent for small, highly scoped code. But AI doesn’t have the context or experience a senior developer does in the full system architecture of a mid-sized company.
Sure, but by that time you've IPO'd and dumped the bag on the public, founders and seed-round investors make bank and are long gone by then.
I'm only half-joking: personally I'll be looking very closely at the IPO prospectus of any company founded in/after 2024 or so, to know how much vibe coding risk I can expect in its long-term prospects.
I don't think this behavior has anything to do with AI although it seems like it's been used as an excuse to justify it. Everyone seems to be in a belt-tightening risk averse mode right now and that means cutting junior positions and leaning on smaller teams of senior staff. You can see this behavior in more than just tech and more than just positions that can be replaced by AI. The job boards betray it as well, job postings for junior staff have dried up.
As someone who spends his non work hours convincing a half baked quasi person to not do dumb things (a two year old), I have zero interest in convincing a half baked quasi person to not do dumb things during work hours (most coding agents).
I’ve had good results with Claude, it just takes too long. I also don’t think I can context switch fast enough to do something else while it’s churning away.
I think it’s allowed me to spend more time being an architect and thinking about processes, problem solving. To put it another way, I’m still a developer, possibly to a higher degree (because I can spend more time doing it), and less of a coder.
I have very few issues with Claude. If I just tell it what the goal is, it will make some sensible suggestions, and I can tell it to start coding towards it. It rarely messes up, and when it does I catch it in the act.
You don't necessarily want to completely tune out while you're using the AI. You want to know what it's up to, but you don't need to be at your highest level of attention to do it. This is what makes it satisfying for me, because often it eats up several minutes to hunt down trivial bugs. Normally when you have some small thing like that, you have to really concentrate to find it, and it's frustrating.
When the AI is on a multi-file edit that you understand, that's when you can tune out a bit. You know that it is implementing some edit across several instances of the same interface, so you can be confident that in a few minutes everything will build and you will get a notification.
It's as if I can suddenly make all the architecture level edits without paying the cost in time that I had previously.
I was going to point out that what you are describing is exactly like what it is to be a leader/director of people working in most efforts, i.e., managing people, when it occurred to me that maybe what we are dealing with in this conflict and mud slinging around AI is the similar conflict of coders not wanting to become managers as they are often even not really good at being managers. Devs work well together at a shared problem solving (and even that often only sometimes), but it strikes me as the same problem as when devs are forced to become managers and they really don't like it, they hate it even, sometimes even leaving their company for that reason.
When you are working with AI, you are effectively working with a group of novice people, largely with basic competence, but lacking many deeper skills that are largely developed from experience. You kind of get what you put into it with proper planning, specificity in requests/tasks, proper organization based on smart structuring of skillsets and specializations, etc.
This may ruffle some feathers, but I feel like even though AI has its issues with coding in particular, this issue is really a leadership question; lead and mentor your AI correctly, adequately, and appropriately and you end up with decent, workable outcomes. GIGO
If you add an AGENTS.md, the AI agent will work more efficiently, and there will be far fewer problems like the ones you’re facing. You can include sections such as Security, coding style guidelines, writing unit tests, etc.
Doesn't that depend on the implementation? There's a trade-off between performance and determinism for sure, but if determinism is what you want then it should be possible.
If you fix random seeds, disable dropout, and configure deterministic kernels, you can get reproducible outputs locally. But you still have to control for GPU non-determinism, parallelism, and even library version differences. Some frameworks (like PyTorch) have flags (torch.use_deterministic_algorithms(True)) to enforce this.
TL;DR: assuming you've squashed all regular non-determinism (itself a tall ask), you either need to ensure you always batch requests deterministically, or ensure all kernels are "batch invariant" (which is absolutely not common practice to do).
This. I’m still amazed how many people don’t understand how this technology actually works. Even those you would think would have a vested interest in understanding it.
How do the colleagues of people "vibe-coding" feel about that?
Does it end up like having colleagues who are aren't doing or understanding or learning from their own work, and are working like they offshored their job to an overnight team of juniors, and then just try to patch up the poor quality, before doing a pull request and calling their sprint done?
Or is it more like competent mechanical grunt work (e.g., "make a customer contact app with a Web form with these fields, which adds a row to the database"), that was always grunt work, and it's pretty hard to mess up, and nothing that normally the person you assigned it to would understand further anyway by doing it themself?
random example, may or may not come from a real situation that just happened:
- other team opens jira ticket requesting a new type of encabulator
- random guy who doesn't know anything of how the current encabulator system works picks up the ticket for whatever reason
- 10 minutes later opens a 2000 line vibe coded pr with the new encabulator type and plenty of unit tests
- assigns ticket to the person who designed the current encabulator system for review
- encabulator system designer finds out about this ticket for the first time this way
- "wait wtf what is this? why are we doing any of this? the current system is completely generic and it takes arbitrary parameters?"
- waste an hour talking to the guy and to the team that requested this
- explain they could just use the parameters, close pr and ticket
> working like they offshored their job to an overnight team of juniors
Reviewing a co-worker of 13 years now gives me the exact same unpleasant feeling as opening an MR from a junior where I know it's going to be garbage and I'm tired of struggling to let them down easy.
Had our junior dev working on a case actually ran the code in the merge request and looked at the result, it would have been obviously broken and unusable. Like not even wrong value, it would have spewed a bunch of HTML into the page.
> Does it end up like having colleagues who are aren't doing or understanding or learning from their own work, and are working like they offshored their job to an overnight team of juniors, and then just try to patch up the poor quality, before doing a pull request and calling their sprint done?
What I’ve seen also happen is senior devs suddenly starting to put out garbage code and PRs. One senior dev in our project has become a menace and the quality of his work has dramatically dropped.
The worst is when I have to baby-sit someone else’s AI. It’s so frustrating to get tagged to review a PR, open it up, and find 400 lines of obviously incorrect slop. Some try to excuse by marking the PR [vibe] but like what the hell, at least review your own goddamn ai code before asking me to look at it. Usually I want to insta reject just for the disrespect for my time.
In particular, under v0.1.0 see `decode-branch.md` prompt and it's associated generated diff which implements memoization for backtracking while performing decoding.
It's a tight PR that fits the existing codebase and works well, you just need a motivating example you can reproduce which can help me you quickly determine if the proposed solution is working. I usually generate 2-3 solutions initially and then filter them quickly based on a test case. And as you can see from the prompt, it's far from well formatted or comprehensive, just "slap dash" listing of potentially relevant information similar to what would be discussed at an informal whiteboard session.
Junior dev vibe codes some slop to solve a problem that was barely a problem. The "solution" to that not-a-problem was 2500 lines of slop. Unused variables, spaghetti logic, unit tests that were clearly write-once, read-never.
The slop at that point is the meta-problem, the real problem becomes me trying to convince (through multiple rounds of code review) that this was not maintainable code, and not a tenable development model.
All those rounds of review take TIME and mental effort. At some point, when the code review takes more effort than the code owner contributed, the value proposition is shot.
Generate the code however you want (llm, bit twiddling, whatever), but the effort and care must be there. If you can't use an llm skillfully and vouch for the output, it's gonna be a no from me dawg
lol if I saw a teammember create a PR with "[vibe]" in the title I would:
1) Gape in disbelief
2) Shout obscenities as loud as I can
3) Take a deep breath
4) reject the PR and seriously consider firing them.
Yeah "consider firing" is a bit of an extreme kneejerk reaction, but I just feel like we HAVE to create consequences for this nonsense.
There seems to be a gap between the theory you're advocating...which I actually agree with and the practical execution in your own public projects which you talk about heavily.
I haven't been able to get any of your recently published projects (Scribe, volknut, a few others) running successfully on linux, whether following your documentation, using the tagged releases, or reverse engineering the true functionality and CLI arguments working from your provided source code I found to have wasted my time.
It's difficult to believe you when your own documentation is entirely wrong or 404'd.
I'd genuinely love to see your projects in action, since the concepts sound promising, but you need to back it up with shipping properly.
There's a reason I haven't dropped release pages yet. These projects are under heavy development and I push to github to checkpoint mostly rather than release. I'm sorry if you feel like I've misled you, I've tried to be clear that what I'm sharing now is to understand what I'm doing, to give visibility, and it's not ready yet. I'm committed to delivering great software, and when I tell you it's ready you can rest assured that it will work.
Understand that I'm one man working on 20 projects simultaneously, with 5+ under active development at any one moment, so release stabilization and cadence will take a little bit to lock in.
This is why AI code review continues to be mostly useless in my experience. If you instruct an LLM to make PR comments it will come up with some, every time, even when none are warranted.
I've sort of gotten on the bandwagon. I initially used AI auto complete at my previous job and liked it a lot as a better intellisense, but wouldn't have used it to write PRs -- for one I tried it a couple times and it just wasn't good enough.
My new job pushes cursor somewhat heavily and I gave it a try, it works pretty well for me although it's definitely not something I would rely on. I like being able to ask it to do something and let it go off and come back to it in a while to see how it did. For me, I think it makes it easier to start on changes by coming into something (that might be wrong and bad), but for me personally having something in a PR to start from is a nice mental hack.
If it did well enough on the initial attempt I'll try to stick with it to polish it up, but if it failed terribly I'll just write it by hand. Even when it fails it's nice to see what it did as a jumping off point. I do wish it were a bit less prone to "lying" (yada yada anthromorphization it's just tokens etc.,) though, sometimes I'll ask it to do something in a particular way (e.g., add foo to bar and make sure you X, Y, and Z) and it'll conclude (rightfully or not) that it can't do X, but then go on anyway and claim that it did X.
I wish it were easier to manage context switching in cursor though, as it is juggling IDE windows between git repo clones is a pain (this is true for everything though, so not unique to cursor). I wish I could just keep things running on a git branch and come back to them without having to manage a bunch of different clones and windows etc.,. I think this is more of a pain point with cursor since in theory it would allow you to parallelize more tasks but the tooling isn't really there.
edit: the starting point for this is probably worktrees, I remember reading about these a while ago and should probably check them out (heh) but it only solves the problem of having a bunch of clones sitting around, I'd still need to manage N windows.
One improvement is to start using an AGENTS file. There are many. Here's the repo that I personally use thanks to Pontus Abrahamsson and using TypeScript:
Exactly this, if you're babysitting the AI, you are, by definition, _not_ vibe coding. Vibe coding means not reading the resulting code, and accepting that things will break down completely in four or five iterations.
Well you're now having to 'coach' an AI into getting it to do what you want rather than a junior employee.
And yes, you need to develop your skills in asking for what you want, whilst still having to review the outputs similar to a junior employee. It just doesn't really learn or get better based on your feedback.
Q2. How reactionary are you to canceling a running prompt as soon as you see a wrong turn?
Q3. How well do you author PRs without ever saying "well AI did that part." Do you own your PRs?
Re: 1. It's a new skill set. Some engineers are not good at it. Some are amazing. And some are out of this world. You can spend a few minutes crafting a prompt that's so thorough, so well thought out but also succinct and to the point. How it touches on all the possible unknowns the AI might have as it works through the task and has direction. An out of this world prompt gets you between 90% and 95% completely done with a task that before AI would take weeks.
Re: 2. You can do very well with the first prompt but the whole other side of this is watching the AI work. Knowing when to watch very closely, and when it's okay to multi-task. Know when you immediately press ESC and tell the AI exactly what you just saw you didn't like. Catching it quickly is like early detection of cancer. You can save the rest of your day by preventing some messy dead ends before they even happen.
Re: 3. Not everyone on the team will be using AI to the level the Jedi are. And personal responsibility for your code, 90% AI generated or 0%, is still your responsibility. Once you make your PR and ask for feedback, you should no longer be able to say "oh I didn't write this part." Yes you did. No one on the team needs to know what % AI your PR was. And you don't need to know theirs.
Regarding point 1 - when you say "a few minutes," I'm wondering if we're talking about the same thing? I spent two solid months with Claude Max, before they imposed limits, running multiple Opus agents, and never once got anywhere close to "weeks of work" from a single prompt. Not even in the same zip code.
So I'm genuinely asking: could you pretty please share one of these prompts? Just one. I'd honestly prefer discovering I can't prompt my way out of a paper bag rather than facing the reality that I wasted a bucketload of time and money chasing such claims.
So basically your source is: "trust me bro, I'll prove it to you?"
I think it is a relevant question. We cannot go around calling someone genius prompt engineers, and then skip the engineering part of noting down what actually works and how it can be replicated. Could we try to work backwards not from the perspective of the problems of the person asking the question, but perhaps work backwards from your claims (I.e. to where you have them from)
not at all. I'm happy to provide the prompts. Let's just agree on what 2+ weeks coding problem we are trying to solve first. If I just pick one and show you the prompt I used you'll say "no way would that have taken over 2 weeks."
I can have Claude write stuff that does basically the right thing and actually works but, as so many people say, it's like working with a junior dev. The only way I'm going to get code I'd want to present as my own is by sitting looking over their shoulder and telling them what to do constantly, and with a junior dev and with Claude (etc) that takes longer than just writing it myself.
AI-coded/assisted work should be flagged as such so everyone knows that it needs extra scrutiny. If we decided to let a horde of junior devs loose on a codebase we'd naturally pay more attention to PRs from new intern Jimmy than revered greybeard Bob and we shouldn't let Claude hide behind Bob's username.
1 Is easy enough for trivial tasks but in a complex (typically horrible) production codebase nearly all the work is investigation and debugging. However good the initial prompt is, soon the context becomes flooded with log output and code and the LLM goes off the rails quite quickly.
Doing 2 well is the AI babysitting mentioned in the article. Of course you can stop it every minute and tell it to do something else, then watch it like a hawk to make sure it does it right, then clear context when it ignores you and makes the mistake you told it not to make. But that is often then slower than just doing the work yourself to begin with, probably leading to the findings we've all seen that LLM use is actually reducing productivity.
I think living with crappy AI code is the price we currently have to pay for getting development done quicker. Maybe in a year it'll have improved enough that we can ask it to clean up all the mess it made.
(Possibly I just have higher standards than most, other humans can be quite bad too.)
"all the work is investigation and debugging" - Yes! Exactly you can ask the AI a bunch of questions first and really dig into what the codebase currently does. Then spend the time crafting that prompt that explains how to surgically do what needs to be done. If you are watching it every min like a hawk you are doing it wrong. You need to watch it more like a VERY smart junior dev and trust but verify. I'm not saying it's easy to get good at these new skill sets. But simply throwing your hands up and saying "I'll just walk everywhere vs using a bicycle" isn't a strategy that's going to work well.
I for one am sick and tired of hearing opinions about AI coding from people selling tools and "solutions" using AI. It's as if the entirety of tech media is just one massive ad.
I know that it's always been this way to some degree, but this is getting out of hand.
Everyone suggesting maintaining an agent.md file haven’t met the majority of their coworkers who refuse to document anything or read documentation. Perhaps train a model that can generate these agent files for them?
IMO the best possible outcome of this AI hype is better documentation.
It's proven that (agentic) LLMs work better and more efficiently when they have the proper context for the project (AGENT(S).md), the documentation is accessible and up to date (docs/ directory with proper up to date markdown) and the tooling is smooth (git hooks that prevent secrets from being pushed, forced style checking and linting).
...surprise! All of this makes things easier for actual humans too!
I think we’ll just get more documentation not better documentation. There never was a problem generating documentation from code. It’s the business context and use case documentation that’s hard to generate.
There never was a problem generating documentation from code
That doesn't match my experience at all. Before AI, generating documentation from code was either getting some embedded comments for a function call, or just list a function's arguments.
AI reads the implementation, reads the callers and callees, and provides much smarter documentation, with context. Something sorely lacking from previous solutions. Is it sometimes not completely accurate? Probably. But still a giant step forward.
You are touching on a critical point that I have been amused by at times in these types of conversations; people are hating on AI for doing all the things that people have done and still do.
Poor planning, check. No/inadequate documentation, check. Sloppy and janky code, check. Poor practices and methods, check. Poor and miscommunication, check. Poor technical choices, check. What am I missing?
Maybe this is just a matter of some of the top tier 100x devs and teams clutching pearls in disgust at having to look at what goes on below Mt Olympus, but this is also not any different to how code quality cratered and is still really poor due to all the outsourcing and H-1B (sorry, all you H-1B hopefuls) insourcing of quantity over quality.
I say that without any judgement, but reality simply is that this issue has long been a quantity over quality argument even before AI, and mostly for non-dev reasons as the recent de-qualification of R&D funding revealed and had a marked impact on dev jobs because the C-suite could don't use R&D funding for financial shenanigans.
If people want to hate AI, go ahead. People hated and hate on the H-1B abuses and they hate on AI now. I would hope that we can just move beyond griping and mean-girling AI, and get to a point where proper practices and methods are developed to maybe make the outcomes and outputs better.
Because again, AI is not going anywhere less than even H-1B and I am sure the C-suite will find some new way to abuse and play financial shenanigans, but it's simply not going away and we need to learn to live with it since it will seemingly only get "better" and faster as it changes at breakneck speeds.
An advertisement, where the protagonists weigh the pros and cons and then come down on the side of "paying the innovation tax".
Fastly profits from"AI" just like Cloudflare (or should we say "Claudeflare").
This selection of developers does seem representative at all. The new strategy is to acknowledge "AI" weaknesses but still adamantly claim that it is inevitable.
Kind of wild watching engineers, technologists, etc clutch their pearls
AI makes mistakes - so do people - I’ve been paid many a time to go clean up the non-AI generated code humans left behind. I’ve spent countless hours troubleshooting dumb bugs generated by humans
AI brings its own set of problems but I think some people just don’t want to hear what a net benefit it is
Post after post lamenting that no machine will ever replace the human touch is so old - people have been saying that about new technology since the beginning of time
I say embrace it - great to talk about its problems objectively and how to avoid common issues, but it’s tiresome to hear all the reasons why it’s crap all the time
I would instead love articles (you do see them here sometimes) that go over how people coded complex solutions to problems using AI and the challenges they faced along the way
Knowing where the weak points are or where it may be prone to error
If you don’t embrace it, pretty soon kids coming out of college that grew up with it that contrary to popular DO understand fundamentals are going to be running circles around the people who haven’t yet figured out how to get value out of it - while you’re out there saying it’s crap and only idiots use it, these kids are going to be moving mountains - I can’t wait to see the cool stuff people build - I don’t see the future as bleak
Where does this vibe term come from. There is no fucking vibe here. Just ask question to AI service and then check the results. Saves ton of time but one must be vigilant and be able spot bullshit. Often the solution is correct but code is very inefficient. Luckily in many cases AI can correct based on subsequent critique / input.
The meaning has changed very quickly for some reason, but I think the original meaning was that you evaluate the result in a superficial manner by checking if the program seems to work fine. So "vibe" here is as opposed to any methodical or formal approach. No code review etc.
Funny how the article starts with someone using AI — to develop more AI stuff.
This reminds me of web3, where almost all projects were just web3 infrastructure or services, to the point that the purpose of most start-ups was completely inscrutable to outsiders.
I'm having lots more hope for AI though.
Hey now, there’s a lot of Hello World, Mandelbrots and Ugly Tailwind Dashboards too :o)
Great point about web3. People always talk about during gold rushes, selling shovels instead of being the ones digging for gold. And it might be a reasonable idea, but there needs to be actual people getting rich from the gold to be an actual gold rush, if everyone is just selling shovels, it is probably a sign of a bubble.
A "Shovel Rush"
Not really no. ChatGPT is used by 100s of millions of people directly and not for creating more AI apps like web3. So it is nothing like web3.
As an experienced dev who’s gotten his feet wet a little with AI, I feel like , and does anyone else feel this way, that I spend more time telling AI what to do than it would spend actually writing this all out myself?
You may have seen headlines about this paper, which found that while most devs (apparently) feel like AI makes them 20% faster, they’re actually 20% SLOWER with it: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
> We do not provide evidence that:
> AI systems do not currently speed up many or most software developers
> AI systems in the near future will not speed up developers in our exact setting
> There are not ways of using existing AI systems more effectively to achieve positive speedup in our exact setting
I don't understand how it doesn't provide evidence for the first of these. Sure you can't necessarily extrapolate to all software developers but it's still evidence just not proof.
Yup, entirely gave up on AI as a result, and it didn't help that reviewing and fixing AI code made me incredibly bored to the point that if this somehow becomes the new norm of development, I'll have to find a new career, because I want to actually build things, not manage AI.
I agree with this almost completely
Not a dev but I’ve had similar experiences testing AI to help me craft various documents.
I spend ages crafting the perfect prompt only to cross my fingers and hope that the output is what I want it to be when I could have just spent all that time actually crafting the document exactly as I want it to be in the first place. Often I then need to spend more time editing and tweaking the output from the AI anyway.
I’m honestly starting to feel a little crazy because I’m the only one at my workplace that sees it this way.
I've found AI to be a poor fit for writing when the task is getting an idea out of my head and on to the page. Works well enough when the task is taking some collection of already written information and creating writing based on it though.
I usually find this to be the case with an existing codebase, unless I'm doing something extremely generic.
It's just really hard to convert requirements to English when there are a bunch of land mines in there that you know how to avoid.
Absolutely, much harder to describe what I want the software to do in English than logical programming language, and then add to that the time taken to understand the code and make sure it does what was intended. Perhaps the worst part, takes the joy out of it all.
It is nice as a tool to help solve some hairy stuff sometimes.
I find it depends. I just "vibe coded" (I hate this phrase) a simple mobile app for something I needed, without writing a line or code, or indeed ant knowledge of any web/mobile technology. No way realistically I would spend a few days learning Flutter then a few more writing my app.
That’s the key, really. It’s excellent for small, highly scoped code. But AI doesn’t have the context or experience a senior developer does in the full system architecture of a mid-sized company.
Here's the approach I've seen so far at a few startups:
1. Replace junior developers with AI, reducing costs today.
2. Wish and hope that senior developers never retire in the future.
3. ?
3. Hire more developers to decode / devibe code, paying double in the end.
Sure, but by that time you've IPO'd and dumped the bag on the public, founders and seed-round investors make bank and are long gone by then.
I'm only half-joking: personally I'll be looking very closely at the IPO prospectus of any company founded in/after 2024 or so, to know how much vibe coding risk I can expect in its long-term prospects.
1. Replace junior developers with AI, reducing costs today.
2. ?
3. Profit!
https://www.youtube.com/watch?v=tO5sxLapAts
It worked with COBOL.
I don't think this behavior has anything to do with AI although it seems like it's been used as an excuse to justify it. Everyone seems to be in a belt-tightening risk averse mode right now and that means cutting junior positions and leaning on smaller teams of senior staff. You can see this behavior in more than just tech and more than just positions that can be replaced by AI. The job boards betray it as well, job postings for junior staff have dried up.
you'll be retired by 3, good luck kids
As someone who spends his non work hours convincing a half baked quasi person to not do dumb things (a two year old), I have zero interest in convincing a half baked quasi person to not do dumb things during work hours (most coding agents).
I’ve had good results with Claude, it just takes too long. I also don’t think I can context switch fast enough to do something else while it’s churning away.
At least your two year old learns.
I think it’s allowed me to spend more time being an architect and thinking about processes, problem solving. To put it another way, I’m still a developer, possibly to a higher degree (because I can spend more time doing it), and less of a coder.
I have very few issues with Claude. If I just tell it what the goal is, it will make some sensible suggestions, and I can tell it to start coding towards it. It rarely messes up, and when it does I catch it in the act.
You don't necessarily want to completely tune out while you're using the AI. You want to know what it's up to, but you don't need to be at your highest level of attention to do it. This is what makes it satisfying for me, because often it eats up several minutes to hunt down trivial bugs. Normally when you have some small thing like that, you have to really concentrate to find it, and it's frustrating.
When the AI is on a multi-file edit that you understand, that's when you can tune out a bit. You know that it is implementing some edit across several instances of the same interface, so you can be confident that in a few minutes everything will build and you will get a notification.
It's as if I can suddenly make all the architecture level edits without paying the cost in time that I had previously.
Yeah, sounds about like babysitting a toddler.
I was going to point out that what you are describing is exactly like what it is to be a leader/director of people working in most efforts, i.e., managing people, when it occurred to me that maybe what we are dealing with in this conflict and mud slinging around AI is the similar conflict of coders not wanting to become managers as they are often even not really good at being managers. Devs work well together at a shared problem solving (and even that often only sometimes), but it strikes me as the same problem as when devs are forced to become managers and they really don't like it, they hate it even, sometimes even leaving their company for that reason.
When you are working with AI, you are effectively working with a group of novice people, largely with basic competence, but lacking many deeper skills that are largely developed from experience. You kind of get what you put into it with proper planning, specificity in requests/tasks, proper organization based on smart structuring of skillsets and specializations, etc.
This may ruffle some feathers, but I feel like even though AI has its issues with coding in particular, this issue is really a leadership question; lead and mentor your AI correctly, adequately, and appropriately and you end up with decent, workable outcomes. GIGO
If you add an AGENTS.md, the AI agent will work more efficiently, and there will be far fewer problems like the ones you’re facing. You can include sections such as Security, coding style guidelines, writing unit tests, etc.
Vibe Coding is so early 2025, I only check in context, prompt instructions, and seed values to deterministically generate the machine-code.
LLMs aren’t deterministic even with a seed.
Doesn't that depend on the implementation? There's a trade-off between performance and determinism for sure, but if determinism is what you want then it should be possible.
If you fix random seeds, disable dropout, and configure deterministic kernels, you can get reproducible outputs locally. But you still have to control for GPU non-determinism, parallelism, and even library version differences. Some frameworks (like PyTorch) have flags (torch.use_deterministic_algorithms(True)) to enforce this.
what if you set top_p=1, temperature=0, and always run it on the same local hardware
Horace He at Thinking Machines just dropped an awesome article describing exactly this: https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
TL;DR: assuming you've squashed all regular non-determinism (itself a tall ask), you either need to ensure you always batch requests deterministically, or ensure all kernels are "batch invariant" (which is absolutely not common practice to do).
Maybe if you run it on CPU. (Maybe on GPU if all batching is disabled, but I wouldn't bet on it.)
cosmic wave will get you
Yes, that's the joke
This. I’m still amazed how many people don’t understand how this technology actually works. Even those you would think would have a vested interest in understanding it.
How do the colleagues of people "vibe-coding" feel about that?
Does it end up like having colleagues who are aren't doing or understanding or learning from their own work, and are working like they offshored their job to an overnight team of juniors, and then just try to patch up the poor quality, before doing a pull request and calling their sprint done?
Or is it more like competent mechanical grunt work (e.g., "make a customer contact app with a Web form with these fields, which adds a row to the database"), that was always grunt work, and it's pretty hard to mess up, and nothing that normally the person you assigned it to would understand further anyway by doing it themself?
random example, may or may not come from a real situation that just happened:
it's so incredibly productive> working like they offshored their job to an overnight team of juniors
Reviewing a co-worker of 13 years now gives me the exact same unpleasant feeling as opening an MR from a junior where I know it's going to be garbage and I'm tired of struggling to let them down easy.
Patch up?
Had our junior dev working on a case actually ran the code in the merge request and looked at the result, it would have been obviously broken and unusable. Like not even wrong value, it would have spewed a bunch of HTML into the page.
> Does it end up like having colleagues who are aren't doing or understanding or learning from their own work, and are working like they offshored their job to an overnight team of juniors, and then just try to patch up the poor quality, before doing a pull request and calling their sprint done?
yes
What I’ve seen also happen is senior devs suddenly starting to put out garbage code and PRs. One senior dev in our project has become a menace and the quality of his work has dramatically dropped.
The worst is when I have to baby-sit someone else’s AI. It’s so frustrating to get tagged to review a PR, open it up, and find 400 lines of obviously incorrect slop. Some try to excuse by marking the PR [vibe] but like what the hell, at least review your own goddamn ai code before asking me to look at it. Usually I want to insta reject just for the disrespect for my time.
I've got some receipts for what I think is good vibe coding...
I save every prompt and associated ai-generated diff in a markdown file for a steganography package I'm working on.
Check out this document: https://github.com/sutt/innocuous/blob/master/docs/dev-summa...
In particular, under v0.1.0 see `decode-branch.md` prompt and it's associated generated diff which implements memoization for backtracking while performing decoding.
It's a tight PR that fits the existing codebase and works well, you just need a motivating example you can reproduce which can help me you quickly determine if the proposed solution is working. I usually generate 2-3 solutions initially and then filter them quickly based on a test case. And as you can see from the prompt, it's far from well formatted or comprehensive, just "slap dash" listing of potentially relevant information similar to what would be discussed at an informal whiteboard session.
I insta reject. It’s ridiculous how bad it’s made some devs. Are they even testing their work before tossing over the wall?
were they before?
Yep. Claude/copilot definitely has made people lazy.
I would have a talk to whoever create a PR without testing anything.
Hold up, people are starting to mark PRs with [vibe] as in “I don’t stand behind this, good luck.” ??
I do not care if engineers on my team are using AI. In fact, I think they should be. This is the new world and we need to get used to it.
But it’s still your work, your responsibility. You still have to own it. No cop outs.
Recently in that exact situation.
Junior dev vibe codes some slop to solve a problem that was barely a problem. The "solution" to that not-a-problem was 2500 lines of slop. Unused variables, spaghetti logic, unit tests that were clearly write-once, read-never.
The slop at that point is the meta-problem, the real problem becomes me trying to convince (through multiple rounds of code review) that this was not maintainable code, and not a tenable development model.
All those rounds of review take TIME and mental effort. At some point, when the code review takes more effort than the code owner contributed, the value proposition is shot.
Generate the code however you want (llm, bit twiddling, whatever), but the effort and care must be there. If you can't use an llm skillfully and vouch for the output, it's gonna be a no from me dawg
We need to normalize to reject crap PRs. If we don't then things will only continue.
lol if I saw a teammember create a PR with "[vibe]" in the title I would: 1) Gape in disbelief 2) Shout obscenities as loud as I can 3) Take a deep breath 4) reject the PR and seriously consider firing them.
Yeah "consider firing" is a bit of an extreme kneejerk reaction, but I just feel like we HAVE to create consequences for this nonsense.
> and find 400 lines of obviously incorrect slop
I call that a good day. I've seen people push 2000 line PRs. The worst was 5000 lines. FML.
I don’t mind a 2000 line PR if it’s quality code written by a human.
Have your own agent do first pass code reviews, they catch that stuff every time.
And then the PR owner's agent can fix it and then after some number of iterations you get a PR for a new and mysterious system.
It's only new and mysterious if you don't have a tight spec and guardrails.
There seems to be a gap between the theory you're advocating...which I actually agree with and the practical execution in your own public projects which you talk about heavily.
I haven't been able to get any of your recently published projects (Scribe, volknut, a few others) running successfully on linux, whether following your documentation, using the tagged releases, or reverse engineering the true functionality and CLI arguments working from your provided source code I found to have wasted my time.
It's difficult to believe you when your own documentation is entirely wrong or 404'd.
I'd genuinely love to see your projects in action, since the concepts sound promising, but you need to back it up with shipping properly.
There's a reason I haven't dropped release pages yet. These projects are under heavy development and I push to github to checkpoint mostly rather than release. I'm sorry if you feel like I've misled you, I've tried to be clear that what I'm sharing now is to understand what I'm doing, to give visibility, and it's not ready yet. I'm committed to delivering great software, and when I tell you it's ready you can rest assured that it will work.
Understand that I'm one man working on 20 projects simultaneously, with 5+ under active development at any one moment, so release stabilization and cadence will take a little bit to lock in.
AI PR tarpit, anyone? :)
> they catch that stuff every time
This is why AI code review continues to be mostly useless in my experience. If you instruct an LLM to make PR comments it will come up with some, every time, even when none are warranted.
So give them a schema and get them to give a relevance score, then filter below a threshold. Works better than trying to get them not to reply.
Seems like sound advice, unfortunate that the developers of the review tools I've had brushes with did not take it.
I try to avoid participating in the bullshit economy as much as possible
I've sort of gotten on the bandwagon. I initially used AI auto complete at my previous job and liked it a lot as a better intellisense, but wouldn't have used it to write PRs -- for one I tried it a couple times and it just wasn't good enough.
My new job pushes cursor somewhat heavily and I gave it a try, it works pretty well for me although it's definitely not something I would rely on. I like being able to ask it to do something and let it go off and come back to it in a while to see how it did. For me, I think it makes it easier to start on changes by coming into something (that might be wrong and bad), but for me personally having something in a PR to start from is a nice mental hack.
If it did well enough on the initial attempt I'll try to stick with it to polish it up, but if it failed terribly I'll just write it by hand. Even when it fails it's nice to see what it did as a jumping off point. I do wish it were a bit less prone to "lying" (yada yada anthromorphization it's just tokens etc.,) though, sometimes I'll ask it to do something in a particular way (e.g., add foo to bar and make sure you X, Y, and Z) and it'll conclude (rightfully or not) that it can't do X, but then go on anyway and claim that it did X.
I wish it were easier to manage context switching in cursor though, as it is juggling IDE windows between git repo clones is a pain (this is true for everything though, so not unique to cursor). I wish I could just keep things running on a git branch and come back to them without having to manage a bunch of different clones and windows etc.,. I think this is more of a pain point with cursor since in theory it would allow you to parallelize more tasks but the tooling isn't really there.
edit: the starting point for this is probably worktrees, I remember reading about these a while ago and should probably check them out (heh) but it only solves the problem of having a bunch of clones sitting around, I'd still need to manage N windows.
Idk what most people are doing but coding with AI requires special skillset.
Here I put ClaudeCode in a while loop and clone itself: https://www.reddit.com/r/ClaudeCode/s/TY3SNp7eI9
After a week it was ready, you just need a good prompt.
People who say it cannot create anything beyond simple things are wrong. In my experience you can create anything provided your plan is good.
One improvement is to start using an AGENTS file. There are many. Here's the repo that I personally use thanks to Pontus Abrahamsson and using TypeScript:
https://github.com/pontusab/directories
If you prefer AGENTS.md files using markdown, I've extracted them into my own repo:
https://github.com/SixArm/ai-agents-tips
Using AI assistants != Vibe Coding.
AI can be a helpful assistant but they are nowhere near ready for letting loose when the results matter.
Exactly this, if you're babysitting the AI, you are, by definition, _not_ vibe coding. Vibe coding means not reading the resulting code, and accepting that things will break down completely in four or five iterations.
Brother most of them ain't even assisting. Management just forces it.
Well you're now having to 'coach' an AI into getting it to do what you want rather than a junior employee.
And yes, you need to develop your skills in asking for what you want, whilst still having to review the outputs similar to a junior employee. It just doesn't really learn or get better based on your feedback.
Why do the news need to make everything sound sensational. Let's move on.
We're training our own replacement.
https://arstechnica.com/tech-policy/2017/03/public-universit...
I for one am absolutely sick and tired of hearing opinions about AI coding from people selling tools and solutions that use AI coding.
These are the questions I ask engineers now:
Q1. How well do you prompt?
Q2. How reactionary are you to canceling a running prompt as soon as you see a wrong turn?
Q3. How well do you author PRs without ever saying "well AI did that part." Do you own your PRs?
Re: 1. It's a new skill set. Some engineers are not good at it. Some are amazing. And some are out of this world. You can spend a few minutes crafting a prompt that's so thorough, so well thought out but also succinct and to the point. How it touches on all the possible unknowns the AI might have as it works through the task and has direction. An out of this world prompt gets you between 90% and 95% completely done with a task that before AI would take weeks.
Re: 2. You can do very well with the first prompt but the whole other side of this is watching the AI work. Knowing when to watch very closely, and when it's okay to multi-task. Know when you immediately press ESC and tell the AI exactly what you just saw you didn't like. Catching it quickly is like early detection of cancer. You can save the rest of your day by preventing some messy dead ends before they even happen.
Re: 3. Not everyone on the team will be using AI to the level the Jedi are. And personal responsibility for your code, 90% AI generated or 0%, is still your responsibility. Once you make your PR and ask for feedback, you should no longer be able to say "oh I didn't write this part." Yes you did. No one on the team needs to know what % AI your PR was. And you don't need to know theirs.
Regarding point 1 - when you say "a few minutes," I'm wondering if we're talking about the same thing? I spent two solid months with Claude Max, before they imposed limits, running multiple Opus agents, and never once got anywhere close to "weeks of work" from a single prompt. Not even in the same zip code.
So I'm genuinely asking: could you pretty please share one of these prompts? Just one. I'd honestly prefer discovering I can't prompt my way out of a paper bag rather than facing the reality that I wasted a bucketload of time and money chasing such claims.
let's work backwards. Send me a task you agree is weeks of work and then I'll show you a prompt.
So basically your source is: "trust me bro, I'll prove it to you?" I think it is a relevant question. We cannot go around calling someone genius prompt engineers, and then skip the engineering part of noting down what actually works and how it can be replicated. Could we try to work backwards not from the perspective of the problems of the person asking the question, but perhaps work backwards from your claims (I.e. to where you have them from)
not at all. I'm happy to provide the prompts. Let's just agree on what 2+ weeks coding problem we are trying to solve first. If I just pick one and show you the prompt I used you'll say "no way would that have taken over 2 weeks."
A few minutes crafting a prompt to save weeks
Thats like a 2000x productivity boost. AI is amazing
Not sure I agree with 3.
I can have Claude write stuff that does basically the right thing and actually works but, as so many people say, it's like working with a junior dev. The only way I'm going to get code I'd want to present as my own is by sitting looking over their shoulder and telling them what to do constantly, and with a junior dev and with Claude (etc) that takes longer than just writing it myself.
AI-coded/assisted work should be flagged as such so everyone knows that it needs extra scrutiny. If we decided to let a horde of junior devs loose on a codebase we'd naturally pay more attention to PRs from new intern Jimmy than revered greybeard Bob and we shouldn't let Claude hide behind Bob's username.
sounds like bad 1 and then bad execution of 2. If you do 1 and 2 well you get a PR for 3 that is NOT like working with a junior dev.
1 Is easy enough for trivial tasks but in a complex (typically horrible) production codebase nearly all the work is investigation and debugging. However good the initial prompt is, soon the context becomes flooded with log output and code and the LLM goes off the rails quite quickly.
Doing 2 well is the AI babysitting mentioned in the article. Of course you can stop it every minute and tell it to do something else, then watch it like a hawk to make sure it does it right, then clear context when it ignores you and makes the mistake you told it not to make. But that is often then slower than just doing the work yourself to begin with, probably leading to the findings we've all seen that LLM use is actually reducing productivity.
I think living with crappy AI code is the price we currently have to pay for getting development done quicker. Maybe in a year it'll have improved enough that we can ask it to clean up all the mess it made.
(Possibly I just have higher standards than most, other humans can be quite bad too.)
"all the work is investigation and debugging" - Yes! Exactly you can ask the AI a bunch of questions first and really dig into what the codebase currently does. Then spend the time crafting that prompt that explains how to surgically do what needs to be done. If you are watching it every min like a hawk you are doing it wrong. You need to watch it more like a VERY smart junior dev and trust but verify. I'm not saying it's easy to get good at these new skill sets. But simply throwing your hands up and saying "I'll just walk everywhere vs using a bicycle" isn't a strategy that's going to work well.
It makes me wonder if the golden age of programming is over.
I for one am sick and tired of hearing opinions about AI coding from people selling tools and "solutions" using AI. It's as if the entirety of tech media is just one massive ad.
I know that it's always been this way to some degree, but this is getting out of hand.
Everyone suggesting maintaining an agent.md file haven’t met the majority of their coworkers who refuse to document anything or read documentation. Perhaps train a model that can generate these agent files for them?
IMO the best possible outcome of this AI hype is better documentation.
It's proven that (agentic) LLMs work better and more efficiently when they have the proper context for the project (AGENT(S).md), the documentation is accessible and up to date (docs/ directory with proper up to date markdown) and the tooling is smooth (git hooks that prevent secrets from being pushed, forced style checking and linting).
...surprise! All of this makes things easier for actual humans too!
I think we’ll just get more documentation not better documentation. There never was a problem generating documentation from code. It’s the business context and use case documentation that’s hard to generate.
There never was a problem generating documentation from code
That doesn't match my experience at all. Before AI, generating documentation from code was either getting some embedded comments for a function call, or just list a function's arguments.
AI reads the implementation, reads the callers and callees, and provides much smarter documentation, with context. Something sorely lacking from previous solutions. Is it sometimes not completely accurate? Probably. But still a giant step forward.
I think it depends on how good your documentation generator was before. Finding all the callers and callees is not a new idea to LLMs.
Don’t most of these tools create them for you? Just run “/init” and the AI makes the initial agents.md file.
Dunno how helpful that is, but they shouldn’t have to write it from scratch.
I love maintaining my agent file. Yeah, it doesn’t always listen to the instructions, but neither does anyone else.
You are touching on a critical point that I have been amused by at times in these types of conversations; people are hating on AI for doing all the things that people have done and still do.
Poor planning, check. No/inadequate documentation, check. Sloppy and janky code, check. Poor practices and methods, check. Poor and miscommunication, check. Poor technical choices, check. What am I missing?
Maybe this is just a matter of some of the top tier 100x devs and teams clutching pearls in disgust at having to look at what goes on below Mt Olympus, but this is also not any different to how code quality cratered and is still really poor due to all the outsourcing and H-1B (sorry, all you H-1B hopefuls) insourcing of quantity over quality.
I say that without any judgement, but reality simply is that this issue has long been a quantity over quality argument even before AI, and mostly for non-dev reasons as the recent de-qualification of R&D funding revealed and had a marked impact on dev jobs because the C-suite could don't use R&D funding for financial shenanigans.
If people want to hate AI, go ahead. People hated and hate on the H-1B abuses and they hate on AI now. I would hope that we can just move beyond griping and mean-girling AI, and get to a point where proper practices and methods are developed to maybe make the outcomes and outputs better.
Because again, AI is not going anywhere less than even H-1B and I am sure the C-suite will find some new way to abuse and play financial shenanigans, but it's simply not going away and we need to learn to live with it since it will seemingly only get "better" and faster as it changes at breakneck speeds.
PR fatigue is a real thing
Related:
The Software Engineers Paid to Fix Vibe Coded Messes
https://news.ycombinator.com/item?id=45235458
An advertisement, where the protagonists weigh the pros and cons and then come down on the side of "paying the innovation tax".
Fastly profits from"AI" just like Cloudflare (or should we say "Claudeflare").
This selection of developers does seem representative at all. The new strategy is to acknowledge "AI" weaknesses but still adamantly claim that it is inevitable.
And yet it is still so hard to get something sufficiently complete to launch, even with AI.
AI just makes ambition and competition go up.
Not really surprising
AI commentary has been super interesting
Kind of wild watching engineers, technologists, etc clutch their pearls
AI makes mistakes - so do people - I’ve been paid many a time to go clean up the non-AI generated code humans left behind. I’ve spent countless hours troubleshooting dumb bugs generated by humans
AI brings its own set of problems but I think some people just don’t want to hear what a net benefit it is
Post after post lamenting that no machine will ever replace the human touch is so old - people have been saying that about new technology since the beginning of time
I say embrace it - great to talk about its problems objectively and how to avoid common issues, but it’s tiresome to hear all the reasons why it’s crap all the time
I would instead love articles (you do see them here sometimes) that go over how people coded complex solutions to problems using AI and the challenges they faced along the way
Knowing where the weak points are or where it may be prone to error
If you don’t embrace it, pretty soon kids coming out of college that grew up with it that contrary to popular DO understand fundamentals are going to be running circles around the people who haven’t yet figured out how to get value out of it - while you’re out there saying it’s crap and only idiots use it, these kids are going to be moving mountains - I can’t wait to see the cool stuff people build - I don’t see the future as bleak
Where does this vibe term come from. There is no fucking vibe here. Just ask question to AI service and then check the results. Saves ton of time but one must be vigilant and be able spot bullshit. Often the solution is correct but code is very inefficient. Luckily in many cases AI can correct based on subsequent critique / input.
The meaning has changed very quickly for some reason, but I think the original meaning was that you evaluate the result in a superficial manner by checking if the program seems to work fine. So "vibe" here is as opposed to any methodical or formal approach. No code review etc.
Coined by Andrej Karpathy
It comes from people not checking the results thoroughly and taking a YOLO approach to it.
[flagged]
[flagged]