My personal experience: I shipped multiple features at work in the past 6 months that I simply wouldn’t have tried shipping otherwise, since my day job is mostly management. AI wrote maybe 80% of the code, I spent a bit of time rewriting some parts. No major bugs so far (ironically, the one big bug the team had to revert was done entirely by me)
I can guarantee I wouldn’t have shipped ANY of it, since it’d require focus blocks I simply don’t have on the job.
I’m also about to ship a Mac app that’s heavily vibecoded. I wouldn’t even try without AI, since I’m not a Swift developer.
Those aren’t “illusions” of performance. I imagine it’s hard to gauge every single scenario, and sensationalist takes like this research elicit an emotional response on the anti-AI crowd, but denying the impact is simply ignorance at this point…
Given that your job is not software development but management, you spending time delivering features is effectively removing time from doing your job.
If you had spent managing the same time you spent vibe-coding, maybe it would have been a force-multiplier for your reportees and your team might have been more productive as a whole than the added productivity of your vibe-coding.
I tend to agree here. I haven't been allowed to code for a few years now but I spend a lot of my time talking through code with developers. I find many of the people on my teams lack a perspective I can provide to frame a problem or evaluate an approach.
I also help them get to the heart of problems quickly simply because I'm not stuck in the code all day. For example, if I see a developer taking too long to identify the source of a bug, I'll get on a call and get them to take me through that code and get them prove any assumption ("ok, show me the code that checks that value is greater than zero").
By doing this I'm using my coding experience directly without actually coding. I'd consider coding a huge waste of time for me, but spending 30 minutes to unstick a developer when I am sure they should have found the problem by now seems like a really good use of my time.
It also lets people know they can't just spend three days on something that should take a couple hours without someone checking in, which I don't live having to do but it's a reality for some teams I work with.
> Given that your job is not software development but management, you spending time delivering features is effectively removing time from doing your job.
It most definitely isn’t. With all due respect, I know my job and my schedule more than you and your baseless assumptions.
In my experience, I don’t think hallucinations are a big problem anymore in terms of coding as long as you work within your domain of expertise.
The perception that AI tools make development faster is perhaps due to the part we spend a lot of time with thinking about how to write (like commenting) is solved instantly.
I think a lot of the delay is that it’s a new class of tool, and just like last gen IDE it takes a bit of getting used to and know where their strengths are, and know how to effectively fit it into your workflow.
My personal experience is that in well and quickly autocompleted languages I know well (python) then I am roughly the same speed, I'd say...it's just a bit less annoying having AI do simple boilerplate for me, but sometimes annoying whenever I try to use it for larger refactoring where it gets style and structure incorrect.
In languages and libraries I know less well - vuejs+myriad of (especially) js libraries, I would say I'm much faster, especially as I delegate more style and structure to the AI.
Absolutely every layer of abstraction reduces boilerplate, but it's not as if it's entirely gone. I should say, I use boilerplate as a euphemism for 'trivial code'. For example recreating backed request models in the frontend, a simple "create db record" service function, etc.
I mostly use vuejs at the front and fastapi for the backed, and unless I'm missing something big, there is still plenty of 'trivial code' that I find is done a bit faster.
bug creation rates of AI trival code vs my own trivial code has not been evaluated
Lots of HN discussion here - https://news.ycombinator.com/item?id=44522772 - it's an interesting paper but there are reasons we shouldn't make strong claims from it. I would say the most accurate reading would be "experienced developers but inexperienced Cursor users overestimate their immediate productivity gains".
If any of those 16 developers were at the by some allegedly 20x productivity with AI models it would have been positive on average, but none were even close to that.
So I feel its fairly safe to say that models aren't even close to 10x productivity gain for average developers, so developer jobs are not really in jeopardy so far. If it was easy to be more productive with AI models then this study would have found that, the only productivity gain this study could have missed would be if it was really hard or if the gain was really small, and both of those means it wont replace most developers.
There's no way its true. Unless the devs simply don't know how to use AI.
I've been using roocode for about 6 months now and I automated everything. It does in one night what would take 2-3 months by hand. There's no way its not helping good devs who can prompt ai well.
I think it's because of the tasks they did. For run of the mill custom development projects AI certainly speeds things up a great lot. But when i tried to use it for comparatively hard tasks, i found it easier to do things by myself. It was nothing too fancy, just bitwise image manipulation - a custom code for pixel format conversion combined with image resizing that had to work a lot faster than ffmpeg's, but in a narrow set of conditions - no miracles here, it's not at all smarter than ffmpeg's, just specialised. 100% hand-made code took less time to build and was just as fast to execute. Of course, it used intrinsics.
The circumstances are exactly where AI is quite bad, and experienced humans very good - large, complex existing codebase, working on complex, possibly nuanced changes/fixes involving a lot of context, etc.
With all due respect to the people who have studied this, I do not care. I'm relatively young, and yet the pain of typing has started to become unmistakeable and chipped away at my productivity, motivation, and sense of invincibility.
You could tell me AI coding makes me 50% slower. I'm taking it. I refuse to grind my wrists to dust.
I'll chip in with another datapoint that a seriously ergonomic keyboard helped me as well - in my case, the glove80. Expensive, and there might be a better option on the Pareto curve, but I got freaked out, bought the most ergonomic keyboard I could find, and it did solve my problem.
I've heard good things about voice typing with Talon too, but never tried it.
> and yet the pain of typing has started to become unmistakeable
As in _physical pain_? If you haven't already, go to a doctor; that's not normal (it's somewhat common, but it is far from inevitable and there are things you can do about it).
AI just produces very bad but working versions every time I try. Worse performance, more technical debt that I need to refactor, more code overall with less functionality that I need to trim away.
It gets a flawed but working version quicker, but it takes much longer to get to anything I can release.
AI mostly helps me discover new thing about a public API, or show one general solution to a problem, but then I mostly have to solve everything myself anyway to get anything good.
I know the anti AI folks would like to believe AI coding results are a mass hallucination but the completed software that I build in record time with deep functionality says otherwise.
Unless you've timed yourself doing the same task once with AI and once without, while having forgotten everything about said task in meantime, you don't have tangible real results that show otherwise.
You could repeatedly have developers execute tasks with and without ai (two different tasks. Each subject does the same 2 tasks. Randomize order and what task is done with and what task is done without ai) and show significant differences in duration.
You’d need probably at least 30 to 40 people (there’s ways to estimate this, but this is gut feeling from years ago when I did studies like this)
It would take on a few flaws in tfa which has a low number of people who operate in a codebase that they know very well on issues that they self-selected (and thus already will have a (perhaps subconscious) strategy for).
I have no interest in convincing skeptics. They’re welcome to do as they choose. Computing and programming should be about pleasing yourself and programming in the way you like.
My personal experience: I shipped multiple features at work in the past 6 months that I simply wouldn’t have tried shipping otherwise, since my day job is mostly management. AI wrote maybe 80% of the code, I spent a bit of time rewriting some parts. No major bugs so far (ironically, the one big bug the team had to revert was done entirely by me)
I can guarantee I wouldn’t have shipped ANY of it, since it’d require focus blocks I simply don’t have on the job.
I’m also about to ship a Mac app that’s heavily vibecoded. I wouldn’t even try without AI, since I’m not a Swift developer.
Those aren’t “illusions” of performance. I imagine it’s hard to gauge every single scenario, and sensationalist takes like this research elicit an emotional response on the anti-AI crowd, but denying the impact is simply ignorance at this point…
> Those aren’t “illusions” of performance.
Given that your job is not software development but management, you spending time delivering features is effectively removing time from doing your job.
If you had spent managing the same time you spent vibe-coding, maybe it would have been a force-multiplier for your reportees and your team might have been more productive as a whole than the added productivity of your vibe-coding.
This is absolutely an illusion of performance.
I tend to agree here. I haven't been allowed to code for a few years now but I spend a lot of my time talking through code with developers. I find many of the people on my teams lack a perspective I can provide to frame a problem or evaluate an approach.
I also help them get to the heart of problems quickly simply because I'm not stuck in the code all day. For example, if I see a developer taking too long to identify the source of a bug, I'll get on a call and get them to take me through that code and get them prove any assumption ("ok, show me the code that checks that value is greater than zero").
By doing this I'm using my coding experience directly without actually coding. I'd consider coding a huge waste of time for me, but spending 30 minutes to unstick a developer when I am sure they should have found the problem by now seems like a really good use of my time.
It also lets people know they can't just spend three days on something that should take a couple hours without someone checking in, which I don't live having to do but it's a reality for some teams I work with.
> Given that your job is not software development but management, you spending time delivering features is effectively removing time from doing your job.
It most definitely isn’t. With all due respect, I know my job and my schedule more than you and your baseless assumptions.
In my experience, I don’t think hallucinations are a big problem anymore in terms of coding as long as you work within your domain of expertise.
The perception that AI tools make development faster is perhaps due to the part we spend a lot of time with thinking about how to write (like commenting) is solved instantly.
I think a lot of the delay is that it’s a new class of tool, and just like last gen IDE it takes a bit of getting used to and know where their strengths are, and know how to effectively fit it into your workflow.
> In my experience, I don’t think hallucinations are a big problem anymore in terms of coding
Well, unless of course you are building low-risk software in which you don't care about it's correctness then sure.
> ...as long as you work within your domain of expertise.
But again, try tell that to the "vibe-coders" who get stuck when AI agents continue to insert bugs they cannot find.
My personal experience is that in well and quickly autocompleted languages I know well (python) then I am roughly the same speed, I'd say...it's just a bit less annoying having AI do simple boilerplate for me, but sometimes annoying whenever I try to use it for larger refactoring where it gets style and structure incorrect.
In languages and libraries I know less well - vuejs+myriad of (especially) js libraries, I would say I'm much faster, especially as I delegate more style and structure to the AI.
> do simple boilerplate
I just don't buy this.
Everytime you say AI is useless because it can't solve complex problems, people will bring up "oh but it writes boilerplate code for me".
How often do you exactly write boilerplate code??? Do you know what else writes boilerplate code for you? Libraries and framework.
Boilerplate code is a solved problem since way before GenAI was in the public eye.
Absolutely every layer of abstraction reduces boilerplate, but it's not as if it's entirely gone. I should say, I use boilerplate as a euphemism for 'trivial code'. For example recreating backed request models in the frontend, a simple "create db record" service function, etc.
I mostly use vuejs at the front and fastapi for the backed, and unless I'm missing something big, there is still plenty of 'trivial code' that I find is done a bit faster.
bug creation rates of AI trival code vs my own trivial code has not been evaluated
> For example recreating backed request models in the frontend
https://pypi.org/project/pydantic-to-typescript/
> a simple "create db record" service function
https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapp...
> unless I'm missing something big
You are missing a lot of things, yes.
Lots of HN discussion here - https://news.ycombinator.com/item?id=44522772 - it's an interesting paper but there are reasons we shouldn't make strong claims from it. I would say the most accurate reading would be "experienced developers but inexperienced Cursor users overestimate their immediate productivity gains".
I want to believe that this is true. But I don't.
They tested it with 16 developers, so not enough data for conclusive evidence.
If any of those 16 developers were at the by some allegedly 20x productivity with AI models it would have been positive on average, but none were even close to that.
So I feel its fairly safe to say that models aren't even close to 10x productivity gain for average developers, so developer jobs are not really in jeopardy so far. If it was easy to be more productive with AI models then this study would have found that, the only productivity gain this study could have missed would be if it was really hard or if the gain was really small, and both of those means it wont replace most developers.
They tested on 250 tasks, ultimately the number of developers doesn't matter that much here.
There's no way its true. Unless the devs simply don't know how to use AI.
I've been using roocode for about 6 months now and I automated everything. It does in one night what would take 2-3 months by hand. There's no way its not helping good devs who can prompt ai well.
I think it's because of the tasks they did. For run of the mill custom development projects AI certainly speeds things up a great lot. But when i tried to use it for comparatively hard tasks, i found it easier to do things by myself. It was nothing too fancy, just bitwise image manipulation - a custom code for pixel format conversion combined with image resizing that had to work a lot faster than ffmpeg's, but in a narrow set of conditions - no miracles here, it's not at all smarter than ffmpeg's, just specialised. 100% hand-made code took less time to build and was just as fast to execute. Of course, it used intrinsics.
The circumstances are exactly where AI is quite bad, and experienced humans very good - large, complex existing codebase, working on complex, possibly nuanced changes/fixes involving a lot of context, etc.
Ironically, this fell off the HN front page without enough upvotes...neither hackernews.coffee nor Claude suggested it to me...
Oh, itsdrewmiller pointed us in the right direction: https://news.ycombinator.com/item?id=44551579
If the task is large enough you will be faster w/ Claude Code, amp etc with current models.
With all due respect to the people who have studied this, I do not care. I'm relatively young, and yet the pain of typing has started to become unmistakeable and chipped away at my productivity, motivation, and sense of invincibility.
You could tell me AI coding makes me 50% slower. I'm taking it. I refuse to grind my wrists to dust.
I'll chip in with another datapoint that a seriously ergonomic keyboard helped me as well - in my case, the glove80. Expensive, and there might be a better option on the Pareto curve, but I got freaked out, bought the most ergonomic keyboard I could find, and it did solve my problem.
I've heard good things about voice typing with Talon too, but never tried it.
I've had similar issues, in the end switching to a split keyboard and vertical mouse/trackball helped immensely.
Try using AI in a useful manner then, as a voice or shorthand input method.
> and yet the pain of typing has started to become unmistakeable
As in _physical pain_? If you haven't already, go to a doctor; that's not normal (it's somewhat common, but it is far from inevitable and there are things you can do about it).
No. Way. AI is much Faster than I will ever be at debugging, and idea generation.
It's not about working faster or slower, it's getting it right in the most efficient way possible.
AI just produces very bad but working versions every time I try. Worse performance, more technical debt that I need to refactor, more code overall with less functionality that I need to trim away.
It gets a flawed but working version quicker, but it takes much longer to get to anything I can release.
AI mostly helps me discover new thing about a public API, or show one general solution to a problem, but then I mostly have to solve everything myself anyway to get anything good.
Lived experience says different.
This is literally what the study says: the developers said they felt they were faster after having completed the task, when in fact they were slower.
Your "lived experience" just follows the same bias.
Ok let me rephrase.
Actual tangible real results say different.
I know the anti AI folks would like to believe AI coding results are a mass hallucination but the completed software that I build in record time with deep functionality says otherwise.
> Actual tangible real results say different.
Unless you've timed yourself doing the same task once with AI and once without, while having forgotten everything about said task in meantime, you don't have tangible real results that show otherwise.
I feel LLMs save me time too but I worry my perceptions don't match the reality.
Got some data?
>> Got some data?
There is nothing I could say that would convince a skeptic.
You could repeatedly have developers execute tasks with and without ai (two different tasks. Each subject does the same 2 tasks. Randomize order and what task is done with and what task is done without ai) and show significant differences in duration.
You’d need probably at least 30 to 40 people (there’s ways to estimate this, but this is gut feeling from years ago when I did studies like this)
It would take on a few flaws in tfa which has a low number of people who operate in a codebase that they know very well on issues that they self-selected (and thus already will have a (perhaps subconscious) strategy for).
That would help convincing skeptics.
Didn't this study do that and proved the skeptics right? It is hard to convince skeptics when the skeptics are right.
I have no interest in convincing skeptics. They’re welcome to do as they choose. Computing and programming should be about pleasing yourself and programming in the way you like.
Or a believer.
How did you measure your before and after?