In my experience there is something beneficial about having a different context review and prompted differently, but it's not used very well in practice. Your previous context does create its own blind spot, but you're also getting at a problem in non AI reviews which is "what am I reviewing for?". Some companies like Google have traditionally had very tight guidelines for what a PR reviewer is supposed to criticize and what not to worry about, I don't think AI code review is much different on this topic
With that, we're back to square one again. Why not just provide that set of guidelines, prompts, and skills locally, which is very likely something modern LLMs can already handle?
Can't really see the point of settings up a separate AI code review step and make that into some sort of stage that everybody pretends to care. Just to make it look like something extra is, hopefully, happening?
Plus, I feel like any previous context a developer had before submitting the PR must have the most accurate context rather than a code review AI.
Merging blindly is bad for you as the owner of the code, LLMs are not that good yet, we’re still finding it make mistales and write slop and our responsibility as engineers is to take ownership and verify it.
The AI reviewers just make this easier for us, I’m not talking about the walls of text it adds as it is exhausting to read (I know) but the fact that it could catch real bugs before you even read the actual code is the benefit, and I’ve integrated this flow into my routine, a loop of a coder and a reviewer taking turns before handing me the results to read myself.
The more I find stuff, the more I improve my own code review skill. And I actually created my initial skill by distilling the code reviews and comments I made myself on github for the past 2 years, training an AI on how I review and give feedback on PRs, that produced a skill that is like another copy of me reviewing the code and refining the code before I read it myself.
In my experience there is something beneficial about having a different context review and prompted differently, but it's not used very well in practice. Your previous context does create its own blind spot, but you're also getting at a problem in non AI reviews which is "what am I reviewing for?". Some companies like Google have traditionally had very tight guidelines for what a PR reviewer is supposed to criticize and what not to worry about, I don't think AI code review is much different on this topic
With that, we're back to square one again. Why not just provide that set of guidelines, prompts, and skills locally, which is very likely something modern LLMs can already handle?
Can't really see the point of settings up a separate AI code review step and make that into some sort of stage that everybody pretends to care. Just to make it look like something extra is, hopefully, happening?
Plus, I feel like any previous context a developer had before submitting the PR must have the most accurate context rather than a code review AI.
It's more on pure curiosity than ranting.
I dont use ai code reviewers. I personally use this tool very frequently before making any commit to github :- https://github.com/mukundzha/avouch
Merging blindly is bad for you as the owner of the code, LLMs are not that good yet, we’re still finding it make mistales and write slop and our responsibility as engineers is to take ownership and verify it.
The AI reviewers just make this easier for us, I’m not talking about the walls of text it adds as it is exhausting to read (I know) but the fact that it could catch real bugs before you even read the actual code is the benefit, and I’ve integrated this flow into my routine, a loop of a coder and a reviewer taking turns before handing me the results to read myself.
The more I find stuff, the more I improve my own code review skill. And I actually created my initial skill by distilling the code reviews and comments I made myself on github for the past 2 years, training an AI on how I review and give feedback on PRs, that produced a skill that is like another copy of me reviewing the code and refining the code before I read it myself.
Cut velocity and increase supervision
Nope I always ignore it
[dead]
[flagged]