AI-authored code needs more attention, contains worse bugs
: CodeRabbit review of pull requests shows meatbags beat clankers
in other news, the North Pole is colder than the Indonesian rainforest
Bad article, the number of issues aren't normalized to anything other than PRs. I expect AI authorized PRs to be somewhat bigger than a fully human authorized one.
Ideally they'd compare time to write + time to fix. My experience is that if you use test driven development, LLM isn't too bad. No worse than an intern.
I think it comes down to who is using the LLM. I had a junior dev once "presumably" AI gen a ton of code (broken trash). Then to fix it, they wrapped each function in a try catch block that dropped the error. Unit tests were mocked out to the extent they didn't test anything.
When I use an LLM, I have tests and hard constraints on the LLM. It isn't good enough to do everything, but it can generate about 80% of a simple app
While I can get speedups for rapid prototypes where the code doesn’t need to be good, just mostly working, to get actually good code it’s inevitably significantly faster for me to write everything than to fix what an LLM starts.
For LLM generated code, it can also take a whole to read and understand. When I write code myself, I understand the intention, architecture, and so on. Machine written code is very different. I need to understand how it works. There’s often extraneous stuff in there or weird patterns.
I fully concur with the intern metaphor. That's what an LLM is - an intern with VAST amounts of lexical knowledge but little understanding of large scale system design, or generally, experience.
LLM generated code is mostly okay. You do have to review it, like you would an intern's contributions, but generally, it's good quality code and well documented.
It's also kinda good¿ For writing scripts and write regex, specially claude !
If have some very basic knowledge in bash and not enough lifetime to invest into a 1 year full bash course, python or regex patterns... It's already very time consuming to self-host everything :/.
And while it mostly does what I want/need I also learned a few cool bash tricks i wasn't aware... Like how to write and use functions !
Why would you expect that?
Because an LLM can generate a lot of (garbage?) code quickly.
I wonder how long until the curves cross.
it'll be when there's no one left to point out the bugs.
But does it need Flash Attention, Sage Attention, or Scaled dot product Attention?
Kinda funny the juxtaposition between the programmers' reaction to this compared to the "techies" reaction on the crosspost.
Maybe we're still early yet so I'll write the difference right now for posterity: Programming post is generally critical of the article and has several suggestions on how to improve the quality of agent-assisted code.
Technology post is pretty much just "REEEEEEEEEEE AI BAD"
I bet if you get in a hot tub that's a bit too warm you'll go "Ooh, that's a bit to warm".
And I bet if you jump into a boiling tub you'll scream "REEEEEEE HOOOOOT".
It's a matter of degrees, you see?
I'm not sure I get your analogy. This is more to me like two people got into a bath and one went "Ooh, that's a bit too warm" while the other screamed "REEEEEEE HOOOOOT”. The degree is the same. The response is not.
This article does not show as cross posted to any other community in the Voyager app.
EDIT: When I open the post in a web browser, the cross posts are visible. 🤷♂️
I seem to have missed those comments, could you point any out?
