AI-Generated Code is Causing Outages and Security Issues in Businesses
Businesses using artificial intelligence to generate code are experiencing downtime and security issues, according to Sonar CEO.
You're viewing a single thread.
If I was still in a senior dev position, I’d ban AI code assistants for anyone with less than around 10 years experience. It’s a time saver if you can read code almost as fluently as you can read your own native language but even besides the A.I. code introducing bugs, it’s often not the most efficient way. It’s only useful if you can tell that at a glance and reject its suggestions as much as you accept them.
Which, honestly, is how I was when I was first starting out as a developer. I thought I was hot shit and contributing and I was taking half a day to do tasks an experienced developer could do in minutes. Generative AI is a new developer: irrationally confident, not actually saving time, and rarely doing things the best way.
I've found they're great as a learning tool where decent docs are available. Or as an interactive docs you can ask follow up questions to.
We mostly use c# and it's amazing at digging into the MS docs to pull out useful things from the bcl or common patterns.
Our new juniors got up to speed so fast by asking it to explain stuff in the existing codebases. Which in turn takes pressure off more senior staff.
I got productive in vuejs in a large codebase in a couple days that way.
Using to generate actual code is insanely shit haha It is very similar to just copy pasting code and hacking it in without understanding it.
You make a good point about using it for documentation and learning. That’s a pretty good use case. I just wouldn’t want young developers to use it for code completion any more than I’d want college sophomores to use it for writing essays. Professors don’t have you write essays because they like reading essays. Sometimes, doing a task manually is the point of the assignment.
What setup are you using for junior devs to be able to query your codebase?
I was talking more generally about LLMs here
Even worse than it being wrong, is that by nature of the tool it looks right.
Eh, I'm a senior dev, and I don't ban it (my boss, the director, does that for me lol; he's worried about company secrets leaking).
In fact, we had an interview for a senior dev position, and the applicant asked if they could use AI, and I told them to use whatever tools they normally would for development. It shouldn't come as a surprise that they totally botched the programming challenge because of it (introduced the same bug twice, then said they were very confident in the correctness of the code...), and that made it so much easier to filter them out from our hiring pool. If you're going to use a tool in an interview, you better feel confident with it. If that dev had solved the problem significantly faster than our other applicants, I would've taken that to my boss to have the team experiment with it. We target budget 30 min for our challenges, and our seniors generally finish in under 20, and it took them more than our allotted time to get the code to actually run properly (and that's with us pointing out certain mistakes the AI generated).
But no, I haven't seen an actually productive use of AI for software development, beyond searching for docs online (which you can totally do w/ Bing or Google w/o involving our codebase). You may feel more productive because more code is appearing on the screen, but the increase in bugs likely reduces overall productivity. We're always looking for ways to improve, but when I can solve the same problem in my bare-bones editor (vim) faster than my more junior colleagues can with their fancy IDEs, I really don't think AI is going to be the thing that improves our productivity, actually understanding logic will. If someone demonstrates that AI does save time, I'll try it out and campaign for it.
Anyway, that's my take as someone who has been in the industry for something like 15 years. Knowing your tools is more important, IMO, than having more tools.
I had my suspicions before but the moment I realized for certain Elon Musk couldn’t run a software company was when he judged people by lines of code written.
Ew, I would hate to be in charge of code reviews at an org like that.
The proper metric is success of the actual product. We have our engineers give estimates, then hold them to those estimates and evaluate based on consistency of on-time releases and number of production bugs. At the end of the day, predictable, high quality delivery is usually more valuable than faster time to market, unless you're in a startup or something and just need to get early adopters on-board. Judge QA by defects discovered in production and devs by defects found by QA and in production. It's really not that hard.
The one time some manager voiced such an idea, I very overtly in front of everybody offered to make "loop unrolling" software working at the source level (compilers already do it at the Assembly level in some cases for performance) for me and my colleagues to really boost that code line count (while totally screwing maintenability).
Mind you, all devs in that meeting were loudly against measuring performance by code lines, but I like to think that suggestion of mine really hammered down the coup the grace on that "brilliant" idea.
Not trying to defend him, but I thought the reasoning behind doing that was to get the least obedient people to leave the company so that there won't be a delayed push back from the employees.
In my experience working for almost 3 decades in software development, passive-agressive shit from upper management just causes the best people to leave (as they're the ones who easilly find better jobs) leaving behind mainly a mix of the incompetent and those who never worked anywhere else (who are either already incompetent or will become so, as only ever having worked in just one company is far too narrow professional experience for anything beyond junior/mid level - you need to have seen more than one way of doing things to understand certain higher level concerns and choices in software development).
Yeah and I'd say these people left are exactly those Elon wants, he doesn't want white guys in their 50s, he wants obedient young guys.
Sound like a variant of the good old saying "pay peanuts, get monkeys" only using a stick and threats instead of payment.
Mind you, it does sound like the kind of think somebody with his kind of personality - narcissistic shameless and dishonest salesman - would think it's a great idea.
I've worked as a freelancer (specifically as a Contractor) in Software Development for over a decade and more often than not I ended up having to work with some existing code base, having to deal with the design choices, coding style and bugs of somebody else, often multiple somebody elses.
There's nothing quite as "entertaining" as having to deal with 3+ different code and design styles in the same code base because all previous developer thought their own way of doing things was the superior way so just added one more layer of their style (not just coding but, worse, software design) on top of what was already there increasing the mess, rather than work within the existing structure and style and doing some refactoring.
Anyway, in my experience having to read, understand and work with existing code that you yourself did not made is way more time costly and less pleasant than actually doing your stuff from scratch.