A weird phrase is plaguing scientific papers – and we traced it back to a glitch in AI training data
A weird phrase is plaguing scientific papers – and we traced it back to a glitch in AI training data
The phrase is "vegetative electron microscopy"
The lede is buried deep in this one. Yeah, these dumb LLMs got bad training data that persists to this day, but more concerning is the fact that some scientists are relying upon LLMs to write their papers. This is literally the way scientists communicate their findings to other scientists, lawmakers, and the public, and they're using fucking predictive text like it has cognition and knows anything.
Sure, most (all?) of those papers got retracted, but those are just the ones that got caught. How many more are lurking out there with garbage claims fabricated by a chatbot?
Thankfully, science will inevitably sus those papers out eventually, as it always does, but it's shameful that any scientist would be so fatuous to put out a paper written by a dumb bot. You're the experts. Write your own goddamn papers.
In some cases, it’s people who’ve done the research and written the paper who then use an LLM to give it a final polish. Often, it’s people who are writing in a non-native language.
Doesn’t make it good or right, but adds some context.
Adding extra polish like nonsense phrases. Nobody is supervising it then.
Thankfully, science will inevitably sus those papers out eventually, as it always does,
In the future, all search engines will have an option to ignore any results from 2022-20xx, the era of AI slop.
oh yea,not to mention alot of papers tend to be low quality before the AI was used, ive been hearing people are writing dozens of papers just to fluff up thier resume/cv. it was quanitity over quality. i was in an presentation where the guy presenting thier research wrote 40+ papers just to get hired a university somewhere.
The scientific community needs to gather and reach a consensus where AI is banned from writing their papers. (Yes, even for translation)
Don't use fucking AI to write scientific papers and the problem is solved. Wtf.
More salient takeaway is, don't use a LLM to translate a scientific paper. Because it can't translate a scientific paper. It can only rewrite the entire paper, in a different language. And it will introduce misunderstandings and hallucinations.
GIGO overcomes all
Let’s delve into the issue
Super interesting. But also, super boring.