Skip Navigation

rook

@ rook @awful.systems

Posts
1
Comments
193
Joined
3 yr. ago

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 28th September 2025 - awful.systems

    Jump

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 28th September 2025 - awful.systems

    Jump

  • Does ruby just die now?

    Part of the background to this issue is the development of rv which apparently offers a future where rubygems is much less important, and some folk seem to be taking that as a threat.

    Whether or not the new tooling delivers, the rubygems debacle has probably helped the new project considerably.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 21st September 2025

    Jump

  • Haven’t read the source paper yet (apparently it came out two weeks ago, maybe it already got sneered?) but this might be fun: OpenAI admits AI hallucinations are mathematically inevitable, not just engineering flaws.

    Full of little gems like

    Beyond proving hallucinations were inevitable, the OpenAI research revealed that industry evaluation methods actively encouraged the problem. Analysis of popular benchmarks, including GPQA, MMLU-Pro, and SWE-bench, found nine out of 10 major evaluations used binary grading that penalized “I don’t know” responses while rewarding incorrect but confident answers.

    I had assumed that the problem was solely technical, that the fundamental design of LLMs meant that they’d always generate bullshit, but it hadn’t occurred to me that the developers actively selected for bullshit generation.

    It seems kinda obvious in retrospect… slick bullshit extrusion is very much what is selling “AI” to upper management.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 21st September 2025

    Jump

  • Woke up to some hashtag spam this morning

    AI’s Biggest Security Threat May Be Quantum Decryption

    which appears to be over of those evolutionary “transitional forms” between grifts.

    The sad thing is the underlying point is almost sound (hoarding data puts you at risk of data breaches, and leaking sensitive data might be Very Bad Indeed) but it is wrapped up in so much overhyped nonsense it is barely visible. Naturally, the best and most obvious fix — don’t hoard all that shit in the first place — wasn’t suggested.

    (it also appears to be a month-old story, but I guess there’s no reason for mastodon hashtag spammers to be current 🫤)

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 7th September 2025

    Jump

  • One to watch from a safe distance: dafdef, an “ai browser” aimed at founders and “UCG creators”, named using the traditional amazon-keysmash naming technique and and following the ai-companies-must-have-a-logo-suggestive-of-an-anus style guide.

    Dafdef learns your browsing patterns and suggests what you'd do next After watching you fill out similar forms a few times, Dafdef starts autocompleting them. Apply with your startup to YC, HF0 and A16z without wasting your time.

    So… spicy autocomplete.

    But that’s not all! Tired of your chatbot being unable to control everything on your iphone, due to irksome security features implemented by those control freaks at apple? There’s a way around that!

    Introducing the “ai key”!

    A tiny USB-C key that turns your phone into a trusted AI assistant. It sees your screen, acts on your behalf, and remembers — all while staying under your control.

    I’m sure you can absolutely trust an ai browser connected to a tool that has nearly full control over your phone to not do anything bad, because prompt injection isn’t a thing, right?

    (I say nearly full, because I think Apple Pay requires physical interaction with a phone button or face id, but if dafdef can automate the boring and repetitive parts of using your banking app then having full control of the phone might not matter)

    h/t to ian coldwater

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 22nd June 2025

    Jump

  • Bad brainwaves: ChatGPT makes you stupid

    Jump

  • It isn’t clear to me at this point that such research will ever be funded in english-speaking places without a significant set of regime changes… no politician or administrator can resist outsourcing their own thinking to llm vendors in exchange for funding. I expect the US educational system will eventually provide a terrible warning to everyone (except the UK, whose government looks at the US and says “oh my god, that’s horrifying. How can we be more like that?”).

    I’m probably just feeling unreasonably pessimistic right now, though.

  • Bad brainwaves: ChatGPT makes you stupid

    Jump

  • Google's Gemini 2.5 pro is out of beta.

    Jump

  • It is related, inasmuch as it’s all generated from the same prompt and the “answer” will be statistically likely to follow from the “reasoning” text. But it is only likely to follow, which is why you can sometimes see a lot of unrelated or incorrect guff in “reasoning” steps that’s misinterpreted as deliberate lying by ai doomers.

    I will confess that I don’t know what shapes the multiple “let me just check” or correction steps you sometimes see. It might just be a response stream that is shaped like self-checking. It is also possible that the response stream is fed through a separate llm session when then pushes its own responses into the context window before the response is finished and sent back to the questioner, but that would boil down to “neural networks pattern matching on each other’s outputs and generating plausible response token streams” rather than any sort of meaningful introspection.

    I would expect the actual systems used by the likes of openai to be far more full of hacks and bodges and work-arounds and let’s-pretend prompts that either you or I could imagine.

  • Google's Gemini 2.5 pro is out of beta.

    Jump

  • It’s just more llm output, in the style of “imagine you can reason about the question you’ve just been asked. Explain how you might have come about your answer.” It has no resemblance to how a neural network functions, nor to the output filters the service providers use.

    It’s how the ai doomers get themselves into a flap over “deceptive” models… “omg it lied about its train of thought!” because if course it didn’t lie, it just edited a stream of tokens that were statistically similar to something classified as reasoning during training.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 22nd June 2025

    Jump

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 22nd June 2025

    Jump

  • I might be the only person here who thinks that the upcoming quantum bubble has the potential to deliver useful things (but boring useful things, and so harder to build hype on) but stuff like this particularly irritates me:

    https://quantumai.google/

    Quantum fucking ai? Motherfucker,

    • You don’t have ai, you have a chatbot
    • You don’t have a quantum computer, you have a tech demo for a single chip
    • Even if you had both of those things, you wouldn’t have “quantum ai”
    • if you have a very specialist and probably wallet-vaporisingly expensive quantum computer, why the hell would anyone want to glue an idiot chatbot to it, instead of putting it in the hands of competent experts who could actually do useful stuff with it?

    Best case scenario here is that this is how one department of Google get money out of the other bits of Google, because the internal bean counters cannot control their fiscal sphincters when someone says “ai” to them.

  • Disney sues AI image generator Midjourney

    Jump

  • LLMs aren’t profitable even if they never had to pay a penny on license fees. The providers are losing money on every query, and can only be sustained by a firehose of VC money. They’re all hoping for a miracle.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 25th May 2025

    Jump

  • I like that Soylent Green was set in the far off and implausible year of 2022, which coincidentally was the year of ChatGPT’s debut.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 25th May 2025

    Jump

  • I am absolutely certain that letting a hallucination-as-a-service system call the police if it suspects a user is being nefarious is a great plan. This will definitely ensure that all the people threatening their chatbots with death will think twice about their language, and no-one on the internet will ever be naughty ever again. The police will certainly thank anthropic for keeping them up to date with the almost certainly illegal activities of a probably small number of criminal users.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 23 February 2025

    Jump

  • He’s right that current quantum computers are physics experiments, not actual computers, and that people concentrate too much on exotic threats, but he goes a bit off the rails after that.

    Current post quantum crypto work is a hedge, because no-one who might face actual physical or financial or military risks is prepared to say that there will be no device in 10-20 years time that can crack eg. an ECDH key exchange in the blink of an eye. You’ve got to start work on PQC now, because you want to be able subject it to a lot of classical cryptanalysis work because quantum-resistant is no good by itself (see also, SIKE which turned out to be trivially crackable).

    The attempt to project factorising capabilities of future quantum computers is pretty stupid because there’s too little data to work with, so the capabilities and limitations of future devices can’t usefully be guessed at yet. Personally, I’d expect them to remain physics experiments for at least another 5-10 years, but once a bunch of current issues are resolved you’ll see rapid growth in practical devices by which time it is a bit late to start casting around for replacement crypto systems.

  • OpenAI is so cooked and I'm all here for it

    Jump

  • A real ceo does everything. Delegation is for losers who can’t cope. Can’t move fast enough and break enough things if you’re constantly waiting for your lackeys to catch up.

    If those numbers people were cleverer than the ceo, they’d be the ones in charge, and they aren’t. Checkmate. Do you even read Ayn Rand, bro?