Skip Navigation

User banner
InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)S
scruiser @ scruiser @awful.systems
Posts
3
Comments
243
Joined
2 yr. ago

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 31st August 2025 - awful.systems

    Jump

  • I have context that makes this even more cringe! "Lawfulness concerns" refers to like, Dungeons and Dragons lawfulness. Specifically the concept of lawfulness developed in the Pathfinder fanfiction we've previously discussed (the one with deliberately bad BDSM and eugenics). Like a proper Lawful Good Paladin of Iomedae wouldn't put you in a position where you had to trust they hadn't rigged the background prompt if you went to them for spiritual counseling. (Although a Lawful Evil cleric of Asmodeus totally would rig the prompt... Lawfulness as a measuring stick of ethics/morality is a terrible idea even accepting the premise of using Pathfinder fanfic to develop your sense of ethics.)

  • the quantum hype bandwagon is rolling out

    Jump

  • you can’t have an early version that you’ll lie about being a “big step towards General Quantum Computing” or whatever

    So you might think that... but I recall some years ago an analog computer was labeled as quantum annealing or something like that... oh wait, found the wikipedia article: https://en.wikipedia.org/wiki/Quantum_annealing and https://en.wikipedia.org/wiki/D-Wave_Systems . So it sounds to a naive listener like the same sort of thing as the quantum computers that are supposed to break cryptography and even less plausible things, but actually it can only do one very specific algorithm.

    I bet you could squeeze the "quantum" label onto a variety of analog computers well short of general quantum computing and have it technically not be fraud and still fool lots of idiot VCs!

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 31st August 2025 - awful.systems

    Jump

  • It's a nice master post that gets all his responses and many useful articles linked into one place. It's all familiar if you've kept up with techtakes and Zitron's other posts and pivot-to-ai, but I found a few articles I had previously missed reading.

    Related trend to all the but achskhually's AI booster's like to throw out. Has everyone else noticed the trend where someone makes a claim of a rumor they heard about an LLM making a genuine discovery in some science, except it's always repeated second hand so you can't really evaluate it, and in the rare cases they do have a link to the source, it's always much less impressive than they made it sound at first...

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 31st August 2025 - awful.systems

    Jump

  • Even for the people that do get email notifications of Zitron's excellent content (like myself), I appreciate having a place here to discuss it.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • Apparently Eliezer is actually against throwing around P(doom) numbers: https://www.lesswrong.com/posts/4mBaixwf4k8jk7fG4/yudkowsky-on-don-t-use-p-doom ?

    The objections to using P(doom) are relatively reasonable by lesswrong standards... but this is in fact once again all Eliezer's fault. He started a community centered around 1) putting overconfident probability "estimates" on subjective uncertain things 2) need to make a friendly AI-God, he really shouldn't be surprised that people combine the two. Also, he has regularly expressed his certainty that we are all going to die to Skynet in terms of ridiculously overconfident probabilities, he shouldn't be surprised that other people followed suit.

  • AI winter is in the air! But we think the AI bubble keeps going until 2027

    Jump

  • Guns don't kill people, people kill people.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • I missed that it’s also explicitly meant as rationalist esoterica.

    It turns in that direction about 20ish pages in... and spends hundreds of pages on it, greatly inflating the length from what could be a much more readable length. It then gets back to actual plot events after that.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • I hadn't heard of MAPLE before, is it tied to lesswrong? From the focus on AI it's at least adjacent to it... so I'll add that to the list of cults lesswrong is responsible for. So all in all, we've got the Zizians, Leverage Research, and now Maple for proper cults, and stuff like Dragon Army and Michael Vassar's groupies for "high demand" groups. It really is a cult incubator.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • I actually think "Project Lawful" started as Eliezer having fun with glowfic (he has a few other attempts at glowfics that aren't nearly as wordy... one of them actually almost kind of pokes fun at himself and lesswrong), and then as it took off and the plot took the direction of "his author insert gives lectures to an audience of adoring slaves" he realized he could use it as an opportunity to squeeze out all the Sequence content he hadn't bothered writing up in the past decade^ . And that's why his next attempt at a HPMOR-level masterpiece is an awkward to read rp featuring tons of adult content in a DnD spinoff, and not more fanfiction suitable for optimal reception to the masses.

    ^(I think Eliezer's writing output dropped a lot in the 2010s compared to when he was writing the sequences and the stuff he has written over the past decade is a lot worse. Like the sequences are all in bite-size chunks, and readable in chunks in sequence, and often rephrase legitimate science in a popular way, and have a transhumanist optimism to them. Whereas his recent writings are tiny little hot takes on twitter and long, winding, rants about why we are all doomed on lesswrong.)

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • Weird rp wouldn't be sneer worthy on it's own (although it would still be at least a little cringe), it's contributing factors like...

    • the constant IQ fetishism (Int is superior to Charisma but tied with Wis and obviously a true IQ score would be both Int and Wis)
    • the fact that Eliezer cites it like serious academic writing (he's literally mentioned it to Yann LeCunn in twitter arguments)
    • the fact that in-character lectures are the only place Eliezer has written up many of his decision theory takes he developed after the sequences (afaik, maybe he has some obscure content that never made it to lesswrong)
    • the fact that Eliezer think it's another HPMOR-level masterpiece (despite how wordy it is, HPMOR is much more readable, even authors and fans of glowfic usually acknowledge the format can be awkward to read and most glowfics require huge amounts of context to follow)
    • the fact that the story doubles down on the HPMOR flaw of confusion of which characters are supposed to be author mouthpieces (putting your polemics into the mouths of character's working for literal Hell... is certainly an authorial choice)
    • and the continued worldbuilding development of dath ilan, the rationalist utopia built on eugenics and censorship of all history (even the Hell state was impressed!)

    ...At least lintamande has the commonsense understanding of why you avoid actively linking your bdsm dnd roleplay to your irl name and work.

    And it shouldn't be news to people that KP supports eugenics given her defense of Scott Alexander or comments about super babies, but possibly it is and headliner of weird roleplay will draw attention to it.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • To be fair to DnD, it is actually more sophisticated than the IQ fetishists, it has 3 stats for mental traits instead of 1!

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • It's always the people you most expect.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 24th August 2025

    Jump

  • It's pretty screwed up that humble bragging about putting their own mother out of a job is a useful opening to selling a scam-service. At least the people that buy into it will get what they have coming?

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • I’d probably be exaggerating if I said that every time I looked under the hood of Wikipedia, it reaffirmed how I don’t have the temperament to edit there.

    The lesswrongers hate dgerad's Wikipedia work because they perceive it as calling them out, but if anything Wikipedia's norms makes his "call outs" downright gentle and routine.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • Keep in mind I was wildly guessing with a lot of numbers... like I'm sure 90 GB vRAM is enough for decent quality pictures generated in minutes, but I think you need a lot more compute to generate video at a reasonable speed? I wouldn't be surprised if my estimate is off by a few orders of magnitude. $.30 is probably enough that people can't spam lazily generated images, and a true cost of $3.00 would keep it in the range of people that genuinely want/need the slop... but yeah I don't think it is all going cleanly away once the bubble pops or fizzles.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • After GPT-3 failed to be it, they aimed at five iterations instead because that sounded like a nice number to give to investors, and GPT-3.5 and GPT-4o are very much responses to an inability to actually manifest that AGI on a VC-friendly timetable.

    That's actually more batshit than I thought! Like I thought Sam Altman knew the AGI thing was kind of bullshit and the hesitancy to stick a GPT-5 label on anything was because he was saving it for the next 10x scaling step up (obviously he didn't even get that far because GPT-5 is just a bunch of models shoved together with a router).

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump
    1. Even if was noticeably better, Scam Altman hyped up GPT-5 endlessly, promising a PhD in your pocket, and an AGI and warning that he was scared of what he created. Progress has kind of plateaued, so it isn't even really noticeably better, it scores a bit higher on some benchmarks, and they've patched some of the more meme'd tests (like counting rs in strawberry... except it still can't count the r's in blueberry, so they've probably patched the more obvious flubs with loads of synthetic training data as opposed to inventing some novel technique that actually improves it all around). The other reason the promptfondlers hate it is because, for the addicts using it as a friend/therapist, it got a much drier more professional tone, and for the people trying to use it in actual serious uses, losing all the old models overnight was really disruptive.
    2. There are a couple of speculations as to why... one is that GPT-5 variants are actually smaller than the previous generation variants and they are really desperate to cut costs so they can start making a profit. Another is that they noticed that there naming scheme was horrible (4o vs o4) and confusing and have overcompensated by trying to cut things down to as few models as possible.
    3. They've tried to simplify things by using a routing model that makes the decision for the user as to what model actually handles each user interaction... except they've screwed that up apparently (Ed Zitron thinks they've screwed it up badly enough that GPT-5 is actually less efficient despite their goal of cost saving). Also, even if this technique worked, it would make ChatGPT even more inconsistent, where some minor word choice could make the difference between getting the thinking model or not and that in turn would drastically change the response.
    4. I've got no rational explanation lol. And now they overcompensated by shoving a bunch of different models under the label GPT-5.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • There are techniques for caching some of the steps involved with LLMs. Like I think you can cache the tokenization and maybe some of the work of the attention head is doing if you have a static, known, prompt? But I don't see why you couldn't just do that caching separately for each model your model router might direct things to? And if you have multiple prompts you just do a separate caching for each one? This creates a lot of memory usage overhead, but not more excessively more computation... well you do need to do the computation to generate each cache. I don't find it that implausible that OpenAI couldn't manage to screw all this up somehow, but I'm not quite sure the exact explanation of the problem Zitron has given fits together.

    (The order of the prompts vs. user interactions does matter, especially for caching... but I think you could just cut and paste the user interactions to separate it from the old prompt and stick a new prompt on it in whatever order works best? You would get wildly varying quality in output generated as it switches between models and prompts, but this wouldn't add in more computation...)

    Zitron mentioned a scoop, so I hope/assume someone did some prompt hacking to get GPT-5 to spit out some of it's behind the scenes prompts and he has solid proof about what he is saying. I wouldn't put anything past OpenAI for certain.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • If they got a lot of usage out of a model this constant cost would contribute little to the cost of each model in the long run... but considering they currently replace/retrain models every 6 months to 1 year, yeah this cost should be factored in as well.

    Also, training compute grows quadratically with model size, because its is a multiple of training data (which grows linearly with model size) and the model size.