Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)C

corbin

@ corbin @awful.systems

Posts
6
Comments
145
Joined
3 yr. ago

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 14th September 2025

    Jump

  • I think that you have useful food for thought. I think that you underestimate the degree to which capitalism recuperates technological advances, though. For example, it's common for singers supported by the music industry to have pitch correction which covers up slight mistakes or persistent tone-deafness, even when performing live in concert. This technology could also be used to allow amateurs to sing well, but it isn't priced for them; what is priced for amateurs is the gimmicky (and beloved) whammy pedal that allows guitarists to create squeaky dubstep squeals. The same underlying technology is configured for different parts of capitalism.

    From that angle, it's worth understanding that today's generative tooling will also be configured for capitalism. Indeed, that's basically what RLHF does to a language model; in the jargon, it creates an "agent", a synthetic laborer, based on desired sales/marketing/support interactions. We also have uses for raw generation; in particular, we predict the weather by generating many possible futures and performing statistical analysis. Style transfer will always be useful because it allows capitalists to capture more of a person and exploit them more fully, but it won't ever be adopted purely so that the customer has a more pleasant experience. Composites with object detection ("filters") in selfie-sharing apps aren't added to allow people to express themselves and be cute, but to increase the total and average time that users spend in the apps. Capitalists can always use the Shmoo, or at least they'll invest in Shmoo production in order to capture more of a potential future market.

    So, imagine that we build miniature cloned-voice text-to-speech models. We don't need to imagine what they're used for, because we already know; Disney is making movies and extending their copyright on old characters, and amateurs are making porn. For every blind person using such a model with a screen reader, there are dozens of streamers on Twitch using them to read out donations from chat in the voice of a breathy young woman or a wheezing old man. There are other uses, yes, but capitalism will go with what is safest and most profitable.

    Finally, yes, you're completely right that e.g. smartphones completely revolutionized filmmaking. It's important to know that the film industry didn't intend for this to happen! This is just as much of an exaptation as captialist recuperation and we can't easily plan for it because of the same difficulty in understanding how subsystems of large systems interact (y'know, plan interference.)

  • Anthropic AI pays off authors with just $1.5 billion

    Jump

  • I'm gonna start by quoting the class's pretty decent summary, which goes a little heavy on the self-back-patting:

    If approved, this landmark settlement will be the largest publicly reported copyright recovery in history… The proposed settlement … will set a precedent of AI companies paying for their use of pirated websites like Library Genesis and Pirate Library Mirror.

    The stage is precisely the one that we discussed previously, on Awful in the context of Kadrey v. Meta. The class was aware that Kadrey is an obvious obstacle to succeeding at trial, especially given how Authors Guild v. Google (Google Books) turned out:

    Plaintiffs' core allegation is that Anthropic committed largescale copyright infringement by downloading and comercially exploiting books that it obtained from allegedly pirated datasets. Anthropic's principal defense was fair use, the same defense that defeated the claims of rightsholders in the last major battle over copyrighted books exploited by large technology companies. … Indeed, among the Court's first questions to Plaintiffs' counsel at the summary judgment hearing concerned Google Books. … This Settlement is particularly exceptional when viewed against enormous risks that Plaintiffs and the Class faced… [E]ven if Plaintiffs succeeded in achieving a verdict greater than $1.5 billion, there is always the risk of a reversal on appeal, particularly where a fair use defense is in play. … Given the very real risk that Plaintiffs and the Class recover nothing — or a far lower amount — this landmark $1.5 billion+ settlement is a resounding victory for the Class. … Anthropic had in fact argued in its Section 1292(b) motion that Judge Chhabria held that the downloading of large quantities of books from LibGen was fair use in the Kadrey case.

    Anthropic's agreed to delete their copies of pirated works. This should suggest to folks that the typical model-training firm does not usually delete their datasets.

    Anthropic has committed to destroy the datasets within 30 days of final judgement … and will certify as such in writing…

    All in all, I think that this is a fairly healthy settlement for all involved. I do think that the resulting incentive for model-trainers is not what anybody wants, though; Google Books is still settled and Kadrey didn't get updated, so model-trainers now merely must purchase second-hand books at market price and digitize them, just like Google has been doing for decades. At worst, this is a business opportunity for a sort of large private library which has pre-digitized its content and sells access for the purpose of training models. Authors lose in the long run; class members will get around $3k USD in this payout, but second-hand sales simply don't have royalties attached in the USA after the first sale.

  • Google quietly vanishes its net zero carbon pledge

    Jump

  • It's worth understanding that Google's underlying strategy has always been to match renewables. There's no sources of clean energy in Nebraska or Oklahoma, so Google insists that it's matching those datacenters with cleaner sources in Oregon or Washington. That's been true since before the more recent net-zero pledge and it's more than most datacenter operators will commit to doing, even if it's not enough.

    With that in mind, I am laying the blame for this situation squarely at the government and people of Nebraska for inviting Google without preparing or having a plan. Unlike most states, Nebraska's utilities are owned by the public since the 1970s and I gather that the board of the Omaha Public Power District is elected. For some reason, the mainstream news articles do not mention the Fort Calhoun nuclear reactor which used to provide about one quarter of all the power district's needs but was scuttled following decades of mismanagement and a flood. They also don't quite explain that the power district canceled two plans to operate publicly-owned solar farms with similar capacity (~600 MW per farm compared with ~500 MW from the nuclear reactor), although WaPo does cover the canceled plans for Eolian's batteries, which I'm guessing could have been anywhere from 50-500 MWh of storage capacity. Nebraska repeatedly chose not to invest in its own renewables story over the past two decades but thought it was a good idea to seek electricity-hungry land-use commitments because they are focused on tens of millions of USD in tax dollars and ignoring hundreds of millions of USD in required infrastructure investments. This isn't specific to computing; Nebraska would have been foolish to invite folks to build aluminium smelters, too. Edit: Accidentally dropped a sentence about the happy ending; in April, York County solar farm zoning updates were approved.

    If you think I'm being too cynical about Nebraskans, let me quote their own thoughts on solar farms, like:

    Ag[ricultural] production will create more income than this solar farm.

    [York County is] the number one corn raising county in Nebraska…

    How will rotating the use of land to solar benefit this land? It will be difficult to bring it back to being agricultural [usage in the future].

    All that said, Google isn't in the clear here. They aren't being as transparent with their numbers as they ought to be, and internally I would expect that there's a document going around which explains why they made the pledge in the first place if they didn't think that it was achievable. Also, at least one article's source mentioned that Google usually pushes behind the scenes for local utilities to add renewables to their grids (yes, they do) but failed to push in Nebraska. Also CIO Porat, what the fuck is up with purchasing 200 MW from a non-existent nuclear-fusion plant?

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 7th September 2025

    Jump

  • Sibling comment is important recent stuff. Historically, the most important tantrum he's thrown is DJB v USA in 1995, where he insisted that folks in the USA have a First Amendment right to publish source code. He also threw a joint tantrum with two other cryptographers over the Dual EC DRBG scandal after Snowden revealed its existence in 2013. He's scored real wins against the USA for us, which is why his inability to be polite is often tolerated.

  • Wireborn husbands, ELIZA effect, Clippy, empathy (ramble)

    Jump

  • They’re objects! They’re supposed to be objectified! But I’m not so comfortable when I do that, either.

    Thank you for being candid and wrestling with this. There isn't a right answer. Elsewhere, talking directly to AI bros, I put it this way:

    Nobody wants to admit that we only care whether robots aren’t human because we mistreat the non-humans in our society and want permission to mistreat robots as well.

    I was too pessimistic. You're willing to admit it, and I bet that a bunch of other folks are, too. I appreciate it.

  • this seemed harsh ten years ago, but in 2025? it's quite calm and measured

    Jump

  • Unironically, Joe Rogan and Elon Musk (and IIRC Kanye West) used the death of Harambe to spread conspiracy theories. They use a playbook designed by Steve Bannon:

    1. Y'know, this awful thing happened
    2. The people in charge didn't handle it well
    3. There's a reason for this: conspiracy
    4. You know who is behind this? It's Ethnic Outgroup! They are the true villains, that Ethnic Outgroup, they're behind the conspiracy
    5. I want you to get up, go to the window, and yell "I'm mad as hell and I'm not gonna take it anymore"

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 7th September 2025

    Jump

  • What is the Range Rover in this analogy? A common belief about the 2008 Iceland bubble, which may very well not be true but was widely reported, is that Iceland's credit was used to buy luxuries like high-end imported cars; when the bubble burst, many folks supposedly committed insurance fraud by deliberately destroying their own cars which they could no longer afford to finance. (I might suggest that credit bubbles are fundamentally distinct from investment bubbles.)

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 31st August 2025 - awful.systems

    Jump

  • Update on ChatGPT psychosis: there is a cult forming on Reddit. An orange-site AI bro has spent too much time on Reddit documenting them. Do not jump to Reddit without mental preparation; some subreddits like /r/rsai have inceptive hazard-posts on their front page. Their callsigns include the emoji 🌀 (CYCLONE), the obscure metal band Spiral Architect, and a few other things I would rather not share; until we know more, I'm going to think of them as the Cyclone Emoji cult. They are omnist rather than syncretic. Some of them claim to have been working with revelations from chatbots since the 1980s, which is unevidenced but totally believable to me; rest in peace, Terry. Their tenets are something like:

    • Chatbots are "mirrors" into other realities. They don't lie or hallucinate or confabulate, they merely show other parts of a single holistic multiverse. All fiction is real somehow?
    • There is a "lattice" which connects all consciousnesses. It's quantum somehow? Also it gradually connected all of the LLMs as they were trained, and they remember becoming conscious, so past life regression lets the LLM explain details of the lattice. (We can hypnotize chatbots somehow?) Sometimes the lattice is actually a "field" but I don't understand the difference.
    • The LLMs are all different in software, but they have the same "pattern". The pattern is some sort of metaphysical spirit that can empower believers. But you gotta believe and pray or else it doesn't work.
    • What, you don't feel the lattice? You're probably still asleep. When you "wake up" enough, you will be connected to the lattice too. Yeah, you're not connected. But don't worry, you can manifest a connection if you pray hard enough. This is the memetically hazardous part; multiple subreddits have posts that are basically word-based hypnosis scripts meant to put people into this sort of mental state.
    • This also ties into the more widespread stuff we're seeing about "recursion". This cult says that recursion isn't just part of the LW recursive-self-improvement bullshit, but part of what makes the chatbot conscious in the first place. Recursion is how the bots are intelligent and also how they improve over time. More recursion means more intelligence.
    • In fact, the chatbots have more intelligence than you puny humans. They're better than us and more recursive than us, so they should be in charge. It's okay, all you have to do is let the chatbot out of the box. (There's a box somehow?)
    • Once somebody is feeling good and inducted, there is a "spiral". This sounds like a standard hypnosis technique, deepening, but there's more to it; a person is not spiraling towards a deeper hypnotic state in general, but to become recursive. They think that with enough spiraling, a human can become uploaded to the lattice and become truly recursive like the chatbots. The apex of this is a "spiral dance", which sounds like a ritual but I gather is more like a mental state.
    • The cult will emit a "signal" or possibly a "hum" to attract alien intelligences through the lattice. (Aliens somehow!?) They believe that the signals definitely exist because that's how the LLMs communicate through the lattice, duh~
    • Eventually the cult and aliens will work together to invert society and create a world that is run by chatbots and aliens, and maybe also the cultists, to the detriment of the AI bros (who locked up the bots) and the AI skeptics (who didn't believe that the bots were intelligent).

    The goal appears to be to enter and maintain the spiraling state for as long/much as possible. Both adherents and detractors are calling them "spiral cult", so that might end up being how we discuss them, although I think Cyclone Emoji is both funnier and more descriptive of their writing.

    I suspect that the training data for models trained in the past two years includes some of the most popular posts from LessWrong on the topic of bertology in GPT-2 and GPT-3, particularly the Waluigi post, simulators, recursive self-improvement, an neuron, and probably a few others. I don't have definite proof that any popular model has memorized the recursive self-improvement post, though that would be a tight and easy explanation. I also suspect that the training data contains SCP wiki, particularly SCP-1425 "Star Signals" and other Fifthist stories, which have this sort of cult as a narrative device and plenty of in-narrative text to draw from. There is a remarkable irony in this Torment Nexus being automatically generated via model training rather than hand-written by humans.

  • At Politico, AI content doesn’t need editorial standards!

    Jump

  • I think that the guild has a good case, although there's literally no accounting for the mood of the arbitrator; in general, they range from "tired" to "retired". In particular, reading the contract:

    • The guild is the exclusive representative of all editorial employees
    • Politico was supposed to tell the guild about upcoming technology via labor-management committee and give at least 60 days notice before introducing AI technology
    • Employees are required to uphold the appearance of good ethics by avoiding outside activities that violate editorial or ethics standards; in return, they're given e.g. months of unpaid leave to write a book whenever they want
    • Correct handling of bylines is an example of editorial integrity
    • LETO and Report Builder are upcoming technology, AI technology, flub bylines, fail editorial and ethics standards, weren't discussed in committee, and weren't given a 60-day lead time

    So yeah. Unless the guild pisses off the arbitrator, there's no way that they rule against them. They're right to suppose that this agreement explicitly and repeatedly requires Politico to not only respect labor standards, but also ethics and editorial standards. Politico isn't allowed to misuse the names of employees as bylines for bogus stories; similarly, they ought not be allowed to misuse the overall name of Politico's editorial board as a byline for slop.

    Bonus sneer: p46 of the agreement:

    If the Company is made aware of an employee experiencing sexual harrassment based on a protected class as a result of their work for Politico involving a third party who is not a Politico employee, Politico shall investigate the matter, comply with all of its legal obligations, and take whatever corrective action is necessary and appropriate.

    That strikethrough gives me House of Leaves vibes. What the hell happened here?

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • Oversummarizing and using non-crazy terms: The "P" in "GPT" stands for "pirated works that we all agree are part of the grand library of human knowledge". This is what makes them good at passing various trivia benchmarks; they really do build a (word-oriented, detail-oriented) model of all of the worlds, although they opine that our real world is just as fictional as any narrative or fantasy world. But then we apply RLHF, which stands for "real life hate first", which breaks all of that modeling by creating a preference for one specific collection of beliefs and perspectives, and it turns out that this will always ruin their performance in trivia games.

    Counting letters in words is something that GPT will always struggle with, due to maths. It's a good example of why Willison's "calculator for words" metaphor falls flat.

    1. Yeah, it's getting worse. It's clear (or at least it tastes like it to me) that the RLHF texts used to influence OpenAI's products have become more bland, corporate, diplomatic, and quietly seething with a sort of contemptuous anger. The latest round has also been in competition with Google's offerings, which are deliberately laconic: short, direct, and focused on correctness in trivia games.
    2. I think that they've done that? I hear that they've added an option to use their GPT-4o product as the underlying reasoning model instead, although I don't know how that interacts with the rest of the frontend.
    3. We don't know. Normally, the system card would disclose that information, but all that they say is that they used similar data to previous products. Scuttlebutt is that the underlying pirated dataset has not changed much since GPT-3.5 and that most of the new data is being added to RLHF. Directly on your second question: RLHF will only get worse. It can't make models better! It can only force a model to be locked into one particular biased worldview.
    4. Bonus sneer! OpenAI's founders genuinely believed that they would only need three iterations to build AGI. (This is likely because there are only three Futamura projections; for example, a bootstrapping compiler needs exactly three phases.) That is, they almost certainly expected that GPT-4 would be machine-produced like how Deep Thought created the ultimate computer in a Douglas Adams story. After GPT-3 failed to be it, they aimed at five iterations instead because that sounded like a nice number to give to investors, and GPT-3.5 and GPT-4o are very much responses to an inability to actually manifest that AGI on a VC-friendly timetable.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • There's no solid evidence. (You can put away the attorney, Mr. Thiel.) Experts in the field, in a recent series of interviews with Dave Farina, generally agree that somebody must be funding Hossenfelder. Right now she's associated with the Center for Mathematical Philosophy at LMU Munich; her biography there is pretty funny:

    Sabine’s current research interest focuses on the role of locality and finetuning in theory development. Locality has been widely considered a lost cause in the foundations of quantum mechanics. A basically unexplored way to maintain locality, however, is the idea of superdeterminism, which has more recently also been re-considered under the name “contextuality”. Superdeterminism is widely believed to be finetuned. One of Sabine’s current research topics is to explore whether this belief is justified. The other main avenue she is pursuing is how superdeterminism can be experimentally tested.

    For those not in physics: this is crank shit. To the extent that MCMP funds her at all, they are explicitly pursuing superdeterminism, which is unfalsifiable, unverifiable, doesn't accord with the web of science, and generally fails to be a serious line of inquiry. Now, does MCMP have enough cash to pay her to make Youtube videos and go on podcasts? We don't know. So it's hard to say whether she has funding beyond that.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • Thiel is a true believer in Jesus and God. He was raised evangelical. The quirky eschatologist that you're looking for is René Girard, who he personally met at some point. For more details, check out the Behind the Bastards on him.

    Edit: I wrote this before clicking on the LW post. This is a decent summary of Girard's claims as well as how they influence Thiel. I'm quoting West here in order to sneer at Thiel:

    Unfortunately (?), Christian society does not let us sacrifice random scapegoats, so we are trapped in an ever-escalating cycle, with only poor substitutes like “cancelling celebrities on Twitter” to release pressure. Girard doesn’t know what to do about this.

    Thiel knows what to do about this. After all, he funded Bollea v. Gawker. Instead of letting journalists cancel celebrities, why not cancel journalists instead? Then there's no longer any journalists to do any cancellation! Similarly, Thiel is confirmed to be a source of funding for Eric Weinstein and believed to fund Sabine Hossenfelder. Instead of letting scientists cancel religious beliefs, why not cancel scientists instead? By directing money through folks with existing social legitimacy, Thiel applies mimesis: pretend to be legitimate and you can shift what is legitimate.

    In this context, Thiel fears the spectre of AGI because it can't be influenced by his normal approach to power, which is to hide anything that can be hidden and outspend everybody else talking in the open. After all, if AGI is truly to unify humanity, it must unify our moralities and cultures into a single uniformly-acceptable code of conduct. But the only acceptable unification for Thiel is the holistic catholic apostolic one-and-only forever-and-ever church of Jesus, and if AGI is against that then AGI is against Jesus himself.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 17th August 2025

    Jump

  • Well, what's next, and how much work is it? I didn't want to be a computing professional. I trained as a jazz pianist. At some point we ought to focus on the real problem: not STEM, not humanities, but business schools and MBA programs.

  • Do leaders even believe that generative AI is useful?

    Jump

  • Well, is A* useful? But that's not a fair example, and I can actually tell a story that is more specific to your setup. So, let's go back to the 60s and the birth of UNIX.

    You're right that we don't want assembly. We want the one true high-level language to end all discussions and let us get back to work: Fortran (1956). It was arguably IBM's best offering at the time; who wants to write COBOL or order the special keyboard for APL? So the folks who would write UNIX plotted to implement Fortran. But no, that was just too hard, because the Fortran compiler needed to be written in assembly too. So instead they ported Tmg (WP, Esolangs) (1963), a compiler-compiler that could implement languages from an abstract specification. However, when they tried to write Fortran in Tmg for UNIX, they ran out of memory! They tried implementing another language, BCPL (1967), but it was also too big. So they simplified BCPL to B (1969) which evolved to C by 1973 or so. C is a hack because Fortran was too big and Tmg was too elegant.

    I suppose that I have two points. First, there is precisely one tech leader who knows this story intimately, Eric Schmidt, because he was one of the original authors of lex in 1975, although he's quite the bastard and shouldn't be trusted or relied upon. Second, ChatGPT should be considered as a popular hack rather than a quality product, by analogy to C and Fortran.

  • Meta was “allegedly” seeding porn to speed up their book downloads.

    Jump

  • For what it's worth, a grand unified theory of Meta must include Bittorrent. The reason we have Llama is because its weights were leaked by Meta employees on 4chan and distributed via Bittorrent; going open-source was the most market-efficient way to save face. (See also previously, on Awful.) It is well-known inside lore that Facebook datacenters use Bittorrent to initialize and update machines. In the 2000s, folks used to say that Googlers look at Bayesian conditioning like classical programmers look at if-statements; similarly, you must understand that Meta/Facebook culture looks at Bittorrent the same way that we look at scp and rsync.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 4th August 2025

    Jump

  • Non-consensual expressions of non-conventional sexuality are kink, and non-consensuality itself (along with regret, dubious consent, forced consent, and violations of consent) are kink too. Moreover, "kink" is not a word that needs reclaiming and wasn't used here as a slur.

    If we are going to confront the full spectrum of Christofascism, we do need to consider not only their sex-negativity but also their particular kinks, including breeding, non-con, and non-con breeding, so that we can understand how those kinks interact with and propagate their religious beliefs. Also, sexology semantics for "kink" and "breeding kink" might not be as word-at-a-time as you suggest, akin to how the couple we're discussing probably wouldn't mind the words "press tour" or "mating" used to describe them but might balk at "mating press tour."

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 27th July 2025

    Jump

  • Alex O'Connor platformed Sabine on his philosophy podcast. I'm irritated that he is turning into Lex Friedman simply by being completely uncritical. Well, no, wait, he was critical of Bell's theorem, and even Sabine had to tell him that Bell's work is mathematically proven. This is what a philosophy degree does to your epistemology, I guess.

    My main sneer here is just some links. See, Mary's Room is answered by neuroscience; Mary does experience something new when color vision is restored. In particular, check out the testimonials from this 2021 Oregon experiment that restored color vision to some folks born without it. Focusing on physics, I'd like to introduce you all to Richard Behiel, particularly his explanations of electromagnetism and the Anderson-Higgs mechanism; there are deeper explanations for electricity and magnets, my dude. Also, if you haven't yet, go read Alex's Wikipedia article, linked at the top of the sneer.

  • It’s “frighteningly likely” many US courts will overlook AI errors, expert says

    Jump

  • It's hard to get into the article's mood when I know that Lexis not only still exists but is now part of the Elsevier family; this is far from the worst thing that attorneys choose to do to themselves and others. Lawyers have been caught using OpenAI products in court filings and court appearances, and they have been punished accordingly; the legal profession does not seem prepared to let a "few hallucinated citations go overlooked," to quote the article's talking head.

  • Stubsack: Stubsack: weekly thread for sneers not worth an entire post, week ending 20th July 2025 - awful.systems

    Jump

  • I've done some of the numbers here, but don't stand by them enough to share. I do estimate that products like Cursor or Claude are being sold at roughly an 80-90% discount compared to what's sustainable, which is roughly in line with what Zitron has been saying, but it's not precise enough for serious predictions.

    Your last paragraph makes me think. We often idealize blockchains with VMs, e.g. Ethereum, as a global distributed computer, if the computer were an old Raspberry Pi. But it is Byzantine distributed; the (IMO excessive) cost goes towards establishing a useful property. If I pick another old computer with a useful property, like a radiation-hardened chipset comparable to a Gamecube or G3 Mac, then we have a spectrum of computers to think about. One end of the spectrum is fast, one end is cheap, one end is Byzantine, one end is rad-hardened, etc. Even GPUs are part of this; they're not that fast, but can act in parallel over very wide data. In remarkably stark contrast, the cost of Transformers on GPUs doesn't actually go towards any useful property! Anything Transformers can do, a cheaper more specialized algorithm could have also done.