Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)C

corbin

@ corbin @awful.systems

Posts
13
Comments
226
Joined
3 yr. ago

  • People Are Being Involuntarily Committed, Jailed After Spiraling Into "ChatGPT Psychosis"

    Jump

  • Well, yes. It's not a new concept; it was a staple of Cold War sci-fi like The Three Stigmata, and we know from studies of e.g. Pentacostal worship that it is pretty easy to broadcast a suggestion to a large group of vulnerable people and get at least some of them to radically alter their worldview. We also know a reliable formula for changing people's beliefs; we use the same formula in sensitivity training as we did in MKUltra, including belief challenges, suspension of disbelief, induction/inception, lovebombing, and depersonalization. We also have a constant train of psychologists attempting to nudgelord society, gently pushing mass suggestions and trying to slowly change opinions at scale.

    Fundamentally your sneer is a little incomplete. MKUltra wasn't just about forcing people to challenge their beliefs via argumentation and occult indoctrination, but also psychoactive inhibition-lowering drugs. In this setting, the drugs are administered after institutionalization.

  • Meta beats Kadrey, AI training was fair use — what this means

    Jump

  • Read carefully. On p1-2, the judge makes it clear that "the incentive for human beings to create artistic and scientific works" is "the ability of copyright holders to make money from their works," to the law, there isn't any other reason to publish art. This is why I'm so dour on copyright, folks; it's not for you who love to make art and prize it for its cultural impact and expressive power, but for folks who want to trade art for money.

    On p3, a contrast appears between Chhabria and Alsup (yes, that Alsup); the latter knows what a computer is and how to program it, and this makes him less respectful of copyright overall. Chhabria doesn't really hide that they think Meta didn't earn their summary judgement, presumably because they disagree with Alsup about whether this is a "competitive or creative displacement." That's fair given the central pillar of the decision on p4:

    Llama is not capable of generating enough text from the plantiffs' books to matter, and the plaintiffs are not entitled to the market for licensing their works as AI training data.

    An analogy might make this clearer. Suppose a transient person on a street corner is babbling. Occasionally they spout what sounds like a quote from a Star Wars film. Intrigued, we prompt the transient to recite the entirety of Star Wars, and they proceed to mostly recreate the original film, complete with sound effects and voice acting, only getting a few details wrong. Does it matter whether the transient paid to watch the original film (as opposed to somebody else paying the fee)? No, their recreation might be candid and yet not faithful enough to infringe. Is Lucas entitled to a licensing fee for every time the transient happens to learn something about Star Wars? Eh, not yet, but Disney's working on it. This is why everybody is so concerned about whether the material was pirated, regardless of how it was paid for; they want to say that what's disallowed is not the babbling on the street but the access to the copyrighted material itself.

    Almost every technical claim on p8-9 is simplified to the point of incorrectness. They are talking points about Transformers turned into aphorisms and then axioms. The wrongest claim is on p9, that "to be able to generate a wide range of text … an LLM's training data set must be large and diverse" (it need only be diverse, not large) followed by the claim that an LLM's "memory" must be trained on books or equivalent "especially valuable training data" in order to "work with larger amounts of text at once" (conflating hyperparameters with learned parameters.) These claims show how the judge fails to actually engage with the technical details and thus paints with a broad brush dipped in the wrong color.

    On p12, the technical wrongness overflows. Any language model can be forced to replicate a copyrighted work, or to avoid replication, by sampling techniques; this is why perplexity is so important as a metric. What would have genuinely been interesting is whether Llama is low-perplexity on the copyrighted works, not the rate of exact replications, since that's the key to getting Llama to produce unlimited Harry Potter slash or whatever.

    On p17 the judge ought to read up on how Shannon and Markov initially figured out information theory. LLMs read like Shannon's model, and in that sense they're just like humans: left to right, top to bottom, chunking characters into words, predicting shapes and punctuation. Pretending otherwise is powdered-wig sophistry or perhaps robophobia.

    On p23 Meta cites fuckin' Sega v. Accolade! This is how I know y'all don't read the opinions; you'd be hyped too. I want to see them cite Galoob next. For those of you who don't remember the 90s, the NES and Genesis were video game consoles, and these cases established our right to emulate them and write our own games for them.

    p28-36 is the judge giving free legal advice. I find their line of argumentation tenuous. Consider Minions; Minions are bad, Minions are generic, and Minions can be used to crank out infinite amounts of slop. But, as established at the top, whoever owns Minions has the right to profit from Minions, and that is the lone incentive by which they go to market. However, Minions are arbitrary; there's no reason why they should do well in the market, given how generic and bad they are. So if we accept their argument then copyright becomes an excuse for arbitrary winners to extract rent from cultural artifacts. For a serious example, look up the ironic commercialization of the Monopoly brand.

  • Anthropic AI wins broad fair use for training! — but not on pirated books

    Jump

  • Top-level commenters would do well to read Authors Guild v Google, two decades ago. They're also invited to rend their garments and gnash their teeth at Google, if they like.

  • Stubsack: Stubsack: weekly thread for sneers not worth an entire post, week ending 29th June 2025

    Jump

  • Last Week Tonight's rant of the week is about AI slop. A Youtube video is available here. Their presentation is sufficiently down-to-earth to be sharable with parents and extended family, focusing on fake viral videos spreading via Facebook, Instagram, and Pinterest; and dissecting several examples of slop in order to help inoculate the audience.

  • Disney sues AI image generator Midjourney

    Jump

  • What a deeply dishonorable lawsuit. The complaint is essentially that Disney and Universal deserve to be big powerful movie studios that employ and systematically disenfranchise "millions of" artists (p8).

    Disney claims authorship over Darth Vader (Lucas) and Yoda (Oz), Elsa and Ariel (Andersen), folk characters Aladdin, Mulan, and Snow White; Lightning McQueen & Buzz Lightyear (Lasseter et al), Sully (Gerson & Stanton), Iron Man (Lee, Kirby, et al), and Homer Simpson (Groening). Disney not only did not design or produce any of these characters, but Disney purchased those rights. I will give Universal partial credit for not claiming to invent any of their infamous movie monsters, but they do claim to have created Shrek (Stieg). Still, this is some original-character-do-not-steal snottiness; these avaricious executives and attorneys appropriated art from artists and are claiming it as their own so that they can sue another appropriator.

    Here is a sample of their attitude, p16 of the original complaint:

    Disney's copyright registrations for the entertainment properties in The Simpsons franchise encompass the central characters within.

    See, they're the original creator and designated benefactor, because they have Piece of Paper, signed by Government Authority, and therefore they are Owner. Who the fuck are Matt Groening or Tracey Ullman?

    I will not contest Universal's claim to Minions.

    One weakness of the claim is that it's not clear whether Midjourney infringes, Midjourney's subscribers infringe, or Midjourney infringes when collaborating with its subscribers. It seems like they're going to argue that Midjourney commits the infringing act, although p104 contains hedges that will allow Disney to argue either way. Another weakness is the insistence that Midjourney could filter infringing queries, but chooses not to; this is a standard part of amplifying damages in copyright claims but might not stand up under scrutiny since Midjourney can argue that it's hard to e.g. tell the difference between infringing queries and parodic or satirical queries which infringe but are permitted by fair use. On the other hand, this lawsuit could be an attempt to open a new front in Disney's long-standing attempt to eradicate fair use.

    As usual, I'm not defending Midjourney, who I think stand on their own demerits. But I'm not ever going to suck Disney dick given what they've done to the animation community. I wish y'all would realize the folly of copyright already.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 25th May 2025

    Jump

  • Oh, sorry. We're in agreement and my sentence was poorly constructed. The computation of a matrix multiplication usually requires at least pencil and paper, if not a computer. I can't compute anything larger than a 2 × 2. But I'll readily concede that Strassen's specific trick is simple enough that a mentalist could use it.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 25th May 2025

    Jump

  • Your understanding is correct. It's worth knowing that the matrix-multiplication exponent actually controls multiple different algorithms. I stubbed a little list a while ago; important examples include several graph-theory algorithms as well as parsing for context-free languages. There's also a variant of P vs NP for this specific problem, because we can verify that a matrix is a product in quadratic time.

    That Reddit discussion contains mostly idiots, though. We expect an iterative sequence of ever-more-complicated algorithms with ever-slightly-better exponents, approaching quadratic time in the infinite limit. We also expected a computer to be required to compute those iterates at some point; personally I think Strassen's approach only barely fits inside a brain and the larger approaches can't be managed by humans alone.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 25th May 2025

    Jump

  • Read it to the end and then re-read 2009's The Gervais Principle. I hope Ed eventually comes back to Rao's rant because they complement each other perfectly; Zitron's Business Idiot is Rao's Clueless! What Rao brings to the table is an understanding that Sociopaths exist and steer the Clueless, and also that the ratio of (visible) Clueless to Sociopaths is an indication of the overall health of an (individual) business; Zitron's argument is then that we are currently in an environment (the "Rot Economy" in his writing) which is characterized by mostly Clueless business leaders.

    Then re-read Doctorow's 2022 rant Social Quitting, which introduced "enshittification", an alternate understanding of Rao's process. To Rao, a business pivots from Sociopath to Clueless leadership by mere dilution, but for Doctorow, there's a directed market pressure which eliminates (or M&As) any businesses not willing to give up some Sociopathy in favor of the more generally-accepted Clueless principles. Concretely relevant to this audience, note how Sociopathic approaches to cryptocurrency-oriented banking have failed against Clueless GAAP accounting, not just at the regulatory level but at the level of handshakes between small-business CEOs.

    Somebody could start a new flavor of Marxism here, one which (to quote an old toot of mine @corbin@defcon.social that I can't find) starts by understanding that management is a failed paradigm of production and that quotes all of these various managers (Galloway, Rao, and Zitron were all management bros at one point, as were their heroes Scott Adams and Mike Judge) as having a modicum of insight cloaked in MBA-speak.

  • If AI is so good at coding … where are the open source contributions?

    Jump

  • If AI is so good at coding … where are the open source contributions?

    Jump

  • It's been almost six decades of this, actually; we all know what this link will be. Longer if you're like me and don't draw a distinction between AI, cybernetics, and robotics.

  • If AI is so good at coding … where are the open source contributions?

    Jump

  • Perhaps! But not because we adopted vibe coding. I do have faith in our ability to climb out of the Turing tarpit (WP, Esolangs) eventually, but only by coming to a deeper understanding of algorithmic complexity.

    Also, from a completely different angle: when I was a teenager, I could have a programmable calculator with 18MHz Z80 in my hand for $100. NASA programmers today have the amazing luxury of the RAD750, a 110MHz PowerPC chipset. We're already past the gourmet phase and well into fusion.

  • Christopher Pelkey — AI ‘victim testimony’ from beyond the grave

    Jump

  • This is going to be grounds for an appeal which might reduce the sentence. I understand why people want to preserve their loved ones, but this isn't helping carry out justice.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 27th April 2025

    Jump

  • I'm sorry you had to learn this way. Most of us find out when SciShow says something that triggers the Gell-Mann effect. Green's background is in biochemistry and environmental studies, and he is trained as a science communicator; outside of the narrow arenas of biology and pop science, he isn't a reliable source. Crash Course is better than the curricula of e.g. Texas, Louisiana, or Florida (and that was the point!) but not better than university-level courses.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 13th April 2025

    Jump

  • It's the cost of the electricity, not the cost of the GPU!

    Empirically, we might estimate that a single training-capable GPU can pull nearly 1 kilowatt; an H100 GPU board is rated for 700W on its own in terms of temperature dissipation and the board pulls more than that when memory is active. I happen to live in the Pacific Northwest near lots of wind, rivers, and solar power, so electricity is barely 18 cents/kilowatt-hour and I'd say that it costs at least a dollar to run such a GPU (at full load) for 6hrs. Also, I estimate that the GPU market is currently offering a 50% discount on average for refurbished/like-new GPUs with about 5yrs of service, and the H100 is about $25k new, so they might depreciate at around $2500/yr. Finally, I picked the H100 because it's around the peak of efficiency for this particular AI season; local inference is going to be more expensive when we do apples-to-apples units like tokens/watt.

    In short, with bad napkin arithmetic, an H100 costs at least $4/day to operate while depreciating only $6.85/day or so; operating costs approach or exceed the depreciation rate. This leads to a hot-potato market where reselling the asset is worth more than operating it. In the limit, assets with no depreciation relative to opex are treated like securities, and we're already seeing multiple groups squatting like dragons upon piles of nVidia products while the cost of renting cloudy H100s has jumped from like $2/hr to $9/hr over the past year. VCs are withdrawing, yes, and they're no longer paying the power bills.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 13th April 2025

    Jump

  • I went into this with negative expectations; I recall being offended in high school that The Flashbulb was artificially sped up, unlike my heroes of neoclassical guitar and progressive-rock keyboards, and I've felt that their recent thoughts on newer music-making technology have been hypocritical. That said, this was a great video and I'm glad you shared it.

    Ears and eyes are different. We deconvolve visual data in the brain, but our ears actually perform a Fourier decomposition with physical hardware. As a result, psychoacoustics is a real and non-trivial science, used e.g. in MP3, which limits what an adversary can do to frustrate classification or learning, because the result still has to sound like music in order to get any playtime among humans. Meanwhile I'm always worried that these adversarial groups are going to accidentally propagate something like McCollough stripes, a genuine cognitohazard that causes edges to become color-coded in the visual cortex for (up to) months after a few minutes of exposure; it's a kind of possible harm that fundamentally defies automatic classification by definition.

    HarmonyCloak seems like a fairly boring adversarial tool for protecting the music industry from the music industry. Their code is incomplete and likely never going to get properly published; again we're seeing an industry-capture research group taking and not giving back to the Free Software community. I think all of the demos shown here are genuine, but he fully admits that this is a compute-intensive process which I estimate is going to slide back out of affordability by the end of 2026. This is going to stop being effective as soon as we get back into AI winter, but I'm not going to cry for Nashville.

    I really like the two attacks shown near the end, starting around 22:00. The first attack, if genuinely not audible to humans, is likely a Mosquito-style frequency that is above hearing range and physically vibrates the components of the microphone. Hofstadter and the Tortoise would be proud, although I'm concerned about the potential long-term effects on humans. The second attack is again adversarial but specific to models on home-assistant devices which are trained to ignore some loud sounds; I can't tell spectrographically whether that's also done above hearing range or not. I'm reluctant to call for attacks on home assistants, but they're great targets.

    Fundamentally this is a video that doesn't want to talk about how musicians actually rip each other off. The "tones and rhythms" that he keeps showing with nice visualizations have been machine-learnable for decades, ranging from beat-finders to frequency-analyzers to chord-spellers to track-isolators built into our music editors. He doubles down on copyright despite building businesses that profit from Free Software. And, most gratingly, he talks about the Pareto principle while ignoring that the typical musician is never able to make a career out of their art.

  • Credulous coverage of AI slop on Wikipedia

    Jump

  • For posterity: English Wikipedia is deletionist, so your burden of proof is entirely backwards. I know this because I quit English WP over it; the sibling replies are from current editors who have fully internalized it. English WP's notability bar is very high and not moved by quantity of sources; it also has suffered from many cranks over the years, and we should not legitimize cranks merely because they publish on ArXiv.

  • "Power Cut" - Ed Zitron on Microsoft's recent pullback on building servers

    Jump

  • Why is Microsoft canceling a Gigawatt of data center capacity while telling everybody that it didn’t have enough data centers to handle demand for its AI products? I suppose there’s one way of looking at it: that Microsoft may currently have a capacity issue, but soon won’t, meaning that further expansion is unnecessary.

    This is precisely it. Internally, Microsoft's SREs perform multiple levels of capacity planning, so that a product might individually be growing and requiring more resources over the next few months, but a department might be overall shrinking and using less capacity over the next few years. A datacenter requires at least 4yrs of construction before its capacity is available (usually more like 5yrs) which is too long of a horizon for any individual product...unless, of course, your product is ChatGPT and it requires a datacenter's worth of resources. Even if OpenAI were siloed from Microsoft or Azure, they would still know that OpenAI is among their neediest customers and include them in planning.

    Source: Scuttlebutt from other SREs, mostly. An analogous situation happened with Google's App Engine product: App Engine's biggest users impacted App Engine's internal capacity planning at the product level, which impacted datacenter planning because App Engine was mostly built from one big footprint in one little Oklahoma datacenter.

    Conclusion: Microsoft's going to drop OpenAI as a customer. Oracle's going to pick up the responsibility. Microsoft knows that there's no money to be made here, and is eager to see how expensive that lesson will be for Oracle; Oracle is fairly new to the business of running a public cloud and likely thinks they can offer a better platform than Azure, especially when fueled by delicious Arabian oil-fund money. Folks may want to close OpenAI accounts if they don't want Oracle billing them someday.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 23 February 2025

    Jump

  • The sibling comment gives a wider perspective. I'm going to only respond narrowly on that final paragraph's original point.

    String theories arise naturally from thinking about objects vibrating in spacetime. As such, they've generally been included in tests of particle physics whenever feasible. The LHC tested and (statistically) falsified some string theories. String theorists also have a sort of self-regulating ratchet which excludes unphysical theories, most recently excluding swampland theories. Most money in particle physics is going towards nuclear power, colliders like LHC or Fermilab's loops, or specialized detectors like SK (a giant tank of water) or LIGO (artfully-arranged laser beams) which mostly have to sit still and not be disturbed; in all cases, that money is going towards verification and operationalization of the Standard Model, and any non-standard theories are only coincidentally funded.

    So just by double-checking the history, we see that some string theories have been falsified and that the Standard Model, not any string theory, is where most funding goes. Hossenfelder and Woit both know better, but knowing better doesn't sell books. Gutmann doesn't realize, I think.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 23 February 2025

    Jump

  • It's been frustrating to watch Gutmann slowly slide. He hasn't slid that far yet, I suppose. Don't discount his voice, but don't let him be the only resource for you to learn about quantum computing; fundamentally, post-quantum concerns are a sort of hard read in one direction, and Gutmann has decided to try a hard read in the opposite direction.

    Page 19, complaining about lattice-based algorithms, is hypocritical; lattice-based approaches are roughly as well-studied as classical cryptography (Feistel networks, RSA) and elliptic curves. Yes, we haven't proven that lattice-based algorithms have the properties that we want, but we haven't proven them for classical circuits or over elliptic curves, either, and we nonetheless use those today for TLS and SSH.

    Pages 28 and 29 are outright science denial and anti-intellectualism. By quoting Woit and Hossenfelder — who are sneerable in their own right for writing multiple anti-science books each — he is choosing anti-maths allies, which is not going to work for a subfield of maths like computer science or cryptography. In particular, p28 lies to the reader with a doubly-bogus analogy, claiming that both string theory and quantum computing are non-falsifiable and draw money away from other research. This sort of closing argument makes me doubt the entire premise.