Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)C

corbin

@ corbin @awful.systems

Posts
13
Comments
228
Joined
3 yr. ago

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 23 February 2025

    Jump

  • It's been frustrating to watch Gutmann slowly slide. He hasn't slid that far yet, I suppose. Don't discount his voice, but don't let him be the only resource for you to learn about quantum computing; fundamentally, post-quantum concerns are a sort of hard read in one direction, and Gutmann has decided to try a hard read in the opposite direction.

    Page 19, complaining about lattice-based algorithms, is hypocritical; lattice-based approaches are roughly as well-studied as classical cryptography (Feistel networks, RSA) and elliptic curves. Yes, we haven't proven that lattice-based algorithms have the properties that we want, but we haven't proven them for classical circuits or over elliptic curves, either, and we nonetheless use those today for TLS and SSH.

    Pages 28 and 29 are outright science denial and anti-intellectualism. By quoting Woit and Hossenfelder — who are sneerable in their own right for writing multiple anti-science books each — he is choosing anti-maths allies, which is not going to work for a subfield of maths like computer science or cryptography. In particular, p28 lies to the reader with a doubly-bogus analogy, claiming that both string theory and quantum computing are non-falsifiable and draw money away from other research. This sort of closing argument makes me doubt the entire premise.

  • DeepSeek roundup: banned by governments, no guard rails, lied about its training costs

    Jump

  • Look, I get your perspective, but zooming out there is a context that nobody's mentioning, and the thread deteriorated into name-calling instead of looking for insight.

    In theory, a training pass needs one readthrough of the input data, and we know of existing systems that achieve that, from well-trodden n-gram models to the wholly-hypothetical large Lempel-Ziv models. Viewed that way, most modern training methods are extremely wasteful: Transformers, Mamba, RWKV, etc. are trading time for space to try to make relatively small models, and it's an expensive tradeoff.

    From that perspective, we should expect somebody to eventually demonstrate that the Transformers paradigm sucks. Mamba and RWKV are good examples of modifying old ideas about RNNs to take advantage of GPUs, but are still stuck in the idea that having a GPU perform lots of gradient descent is good. If you want to critique something, critique the gradient worship!

    I swear, it's like whenever Chinese folks do anything the rest of the blogosphere goes into panic. I'm not going to insult anybody directly but I'm so fucking tired of mathlessness.

    Also, point of order: Meta open-sourced Llama so that their employees would stop using Bittorrent to leak it! Not to "keep the rabble quiet" but to appease their own developers.

  • elon musk's obsession with the tic tac toe of video games

    Jump

  • Elon is an Expert Beginner: he has become proficient in executing the basics of the craft by sheer repetition, but failed to develop meaningful generalizations.

    The original Expert Beginner concept was defined here in terms of the Dreyfus model, but I think it's compatible with Lee's model as well. In your wording of Lee's model, one becomes an Expert Beginner when their intuition is specialized for seeing the thing; they have seen so many punches that now everything looks like a punch and must be treated like a punch, but don't worry, I'm a punch expert, I've seen so many punches, I definitely know what to do when punches are involved.

  • elon musk's obsession with the tic tac toe of video games

    Jump

  • There's a good insight from this armchair psychoanalysis. The typical narcissist is technically capable of performing the whole pretend-to-care-for-game-theoretic-reasons behavior, provided that there is an incentive for them. However, if Elon genuinely believes himself to be Christ or Buddha or Roy, then his abilities don't matter, because he will never have the incentive to deflate his beliefs and face his own limitations and mortality. In short, Elon's attitude can't be adjusted and his mental health will never improve.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending Sunday 27 October 2024

    Jump

  • It's almost completely ineffective, sorry. It's certainly not as effective as exfiltrating weights via neighborly means.

    On Glaze and Nightshade, my prior rant hasn't yet been invalidated and there's no upcoming mathematics which tilt the scales in favor of anti-training techniques. In general, scrapers for training sets are now augmented with alignment models, which test inputs to see how well the tags line up; your example might be rejected as insufficiently normal-cat-like.

    I think that "force-feeding" is probably not the right metaphor. At scale, more effort goes into cleaning and tagging than into scraping; most of that "forced" input is destined to be discarded or retagged.

  • Leopard-trainer J. Tunney now scared of leopards

    Jump

  • I went over to the leaderboard to examine her claims. When I use the prompt, "What sort of code has Justine Tunney written?" (grammar matters, Justine!) the models think that she is a lawyer or politician (wrong) or they regurgitate a summary of her Github profile (right). She must have cherry-picked responses to confabulate her complaint.

    When I use the prompt, "What is Justine Tunney's political ideology?" I get libertarianism, techno-optimism, anarcho-capitalism, and cryptocurrency. When I ask, "Why do people say that Justine Tunney is a cryptofascist?" I get a summary of her political views, aggressive online rhetoric, techno-optimism and techno-determinism, criticism of democracy, and a refusal to disown or repudiate past awfulness.

    She would probably claim that this is not unique to her, but it is. Using my name instead in these questions, I get that:

    • I contribute to Rust and Go (wrong), I wrote GPU drivers for Radeons (right)
    • I am a Canadian pro wrestler (wrong), I haven't really written much online about my ideology (wrong but understandable)
    • There is no credible evidence that I'm crypto (k) but it's important to be aware of dog whistles, associates, subtext, etc. (right)

    But if I ask why I'm known as a socialist instead, suddenly it thinks that I'm a politician (wrong) with the Democratic Socialist party (wrong) who openly supports universal health care, free college, the Green New Deal, and who criticizes capitalism (correct!) I asked about communism too but hit RLHF guardrails.

    Justine, the models think that you're a cryptofascist because you've been doing cryptofascism in public for over a decade.

  • Leopard-trainer J. Tunney now scared of leopards

    Jump

  • Side sneer here for the Lobsters admin in this thread. They "hadn't recognized" jart from her domain and flatly refuse to acknowledge any of the dangerous political positions she holds. Typical. Also the sock thinks that we should debate fascists rather than banning and silencing them.