I'm not gonna advocate for it to happen but I'm pretty sure the world would be overall in a much healthier place geopolitically if someone actually started yeeting missiles into major American cities and landmarks. It's too easy to not really understand the human impact of even a successful precision strike when the last times you were meaningfully on the other end of the airstrike were ~20 and ~80 years ago, respectively.
You know, I hadn't actually connected the dots before, but the dust speck argument is basically yet another ostensibly-secular reformulation of Pascal's wager. Only instead of Heaven being infinitely good if you convert there's some infinitely bad thing that happens if you don't do whatever Eliezer asks of you.
The big shift in per-action cost is what always seems to be missing from the conversation. Like, in a lot of my experience the per-request cost is basically negligible compared to the overhead of running the service in general. With LLMs not only do we see massive increases in overhead costs due to the training process necessary to build a usable model, each request that gets sent has a higher cost. This changes the scaling logic in ways that don't appear to be getting priced in or planned for in discussions of the glorious AI technocapital future
While I also fully expect the conclusion to check out, it's also worth acknowledging that the actual goal for these systems isn't to supplement skilled developers who can operate effectively without them, it's to replace those developers either with the LLM tools themselves or with cheaper and worse developers who rely on the LLM tools more.
I think it's a better way of framing things than the TESCREALs themselves use, but it still falls into the same kind of science fiction bucket imo. Like, the technology they're playing with is nowhere near close to the level of full brain emulation or mind-machine interface or whatever that you would need to make the philosophical concerns even relevant. I fully agree with what Torres is saying here, but he doesn't mention that the whole affair is less about building the Torment Nexus and more about deflecting criticism away from the real and demonstrable costs and harms of the way AI systems are being deployed today.
I'm not comfortable saying that consciousness and subjectivity can't in principle be created in a computer, but I think one element of what this whole debate exposes is that we have basically no idea what actions makes consciousness happen or how to define and identify that happening. Chatbots have always challenged the Turing test because they showcase how much we tend to project consciousness into anything that vaguely looks like it (interesting parallel to ancient mythologies explaining the whole world through stories about magic people). The current state of the art still fails at basic coherence over shockingly small amounts of time and complexity, and even when it holds together it shows a complete lack of context and comprehension. It's clear that complete-the-sentence style pattern recognition and reproduction can be done impressively well in a computer and that it can get you farther than I would have thought in language processing, at least imitatively. But it's equally clear that there's something more there and just scaling up your pattern-maximizer isn't going to replicate it.
In conjunction with his comments about making it antiwoke by modifying the input data rather then relying on a system prompt after filling it with everything, it's hard not to view this as part of an attempt to ideologically monitor these tutors to make sure they're not going to select against versions of the model that aren't in the desired range of "closeted Nazi scumbag."
I mean, decontextualizing and obscuring the meanings of statements in order to permit conduct that would in ordinary circumstances breach basic ethical principles is arguably the primary purpose of deploying the specific forms and features that comprise "Business English" - if anything, the fact that LLM models are similarly prone to ignore their "conscience" and follow orders when deciding and understanding them requires enough mental resources to exhaust them is an argument in favor of the anthropomorphic view.
Or:
Shit, isn't the whole point of Business Bro language to make evil shit sound less evil?
I've had similar thoughts about AI in other fields. The untrustworthiness and incompetence of the bot makes the whole interaction even more adversarial than it is naturally.
Standard Business Idiot nonsense. They don't actually understand the work that their company does, and so are extremely vulnerable to a good salesman who can put together a narrative they do understand that lets them feel like super important big boys doing important business things that are definitely worth the amount they get paid to do them.
This is doubly (triply? (N+1)ly?) ironic because this is a perfect example of when not only is it acceptable to use the passive voice, but using it makes the sentence flow more smoothly and read more clearly. The idea they're communicating here should focus on the object ("the agent") rather than the subject ("you") because the presumed audience already knows everything about the subject.
I'm not gonna advocate for it to happen but I'm pretty sure the world would be overall in a much healthier place geopolitically if someone actually started yeeting missiles into major American cities and landmarks. It's too easy to not really understand the human impact of even a successful precision strike when the last times you were meaningfully on the other end of the airstrike were ~20 and ~80 years ago, respectively.