What the fuck is SpaceX using a large language model for?
- Presumably a document query system, one of the actually useful reasons to use an LLM.
  Having it trawl through thousands of pages of NASA and government files to answer regulatory questions is probably legitimately helpful, even if all it does is point you towards the right pages for a human to review.
  
  That makes sense. Like you, I’ve generally found that LLMs are incredibly useful for certain, highly specific things but people (CEOs especially) need to understand their limitations.
  When it first came out, I purposely used ChatGPT on a trip to evaluate it. I was in a historic city on a business trip where I stayed an extra few days so I was traveling alone. It was good at being a tour guide. Obviously, I could have researched everything and read guidebooks but I was focused on my work stuff. Being able to ask follow-up questions and have a conversation was a real improvement over traditional search.
  That’s obviously a limited use case where I was asking questions that could have been answered in traditional ways but I found that to be a good consumer use case. It knew details that wouldn’t necessarily be in a Wikipedia article or Guidebook that would take me 15 Google searches to answer. Just my own little curiosity questions about an old building or whatever. I cross-checked things later and it didn’t hallucinate. Obviously, a very limited use case but it was good at it.
  
  This is the one I'm waiting the most from the LLM hype. It would be a massive benefit for companies around the world (mine included) if they could just dump their documentation in all shapes and flavours into a model and have it parse standardized documents for you.
  But the generic OpenAI/Copilot models aren't reliable enough just yet, hallucinations and made up data just doesn't go with that. I'm not even sure if those models are ever capable on such task alone, maybe it needs additional component which checks the facts from originals or something to make it actually useful.
  
  I created a relatively complex board/card game and just shoved the rulebook PDF into an LLM, so it’d be easy to answer people’s questions with exact references to cards and rulebook pages. So far it’s successfully corrected ME twice on specific rulings. It’s actually quite useful in this regard.
  
  Generic models are not ready yet and won’t ever be but my company is using something from AWS called “Q” I think (heh, great name this day in age) but because it performs training rounds (*I think!) on a set of documents you define (like process documentation) its actually pretty good at finding exactly the information and suggesting additional reading. We’re using it on a process guide of a couple hundred pages; not sure how it scales to other sizes or what the cost is.
- munching on all that sweet government data that's been swiped by the muskrat goon squad.
- So that their LLM looks like it's getting buisness
- I... You think high tech companies should just be sending all their data to OpenAI, Anthropic, and whoever else?
  
  I didn’t think they should use A.I. yet at all. I don’t think the shitty version of machine learning of today is ready for engineering giant explosive things. As someone else pointed out, document management for regulatory filings and stuff is (hopefully) the use case. I don’t care if it’s used in that way.
  Basically, I think today’s “A.I.” should be treated as alpha software. It has a ton of potential but there is a lot left to do, especially on things involving human or even critter life like rocket science, self-driving cars, or military applications where “edge cases” are life or death situations. (I don’t think it should be used for military applications until it’s really fucking mature tech but it’s already apparently being used for that so the cat’s out the bag there.)
  
  I do not think that even Elon has enough power at SpaceX to force them to design next-generation rockets using Large Language Models. Engineers know better than that. Their next rocket wouldn’t even make it off the pad.
  
  Engineers know better than that.
  I've known too many idiots with PE licenses to share your optimism.

12 comments