1y ago

True

You're viewing a single thread.

119 comments

The worst part of ML is Python package management
- Yeah, I feel like Python is partly responsible for most of this meme. It's easy for very simple scripts and it has lots of ML libraries. But all the stuff in between is made more difficult by the whole ecosystem being focused on scripting...
- The worst part of ML is Python package management
  Do you have some time to talk about our Lord and Savior, venv?
  
  Had to use wsl and manually set environment variables to get accelerate and bitsandbytes to work the other day, why can't pip install just work? Venv is just another layer that conda should be solving, and even that isn't enough to overcome Python's craptastic nature
  
  At that point you may as well go full Vagrant or start using Docker images.
  And no matter how quirky or obtuse venv/conda/pip can be, they will never be as bad as Node. Ever. Node will hold that King Shit crown forever, or at least to God I hope it does.
  Something worse than Node coming around and getting popular might just make me quit IT altogether.
  
  Interesting, my problems with node are usually in the chaining build systems rather than pulling down dependencies. Tbh I prefer node, what problems have you with it?
  
  Aside from the callback chains and API shit, my issues with Node rest almost entirely on the lack of a standard library, because that led to the state of NPM today, which is just an absolute garbage-fire shitshow as far as I'm concerned.
  I have my own separate issues with NPM, namely its dependency resolution (my God, just take dnf's dependency resolution algorithm and use it), trivial packages that other packages list as a dependency (is this an int? Is this running on Windows? Better take this one line and make it a package!), and the relative inability to remove a package from a registry (did a secret slip in there while testing? Tough shit!). The worst of that being the trivial packages, I think, because then you can end up with projects that can have a dependency tree 10s of thousands packages long.
  And all that bullshit wouldn't be even 1/16th of the problem it is today if there were a standard library.
  You should take what I'm saying with a grain of salt, though, I'm just a DevOps Sysadmin, and aside from running some software that uses Node, most of my experience with it is unfucking it when our devs come to me to fix the tangled monster they've created.
- Poetry gang
- Nah, it's pushing inference to prod. Any idiot can make an ipynb to train a model, but packaging everything into an app ecosystem is where you actually need a lot of non-ML software engineering know-how.
  
  So true. I’m on an AI product team. None of the engineers know that much about learning/ai — our expertise is in high availability/scalability/distributed systems.
  The AI part it when a data scientist hands us a notebook and says: implement this algorithm.
- Underrated comment!!!

119 comments