5mo ago

average day in NPM land

126 comments

I've literally told my coworkers "I'm not saying we should never use dependencies. But every time you add a dependency, you should hate yourself a little bit more. Some self flagellation can't hurt either."
- So, every time I use a library to recognize patterns on a picture, to interact with Kafka, do some SSL, or do database mapping, I should hate myself, noted
  We did Elastic API integration in Java by creating and maintaining huge half-codegenerated transformer from code to Elasticsearch's JSONs, it was a pain and it was source of more than one error
  Dependences should be reviewed and audited to make sure they do what you need and they are worth using. Just making everything in-house gets you nowhere most of the time
  
  Nobody is arguing that you should never depend on anything and create everything yourself, but adding a dependency for literally a one liner function is awful. Like one of the Go proverbs goes, a little copying is better than a little dependency.
  
  You code in Java, of course you should self flagellate on a daily basis just for that. The entire ecosystem is completely fucked.
  
  That is what they were getting at by some self flagellation can't hurt either. That sometimes "hating yourself" (adding dependencies) is worth it.

I've honestly never understood why someone at Google or Mozilla hasn't decided to write a JavaScript Standard Library.
I'm not opposed to NPM, because dumb shit like this happens everywhere. If such a package is used millions of times a day, perhaps it would make sense to standardise it and have it as part of the fucking browser or node runtime...
- I’ve honestly never understood why someone at Google or Mozilla hasn’t decided to write a JavaScript Standard Library.
  
  This picture honestly looks more like C++ than JS, and before you yell at me, JS doesn't have any standards let alone competing standards so ¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯
- someone at Google or Mozilla hasn't decided to write a JavaScript Standard Library.
  Core APIs (including data types like strings, collection types like Map, Set, and arrays), Browser, and DOM APIs are pretty good these days. Much better than they used to be, with more features and consistent behaviour across all major browsers. It's uncommon to need browser-specific hacks for those any more, except sometimes in Safari which acts weird at times.
  The main issue is server-side, and neither Google nor Mozilla have a big interest in server side JS. Google mostly uses Python and Java for their server-side code, and Mozilla mostly uses Rust.
  Having said that, there's definitely some improvements that could be made in client-side JS too.
- There's a js runtime called bun that is 90-something% feature equivalent to node and also has built in alternatives to many packages like express and bcrypt. I haven't used it myself so I can't speak to its quality but it's always nice to see a little competition
  
  So is Deno! You can easily import npm: and node: packages and run typescript without transpiling. With Bun and Deno there's no reason to use Node tbh.
  
  Bun is used by us in production, in dev, everywhere. It's great. We don't even use (p)npm to build js packages on our docker images for apps anymore.
- That's basically how Javascript gets extended. I put off learning jQuery for so long that all the features I'd want are now standard.
  
  Vanilla JS is pretty good on the client side, but leaves a lot to be desired on the server side in Node.js, even if you include the standard Node.js modules.
  For example, there's no built-in way to connect to a database of any sort, nor is there a way to easily create a basic HTTP REST API - the built in HTTP module is just raw HTTP, with no URL routing, no way to automatically JSON encode responses, no standardized error handling, no middleware layer, etc.
  This means that practically every Node.js app imports third-party modules, and they vary wildly in quality. Some are fantastic, some are okay, and some are absolutely horrible yet somehow get millions of downloads per week.
- I wish they would replace JavaScript with something that was made for what it's used for. JavaScript should have died years ago.
  
  Sometimes it's hard to detach It, specially dealing with web dev.
  The browser expects JS, since JS was made for the browser, so you make a front in JS. But now you need a back, and hey, you already have all models and repos in JS, might as well make the back with JS.
  It's a vicious cycle. Honestly, JS is fine for either if you are component enough (ie. not using stuff like "is-number"), don't get the hate on It.
  
  I'm still waiting for webassembly to take off
  
  Let's use Flash!
- Support for int64s out of the box and without jumping through hoops would be nice.
- core-js is exactly that, but it focuses on implementing baseline features that can be polyfilled.

The only two people arguing against the change were both authors/contributors of is-number lol
- How many contributors could there possibly be
  
  3, about two lines per contributor
- Is it because they included a crypto miner in the package?

Note that the PR was later merged by a member who got fed up with his colleagues.
- And who hasn't contributed any code to this particular repo (according to github insights).
  
  Not familiar with this exact team, but a skilled reviewer/issue triager is useful. We can hope this person at least tested the changes.

Another day of being extremely thankful I decided not to learn JavaScript
- I mean, the people relying on such packages didn't learn it either.
- Look at what you're missing!
  
  Edit: also, is-odd depends on is-number
  
  These are both made by the same person from this PR (who also made both the package the PR is on, and the is-number package that is being removed as a dep)
  
  Heaven forbid they make a package is-even-or-odd with both. Wait. Don't give them ideas. They'll just make it depend on both.
- This can happen in any project that uses dependencies, javascript or not
  
  Sure, but when was the last time you saw, say, a Python project using some third-party library instead of simply calling isnumeric() from the standard library?
  There's a reason for these jokes always being about Javascript.
- I only glossed over it… but this looks like it’s trying to check dynamic typing issues? It’s like a statically typed language with extra steps?
  
  I don't think typescript exists because JavaScript wasn't designed to be statically typed. I think Typescript exists because JavaScript wasn't really designed, period.
- How’s the view up there on your high horse?
  
  Must be pretty good, considering literally every time I check in on the JavaScript community it is somehow more on fire than it was last time. I guess I must have a front row seat to all their misfortune. Either that or they're just incompetent, but it couldn't be that, could it?

If you think is-number can be replaced with a one-liner, you don't have the enterprise code mindset. What if the world gets more inclusive and MMXXIV, ½ and ⠼⠁ become recognized as numbers? 𒐍𓆾 were numbers in the past but what if people start assigning numeric value to other characters? Are 🖐🔟💯🆢🂵🀌🁅 numbers of the future???
/s
I'm not even all kidding, Regex implementations are split on whether "٣" matches \d.
- All junior devs should read OCs comment and really think about this.
  The issue is whether is_number() is performing a semantic language matter or checking whether the text input can be converted by the program to a number type.
  The former case - the semantic language test - is useful for chat based interactions, analysis of text (and ancient text - I love the cuneiform btw) and similar. In this mode, some applications don't even have to be able to convert the text into eg binary (a 'gazillion' of something is quantifying it, but vaguely)
  The latter case (validating input) is useful where the input is controlled and users are supposed to enter numbers using a limited part of a standard keyboard. Clay tablets and triangular sticks are strictly excluded from this interface.
  Another example might be is_address(). Which of these are addresses? '10 Downing Street, London', '193.168.1.1', 'Gettysberg', 'Sir/Madam'.
  To me this highlights that code is a lot less reusable between different projects/apps than it at first appears.
- It's simple ⅯⅯⅩⅩⅣis a number, MMXXIV is not.
  
  You may argue that writiing 2024 as "MMXXIV" and not "ⅯⅯⅩⅩⅣ" is a mistake but while typists who'd use "2OlO" for "2010" (because they grew up using cost-reduced typewriters) are dying out, you'll never get everyone to use the appropriate Unicode for Roman numerals.
- So the only valid digits are arabic numbers but arabic script numbers are not a valid digit? If we want programming to be inclusive then doesn't that make sense to also include the arabic script number?
  
  So the only valid digits are arabic numbers but arabic script numbers are not a valid digit?
  Some people writing Regex implementations have that opinion. I've refrained from saying mine.
  If we want programming to be inclusive then doesn't that make sense to also include the arabic script number?
  Maybe. IMO, number tests should be chosen/implemented based on the project's requirements. If you want to include every Unicode character or string pattern anyone's ever used to convey a numeric value, that would be a long and growing list. Arguably, it's impossible: the word "elf" means a number if interpreted as German for "eleven" but not if interpreted as English for 🧝.
- Lisp code is already like this. That's why I keep trying to explain it to programmers. Try reading the book SICP, published decades ago by MIT computer researchers.
  https://invidious.privacyredirect.com/watch?v=a3t3IKlXqFU
- Are you asking for treefiddy upvotes?
  
  How many upvotes does 💲🄄Ƽ᱐ buy, really?

It looks like "is-number" was never more than a few simple lines of code. It still has 68 million downloads per week.
https://www.npmjs.com/package/is-number
I checked one of our main projects at work, and it's in there as a dependency 6 levels deep via the "sass" package.
- is-number is a project by John Schlinkert. John has a background in sales and marketing before he became an open source programmer and started creating these types of single function packages. So far he has about 1400 projects. Not all of them are this small, though many are.
  He builds a lot of very basic functionality packages. Get the first n values from an array. Sort an array. Set a non-enumerable property on an object. Split a string. Get the length of the longest item in an array. Check if a path ends with some string. It goes on and on.
  If you browse through it's not uncommon to find packages that do nothing but call another package of his. For example, is-valid-path provides a function to check if a windows path contains any invalid characters. The only thing it does is import and call another package, is-invalid-path, and inverses its output.
  He has a package called alphabet that only exports an array with all the letters of the alphabet. There's a package that provides a list of phrases that could mean "yes." He has a package (ansi-wrap) to wrap text in ANSI color escape codes, then he has separate packages to wrap text in every color name (ansi-red, ansi-cyan, etc).
  To me, 1400 projects is just an insane number, and it's only possible because they are all so trivial. To me, it very much looks like the work of someone who cares a lot about pumping up his numbers and looking impressive. However the JavaScript world also extolled the virtues of these types of micro packages at some point so what do I know.
  
  Wow you're right, he's the author of the infamous "is-odd" and "is-even" packages. What an odd person.
  Someone in the OP PR mentioned the amount of energy used to download these tiny packages and its actually something crazy to think about
  
  To me, 1400 projects is just an insane number, and it's only possible because they are all so trivial.
  Holy shit. I'm going to have to go through my team's dependencies. I don't feel confident that someone "maintaining" that many projects is going to be able to keep all bad actors at bay. Not to mention, none of the examples of his libraries that I've seen SHOULD be libraries.
- What a sassy package depency

Link to the PR? The PR description and the comment somehow contradict each other. Or I am stupid. Or the commenter.
- I tried to edit the 'highlights' into a single image, the top is the description of the PR, the middle is a comment replying to another comment
  https://github.com/micromatch/to-regex-range/pull/17
  
  Thanks!
  What a shit show.

Seems to me the only reason for these kind of dependencies to exist in the first place is that people really, really, really, REALLY can't code.
- Well javascript is the default language of the web, so no surprise it attracts a lot of newbies.
  
  Yeah but if noobs use it as a dependency, who made the package?
  And what projects are noobs working on that trigger 440GB of weekly traffic?
  I fear most noobs remain noobs.
- this is exactly the kind of thing the senior engineers above me at my last job would use.. Their code made me cry

440GB weekly for "is number". What in the world is that package doing?
Anti Commercial-AI license
- is-number is a one-line function. (though it's debatable if a function that complex should be compressed to one line)
  You may have heard of a similar if more extreme "microdependency" called is-even. When you use an NPM package, you also need all the dependencies of that package, and the dependencies of those dependencies recursively. Each package has some overhead, eventually leading to this moment in time.
  
  Web bloat in a nutshell and why we need to switch to things like Web Assembly more than ever. It's not WASM, but I used Laminar which is a Scala.js library, and it's the absolute pinnacle of (frontend) web development. Scala in general is just really great for idiomatic web code, its flexibility is unbeatable.
  Another amazing alternative would be anything Rust. In fact I've used that much more than Scala for web. I've mainly used Leptos for full-stack and and Actix for backend, but I've seen Dioxus and Axum in good use and they both seem really great too.
  Apparently Lemmy uses Leptos for its UI so... that's a +1.
- It handles a few weird edge cases, mostly. Only 7 meaningful lines of code and almost 70M downloads week!
  
  Sadly, it's a stupid dependency of a lot of things.
  Just ran npm explain is-number on one of my projects, and it's a dependency of to-regex-range which is a dependency of fill-range which is a dependency of....and so on up the chain.
  I was hoping I wouldn't find that in there, but alas, it is.
  
  I don't get the concept that depending on 7 lines of code from a third-party package is remotely acceptable. It's expanding the potential attack surface to save a dev from templating 7 lines of boilerplate. There's no net benefit or appreciable time saved.
  I'm glad I don't have to deal with this regularly.
  ETA: The package is even MIT licensed! There's no excuse but laziness and not wanting to understand the code to import this rather than inlining or implementing a novel version. If I can spend the time to write:
  undefined
  
  if err != nil { slog.Warn("well shit", "error", err) return err }
  after every function call...I just didn't get it.
- I'm not sure, this is a valid estimate. If they were to replace is-number with its contents, that would mean that the economy is only in HTTP-related overhead.
  It maybe will make difference because of building phase, lock-files, package-files, but I am not sure that data-traffic difference is that big

I really need a community here solely dedicated to GitHub drama. This is so much better than Twitter drama, more relatable.

It's kind of insane how bad this whole is-number thing is. It's designed to tell you if a string is numeric, but I would argue if you're ever using that you have a fundamental design problem. I hate dynamic typing as much as anyone else, but if forced to use it I would at least try to have some resemblance of sanity by just normalizing it to an actual number first.
Just fucking do this...
javascript

const toRegexRange = (minStr, maxStr, options) => { const min = parseInt(minStr, 10); const max = parseInt(maxStr, 10); if (isNaN(min) || isNaN(max)) throw Error("bad input or whatever"); // ...
Because of the insanity of keeping them strings and only attempting to validate them (poorly) up front you open yourself up to a suite of bugs. For example, it took me all of 5 minutes to find this bug:
javascript

toRegexRange('+1', '+2') // returns "(?:+1|+2)" which is not valid regexp
- The problem is the underlying API. parseInt(“550e8400-e29b-41d4-a716-446655440000”, 10) (this is a UUID) returns 550. If you’re expecting that input to not parse as a number, then JavaScript fails you. To some degree there is a need for things to provide common standards. If your team all understands how parseInt works and agrees that those strings should be numbers and continues to design for that, you’re golden.
  
  Yeah good point. I suppose the problem is this function that operates on numbers allows numeric strings to be passed in in the first place. The only place where I would really expect numeric strings to exist is captured directly from user input which is where the parsing into a numeric data type should happen, not randomly in a library function.

i think programmers need a self inflicted rule of it being less than 500 lines of code means you need to write it instead of using a pre written package/library.
On the other hand, we could make the packages like is-number the worst possible way of checking if something is a number, which would be really fucking funny...
- I have read programs a lot shorter than 500 lines which I don't have the expertise to write.
  
  Shell scripts don't count
  That's not a programming language, that's hieroglyphs
  
  well obviously shit like wozmon exists, but there definitely needs to be a rule to prevent handicap shit like "is-number"
  the vast majority of programmers have the problem solving capability of a child who took shit apart constantly due to autism.
- 500 is fucking massive maybe try 5
  
  true, i was giving the benefit of doubt to idiot programmers lol
- programmers need a self inflicted rule of it being less than 500 lines of code means you need to write it instead of using a pre written package/library.
  That's what I do, but then I end up with similar utils across multiple projects (eg some of these array, map, and set utils: https://github.com/Daniel15/dnstools/tree/master/src/DnsTools.Web/ClientApp/src/utils) and wonder if I should create a library.
  Then I end up doing that (https://github.com/Daniel15/jsframework is my most 'recent' one, now very outdated) but eventually the library gets outdated and you end up deleting most of it and starting again. (edit: practically this entire library is obsolete how)
  It's the circle of life.
  
  i wonder if maybe we just need personal package repos for shit like this, stuff that probably shouldnt be out on the internet and accessible, but that's also worth packaging for regular use. Like a sort of "code macro" which is something i see people doing a lot for certain boilerplate strings.
- Eh, I can see why you'd want something like that in a language like JavaScript where pretty much all native ways of validating input have weird edge cases. Sometimes you just want the community to figure it out for you instead of reinventing the wheel and finding out you missed something later on.
  A whole package that handles validation of inputs, or a math package would be better than a package that just has one function tho.
  
  yeah, generally it seems like you want a more broad package, if for something like validating input, it would be comprehensive across all input for example.
  There's one package on npm called is-even and i think another called is-odd, which according to the author are "learning experiences" which have, considerable amounts of downloads, even though it's literally just is-even checking. Shit like that should probably get you banned from using keyboards for the rest of your life lol

126 comments