One-line node modules
hemanth opened this issue ยท comments
Lot of people don't get the benefits of one line modules, what would you say to those who tend to critic a lot about node modules that have just a line in them?
I'm curious about this too, I see some modules and wonder why they're at all necessary.
This one (not by you) is a good example.
I'm also curious about this. In particular, referring to user-home as one of your examples?
tl;dr You make small focused modules for reusability and to make it possible to build larger more advanced things that are easier to reason about.
This is why I wish you had a blog :p I never was able to explain it that well. Good job.
๐
I guess my question (and this is a legitimate question as I'm relatively new to nodejs!) was, do you see a legitimate reason for splitting user-home out from os-homedir? I mean, obviously you do as you've done it that way, but I was wondering why you just didn't just include a cached version in the original os-homedir
I often find that these simple tasks, like finding user's home folder, turn out to be very non-trivial and hairy the more the code gets used. Making a separate module right away allows to properly test and fix bugs, add features etc, as new issues absolutely will arise.
๐
I guess my question (and this is a legitimate question as I'm relatively new to nodejs!) was, do you see a legitimate reason for splitting user-home out from os-homedir? I mean, obviously you do as you've done it that way, but I was wondering why you just didn't just include a cached version in the original os-homedir
The user-home module was made long before os-homedir. os-homedir was made recently as a polyfill of the forth-coming built-in API in Node. I just made user-home depend on it so it will benefit from the native API. I could of course deprecate user-home now, but there are so many projects depending on it now, it really isn't worth it. I also like how in user-home it's cached by default instead of having to call it as a function. However what you're asking isn't really relevant to the small module thinking. This is just a side-effect of evolution.
Ok, thanks for the clarification! That makes a lot of sense. I just stumbled upon user-home today and my curiosity aligned perfectly with this AMA.
I often find that these simple tasks, like finding user's home folder, turn out to be very non-trivial and hairy the more the code gets used.
๐ Every time :p
Everyone; Keep in mind that I've failed doing the right thing hundreds of times before realizing making a separate module would be a better thing.
Everyone; Keep in mind that I've failed doing the right thing hundreds of times before realizing making a separate module would be a better thing.
This is always helpful to hear: that one's successes are generally built upon the rubble of past failures.
Just stick this reply on Medium and call it a day :)
Seriously. Copy/paste to Medium dude. There are legit insights here that wouldn't require more formatting than what you have here :)
๐ Great explanation
My beef with one liner modules is that I still have to rely on other people doing their job well. Take for example the is-positive module posted by @daryl. I just had to file kevva/is-positive#3 against that because regardless of whether the author was even aware of the distinction, it may have been relevant for your case. If you relied blindly on that module then you may now have a bug in your code. The author was aware, apparently made aware twice, and chooses not to support this feature. Of course that's his choice. I do have opinions about that. Care.
The building block approach works well if there's a tight community that moderates these modules and puts them in a pool with the "moderator approved" logo. This may be a problem npm is facing; no quality control. Back to the jquery crap plugin era. And I still have to roll my own because existing modules do not suffice. But when asking about including trivial stuff in the framework (node) I get the standard "use npm for that". Ciiiiircles argh.
Somebody start a community moderated npm module collection site/thingie/whatchamacall it please. Quality control. How can you even write reliable node apps with out it :/
The building block approach works well if there's a tight community that moderates these modules and puts them in a pool with the "moderator approved" logo.
All the success of npm seems to point to the contrary. npm beat its early competitors because there was no gatekeeper on publishing. Node itself has enough well understood compatibility patters (error first callbacks, streams, etc) that modules can be compatible without specific coordination between the authors.
Arguments about quality seem to center around the idea that a large community has less quality than a small one. This is always true as a matter of percentage (90% of a small community might be of higher quality than all but the top 10% of a larger community) but in a large enough ecosystem this doesn't matter because 10% dwarfes all smaller curated communities.
You still have a discovery problem in large ecosystems, but that is mitigated with investment. The people I see complain most about it aren't investing in it so they can never find what they want. Instead they lock in to tightly integrated frameworks that are vertically integrated and curated, which is fine if that is your preference, but to say that these smaller curated ecosystems are of higher quality than the modules you find in a larger ecosystem once you've invested is simply untrue.
The building block approach works well if there's a tight community that moderates these modules and puts them in a pool with the "moderator approved" logo. This may be a problem npm is facing; no quality control.
You kinda do the quality control yourself when choosing a module. How does the readme look? Who built this module? It it well tested? All don't guarantee anything, but are pretty good parameters to see if something is reliable. That's something that rarely happened back when there was "jQuery crap." We (as in developers) just cared if it looked good.
You kinda do the quality control yourself when choosing a module. How does the readme look? Who built this module? It it well tested?
I would add "does this do only one thing?" and "is the usage a single, simple, function?" because if all my modules fall under these guidelines my application is much easier to reason about. This also re-enforces the single line module argument.
For the record, there is one single line module I won't use and that is @visionmedia's noop module because I actually find it is better to simply do that one inline for readability (not to mention performance).
var noop = function () {}vs.
var noop = require('noop')Yes, @mikeal, I totally agree. I was just giving some examples, the API design should definitly be an important point too. There are way too many bloated modules out there.
@mikeal; Using the success of npm as some kind of de facto counter-argument in this context seems unsound to me. There may be many reasons for it being popular; its users may not care that much about the problem I outlined.
Large vs small communities; there's another actor at play here. The 10% may dwarf the 90% but if they are so scattered and relatively unorganized, it would still not mean they outperform the small one. Think of it the Romans versus the barbarians, if that comparison makes any sense ;)
Investment.. well that may be a whole discussion on its own. One I see myself losing quickly. My main rebuttal would be that if I'm coding a node(/io/bla) app, and need some trivial functionality not exposed by this framework, I have only two choices: roll my own or scrounge npm. Both can take a lot of time because if it's cumbersome to roll my own, it'll probably be just as cumbersome to inspect and pick the right module. And still hope it didn't mess up.
Small modules that do one thing (tm) do mitigate this problem a bit, by design. But even then there are simple cases, easily overlooked, like the is-positive example earlier (oh I'm sorry to that author, I really don't mean to bash on him).
@arthurvr: while maintenance of a module as a whole (docs) is some indication, it all comes down to the code and test cases. When this matters, it comes down to a cold hard code review. And in the end it may still mean you can't use the module because it's indeed insufficient (but of course you always have that risk) or because there's a bug of sorts.
So. I didn't intend this to be a Duty Calls. It wasn't meant like that at all. It's my two cents on why I have trouble with using npm reliably. It may not be relevant for others. So I'll leave it at this.
I think we should make a distinction here.
user-home is actually sweet, because it's based on a purpose: providing the user's home directory. It may be a one-liner, but it could be different in the future. Now we know it won't because Node will take care of the problem, but it's just an example.
On the other hand, negative-zero doesn't convince me. Because it's a one-liner, and it's based on something that won't change anytime soon (i.e., Javascript basic arithmetic specs).
I could use that module. But I also could define an isNegativeZero function on my own in less than 30 seconds, and use that instead. So, which is better?
Using well-known modules saves us from the hassle to fix bugs, writing unit tests and update the code to meet future specs. But for a check like that you might as well expect that there won't be any bugs ever, and no changes either, since the specs are very much frozen on the topic.
Of course, I can always make a mistake writing isNegativeZero (that would also mean that my caffeine level is dangerously low, having an impact on many other things), but it may also happen that I picked the wrong module (because I mistyped it when doing npm), or the wrong version, or even that the maintainer made a silly oversight.
Second point, time spent: my <30s function definitely took me less than the time I'd need:
- googling if there are modules for my needs;
- checking one or some of them, reading the docs;
- verifying if it's actually good for my purposes with some tests;
- including it in the project and write a concise documentation in the readme for the other members.
And I might end up with a module with an obscure name (because the best names were already taken, or because the author doesn't speak English well, or is just an evil mastermind) instead of my well-named isNegativeZero function that needs no explanation.
Not to mention the time the other members must spend to get to know the new module.
But, above all, what would worry me the most is the proliferation of dependencies. A package.json file for a common medium-sized project usually include dozens of dependencies. What's worse, we currently have no way to group them in scope categories: they're all there, sort of "globally defined" modules (relative to the project, of course).
If we lower the complexity of the modules we know down to the most basic things, the list of dependencies might grow exponentially. This can be a problem for projects with many active members, that might end up knowing only a part of the modules used and ignoring the others. Imagine the scenario: "Dude, I require'd this xxx-yy for that, why didn't you use it/did you install yyy-xx instead?" "Sorry, I didn't know/I had no idea you used that/was another module really necessary?"
It's a matter of team coordination.
Oh, but you can say: "Well, if we don't like a big pool of modules, I can make a module that collects all the modules in the same area. Let's start with some math-utils.js, which requires negative-zero, and is-positive, and..."
And you stop, realizing that if you're forced to make a module to organize your one-liner modules, you might as well write the one-liners by yourself in your module.
So, in the end, we all like modules, we like them to no end especially if they're well made packages, with docs, tests, comments, changelogs, semver support and all, but the risk of abusing this system is concrete and should be considered. The control of the components of the probject and thus the coordination of the team might suffer.
Probably what we really need is new tools to handle our modules. Quality control tools, above all.
@qfox Sometimes you will find modules that do almost exactly what you need, but not quite exactly. In these instances, you might be able to nudge the author to improve/document/change/etc the module. Or, you can spend 5 minutes to write your own module, e.g. is-positive-or-zero and be done with it. After which, anyone else stumbling along the same problem as you will now have a clear path ahead of them.
When you embrace the "small module" philosophy you should also be ready to get your own feet wet, and publish your own modules as needed. ๐ Most of the people advocating for small modules (substack, mikolalysenko, hughsk, sindresorhus, etc) are also writing a lot of their own.
But I also could define an isNegativeZero function on my own in less than 30 seconds, and use that instead. So, which is better?
Do people really know how to write this function by heart? It is not something I ever hope to memorize or be quizzed on, and probably not something I could pull out of thin air in under 30 seconds.
x === 0 && 1 / x === -Infinity
Either way; once you know the module exists and is good, it will almost always be faster (and safer) to just npm install it. ๐
Do people really know how to write this function by heart?
Huh, maybe it's just me, but I suppose they do. Once you get to know that in Javascript there is a "negative zero" (and that's the shocking part), the next question should logically be "how can I tell a positive from a negative zero if they bot === 0?" and the answer is "if you divide a positive number by -0, you get -Infinity."
But it's just an example. Consider the following as an alternative:
module.exports = exports = function isSameDay(date1, date2) {
return date1.getFullYear() === date2.getFullYear()
&& date1.getMonth() === date2.getMonth()
&& date1.getDate() === date2.getDate();
};Either way; once you know the module exists and is good, it will almost always be faster (and safer) to just
npm installit. ๐
Sure! Once you know it. And if you're alone in the project and never have to write docs about its introduction and all that boring yadda-yadda.
Otherwise, copy&pasting a function and maybe writing a small docblock for it might be faster.
Also, to be honest, I already have to memorize a ton of language specs. Remembering a plethora of micro-modules for every small thing would be another burden on my ever-waning memory.
But I also could define an isNegativeZero function on my own in less than 30 seconds, and use that instead. So, which is better?
You're still going to have to find the snippet somewhere and ensure it works correctly. It probably doesn't have unit tests if it's a random snippet on StackOverflow, so you'll have to manually write this or trust a random StackOverflow user. Instead, you could search npm, do npm i -S negative-zero, require it and be done.
I'm gonna parrot myself on this:
npm is now my snippet database
I do agree discovery on npm is a problem though.
I do agree discovery on npm is a problem though.
It is, indeed. That's why I said we need more tools.
I'm thinking about the tags, for example. I can use whatever tags I want in my npm packages. Who's going stop me? What are the tagging best practices?
What do a module's tags actually tell me about the module itself?
I can go on, but you surely got the point.
"how can I tell a positive from a negative zero if they both === 0?"
A year ago my first instinct facing the unknown would be to search for a random StackOverflow snippet and hope it works. Now, my instinct is to search for an npm module and clearly see that it will work with my application's inputs by looking at the test.
Regarding memory; I would think it is much easier to remember the words negative-zero than the previously mentioned function body (which looks weird and is not very intuitive). To each their own.
BTW - a lot of the smallest modules you see (e.g. this or this) are largely for the author's own consumption. I published those modules so I could stop copy + pasting the functions from project to project. It just so happens that they can also benefit a lot of other people.
Regarding memory; I would think it is much easier to remember the words negative-zero than the previously mentioned function body
I disagree on this. The problem is that you might remember "negative-zero", but it was actually "is-negative-zero", or "negative_zero". Oh, look, there are both "negative-zero" and "negative_zero": which is the one I'm looking for? And what about this "negzero"?
On the other hand, you don't have to remember the function's body: you remember the specs. You define the function using them.
BTW - a lot of the smallest modules you see (e.g. this or this) are largely for the author's own consumption.
Yes, that's indeed true. And the sad thing is that they're most probably going to lie there unused by anyone else, because other people just don't know they're there! Now, how many Matt's or Sindre's are going to do the same? I.e., using npm as a "snippet database"?
What if everyone is going to use the same practice? Isn't that going to pollute npm's package collection with myriads of micro-modules, making it even more difficult to discover them?
And the sad thing is that they're most probably going to lie there unused by anyone else, because other people just don't know they're there!
Not really. Anyone looking for these functions will see them immediately: is power of two, is negative zero, lerp. (Of course, if developers continue copy + pasting from StackOverflow instead of using npm, they will remain unused!)
You might need to compare a couple of modules, but this doesn't take long. If you find something that suits your needs, you obviously don't need to duplicate it under a different name.
It's not just a snippet database - it's a shared snippet database.
P.S. Sindre, apologies for derailing your AMA! ๐ This is a great idea.
inb4lock
This makes your code more functional.
@sindresorhus, totally agree in concept but with one caveat, we need much better tooling that can cut the time taken for selecting and assessing modules so that better choices can be made up front.
For instance (unless I'm missing sometheing) if I'm not sure what a module name is but know it's to do with testing a zero value, if I search npm for 'zero' or 'negative zero' it would be great to be able to rank and filter by popularity, downloads/month/week/etc, activity, last updated, build status, issues, contributor rating and/or other metrics... This would really help improve that initial meh when you need to find a module to do x for the first time.
Maybe @isaacs can point to how this can be done/improved?
๐ ;)
Great read, Sindre! If you end up turning this into a blog post (please do), might I recommend including a link to FIRST? That one also deserves more attention.
It's now a blog post over at: http://dailyjs.com/2015/07/02/small-modules-complexity-over-size/
๐ Nice
@sindresorhus While I wholeheartedly agree with the bulk and whole of your post:
LOC is pretty much irrelevant.
I utterly disagree with this, specially if you are coming from:
It doesn't matter if the module is one line or hundreds. It's all about containing complexity.
While containing complexity and creating abstractions are the best means we have to continue moving forward, long, crappy, and unnecessarily complex code is just bad. Just pointing that out.
LOC is pretty much irrelevant.
I utterly disagree with this, specially if you are coming from:
That statement is context-sensitive, but to clarify: LOC is irrelevant for whether something should be a module or not.
long, crappy, and unnecessarily complex code is just bad.
Did not intend to imply the contrary. The point I was trying to make is that code is worth being a module even though it's just one line, not that hundreds are necessarily good. That really depends.
Unnecessarily complex code is obviously bad, but it's also easy to think something is unnecessarily complex without seeing the full picture. There have been a lot of lightweight jQuery clones, but they all seems to miss why jQuery is large, it actually fixes a lot of browser bugs. Sometimes code is complex because it has to be. It might handle more edge cases which are found over time and makes the code more durable.
code is worth being a module even if it's just one line
๐ Definitely.
Your argument about jQuery is a valid exception, as well as code that needs to support legacy systems, etc.
Sometimes code is complex because it has to be. It might handle more edge cases which are found over time and makes the code more durable.
Totally! ๐
I wrote an article expressing a counter-argument over here, and @sindresorhus invited me to link to it: https://medium.com/@Rich_Harris/small-modules-it-s-not-quite-that-simple-3ca532d65de4.
It's not a direct rebuttal of the points made here, but rather an attempt to articulate some concerns I have about the small modules philosophy. This (excellent) discussion prompted me to publish them.
๐ I'm loving this discussion .. :)
Awesome ๐
@hemanth Great question, @sindresorhus Fantastic answer. ๐ Awesome ๐ฏ
Late unwanted perspective:
I think theres no hard and fast rule that I can think of to answer the question. Generally i think modularity has huge advantages but there is a point where it potentially becomes unhelpful. People have voiced reasons concerning the friction of discovering and vetting packages. I also think that sometimes when we refactor code with the goal of removing complexity we actually just end up shuffling complexity into a different part of your application. I dont think anyone wants to look at a package.json file with 1000 top level dependencies and try to figure out whats going on. Obviously thats an extreme scenario but I think it illustrates the idea that the returns from maximising modularity can diminish to the point where they are exceeded by the accumulated minor pains associated with managing all these dependencies.
Also the thought of what a new npm package name might look like in 5 years time is vaguely terrifying.
The power of Organizational Resources
Very true and appealing, but there are definitely costs to module creation and maintenance you don't mention.
Here is the rundown of the line count for your 'one-line' negative-zero module. (Edit: And I see you don't have a ).travis.yml either so add some lines for doing that, and some time for checking that out.
wc *
4 18 89 index.js
21 172 1119 license
32 57 559 package.json
36 56 624 readme.md
12 20 259 test.js
105 323 2650 total
Some costs can be alleviated with good tools (e.g. yeoman generator for modules); others like crafting tests and Readmes cannot. I'd love a world where people did what you said- but the real and percieved costs mean that not doing so is rational in many cases.
There's indeed a slight overhead doing it this way, but I think the benefits outweigh that.
others like crafting tests and Readmes cannot
You'd hopefully be writing tests and docs for the code anyways.
Plus one can target what files are published into NPM package using "files" or ".gitignore" or ".npmignore" settings, see http://glebbahmutov.com/blog/smaller-published-NPM-modules/ - the strict boundaries among the packages are really worth the slight increase in size, in my opinion
Just as a thought - perhaps this sort of stuff might be more useful at a community level if it was part of a larger library of tools - something like a micro-lodash?
The primary benefit would be one of discoverability/visibility - people would know where to look if they wanted to check if a oneliner existed for their particular use case and over time people would become more and more familiar with what was availabe. Then of course the individual one liners could be available as individual packages much as lodash functions are today.
While "programming via require/import" seems a little weird to me at first thought, there are benefits as mentioned around testability etc. The more I think about it the more a oneliners library seems like it could be pretty cool.