Use Promises
geigerzaehler opened this issue · comments
There is a lot of asynchronous code in ShareJS. Almost all methods of the public api use callbacks and many internal methods do, too.
Promises would be a good way to deal with this. Of course one should be a able to use the callback-style api if not familiar with promises. But with helpers like nodeify() we could offer both.
My 2 cents:
I've recently solved various problems using promises, e.g., polyglot data migrations, browser/node client libraries for an HTTP API, kv store interfaces & controller code. I've both written completely new code and rewrapped callback-style code.
Personally I'm very pleased with the amount of additional safety and expressiveness I've gained by using promises. I'm not dismissing callbacks per se, but personally I find it much simpler to reason about code using promises than code using callbacks & utility libraries like async. I haven't had the callback tree of hell problem, I've named all my functions and put them into shiny prototypes, but still there's more to promises which can not easily be expressed or communicated with callbacks.
Also, note that Promises work well with generators and will be part of ES6.
Not having to write if (err) return callback(err); or variants thereof is a good step towards correctness, too.
👍 love promises and how simple they make code.
See older discussion here: https://groups.google.com/d/topic/sharejs/OyaNGfvFJSw/discussion
I'm all for it, everytime I use promises in a project the code is much easier to read and much better to maintain, as the equivalent callback code. So 👍 from me :)
What could be a problem is integration with racer/derby, as they're probably callback based and going to stay that way in the near future?
That's why we should use something like return promise.nodefiy(callback) for the public api. If callback is undefined this returns the promise, otherwise it uses the node callback convention. This should be very little effort.
I agree that regardless of how sharejs works internally we should let people use the standard nodejs callback convention externally. (Though we can also return the promise)
As for using promises internally, there are certainly benefits, but I'm still not entirely convinced. That said, its probably just because I'm becoming stodgy. I don't mind callback hell at all - I guess I'm sort of used to it by now. The one big advantage of promises is when you're doing something thats both optional and async:
function barber(callback) {
var dryHair = function() { ... } // this is step 2, but it appears first (!!)
if (dirtyHair) {
washHair(function(err) {
if (err) { ... }
dryHair(); // violates DRY
);
} else {
dryHair(); // If this is the most common case, you allocate an extra function with no benefit
}
};The rest of the time as far as I can tell they just don't buy you that much. And that horrible case shows up maybe twice in all of sharejs. Otherwise promises seem to be a complicated way to cut down on indentation, which I never really had a problem with in the first place (Nodejs programs just grow down-and-right naturally. Its really not a big deal guys).
I used promises in python's twisted core back in the day when they made you feel like a badass with a headache. I expect they're nicer & easier to use in nodejs, but yeah - I'm not entirely sold on what else they buy us. I'm also not against adding them I suppose. @geigerzaehler if you really think they'll make the code simpler, go for it.
I certainly appreciate all the enthusiasm, and there are very specific
cases where promises are elegant. Many others would argue that it isn't
worth introducing another abstraction when Node.js has already standardized
on a callback format which is definitely not changing in the core libraries.
Regardless of that debate, I think that there is a great deal of higher
priority work to be done in ShareJS right now. It doesn't seem like the
best use of everyone's time to discuss and implement. It especially does
not seem like the best use of Joseph's time to review and debug.
I recommend that this is tabled until more of the features that are core
and innovative about ShareJS are complete. At the very least, I would like
to see sufficient test coverage of 0.7 to release it as default on npm.
On Oct 22, 2013 6:14 PM, "Joseph Gentle" notifications@github.com wrote:
I agree that regardless of how sharejs works internally we should let
people use the standard nodejs callback convention externally. (Though we
can also return the promise)As for using promises internally, there are certainly benefits, but I'm
still not entirely convinced. That said, its probably just because I'm
becoming stodgy. I don't mind callback hell at all - I guess I'm sort of
used to it by now. The one big advantage of promises is when you're doing
something thats both optional and async:function barber(callback) {
var dryHair = function() { ... } // this is step 2, but it appears first (!!)if (dirtyHair) {
washHair(function(err) {
if (err) { ... }
dryHair(); // violates DRY
);
} else {
dryHair(); // If this is the most common case, you allocate an extra function with no benefit
}};The rest of the time as far as I can tell they just don't buy you that
much. And that horrible case shows up maybe twice in all of sharejs.
Otherwise promises seem to be a complicated way to cut down on indentation,
which I never really had a problem with in the first place (Nodejs programs
just grow down-and-right naturally. Its really not a big deal guys).I used promises in python's twisted core back in the day when they made
you feel like a badass with a headache. I expect they're nicer & easier to
use in nodejs, but yeah - I'm not entirely sold on what else they buy us.
I'm also not against adding them I suppose. @geigerzaehlerhttps://github.com/geigerzaehlerif you really think they'll make the code simpler, go for it.—
Reply to this email directly or view it on GitHubhttps://github.com//issues/268#issuecomment-26872892
.
I agree that regardless of how sharejs works internally we should let people use the standard nodejs callback convention externally. (Though we can also return the promise)
If there's a good way I'd expose both. I have yet to see good examples of this though.
Internally, it is not just about avoiding callback hell. It is much more about implicit handling of errors. Both exceptions and unsuccessful async actions are handled through the same interface as long as you are plugging promises together. No need for if (err) return callback(err); or try/catch (or variants thereof). As long as your error happens in the promised land (ouch...), it can be handled at any level. Also stacktraces are usually much more helpful.
Regardless of that debate, I think that there is a great deal of higher priority work to be done in ShareJS right now.
Yes, there would need to be a plan for this which ties into the ongoing work that's being done.
On Oct 23, 2013, at 9:23 , Stephan Seidt notifications@github.com wrote:
I agree that regardless of how sharejs works internally we should let people use the standard nodejs callback convention externally. (Though we can also return the promise)
If there's a good way I'd expose both. I have yet to see good examples of this though.
Internally, it is not just about avoiding callback hell. It is much more about implicit handling of errors. Both exceptions and unsuccessful async actions are handled through the same interface as long as you are plugging promises together. No need for if (err) return callback(err); or try/catch (or variants thereof). As long as your error happens in the promised land (ouch...), it can be handled at any level. Also stacktraces are usually much more helpful.
Another subtle way in which promises are helpful: they wrap state that you can pass around.
When you need to access some data only after it has finished processing, you would normally set some sort of flag in the callback that gets the process-complete notification. Then you need to be mindful of the flag on every access. This mixes async and sync coding styles and is normally hard to read and error-prone.
With promises, you simply store the promise for the processed result and always hook a .then() on it when you need it. The .then() gets fired when the result is ready or in the next pass through the event loop if the result is already known. All the code is always async and simpler to understand.
In 0.6 alone we found I think 3 different race conditions that could have been avoided by storing the promise instead.
There are several places in ShareJS that would benefit from this, and it might even result in refactoring the code.
Regardless of that debate, I think that there is a great deal of higher priority work to be done in ShareJS right now.
Yes, there would need to be a plan for this which ties into the ongoing work that's being done.
I don't agree that promises are low priority. They make a fundamental difference in how the code is structured and therefore the sooner they are introduced, the better.
Wout.
@wmertens, good points. Recently I've implemented a ttl-based cache that way:
var cache = {};
function get(key) {
if (key in cache) return cache[key];
var value = getInputPromise(); // e.g. a slow HTTP request
cache[key] = value;
return value;
}
// Cache usage
get('foobar').then(console.log); // MISS: Sets promise
get('foobar').then(console.log); // HIT: Returns promise
get('foobar').then(console.log); // HIT: Returns promise
// Cache expiry
delete cache.foobar;
get('foobar').then(console.log); // MISS: Sets promiseI've seen this implemented using event emitters, manual queues and other things.
In 0.6 alone we found I think 3 different race conditions that could have been avoided by storing the promise instead.
Race conditions are often a symptom of manual control flow handling that can be avoided when using promises and proven utility libraries. While there are great libraries like async there's a limit to what you can do in terms of callback composition.
I don't agree that promises are low priority. They make a fundamental difference in how the code is structured and therefore the sooner they are introduced, the better.
Agreed, but still there needs to be a plan. Even if someone starts doing the work in a branch now, that branch needs to be tested and merged.
(Edit: Fixed get function)
On Oct 23, 2013, at 10:28 , Stephan Seidt notifications@github.com wrote:
Agreed, but still there needs to be a plan. Even if someone starts doing the work in a branch now, that branch needs to be tested and merged.
How about: We pick a module, add Q to it, and start converting callbacks to promises, one at a time (or as many as is feasible in an hour of coding). Input and output callbacks are kept as callbacks.
Once the module was completely converted, we reconvene and decide if it was a good idea or not, and if it's a yay we convert all the modules in the same manner. If it's a nay we switch back to callback style.
This way the promise conversion can happen in the regular tree and there won't be a gigantic merge at the end. At most there will be some inconvenience if we have to convert back.
Wout.
EDIT: Read my followup message before replying.
@ehd your code could just as easily be written without promises:
var cache = {};
function get(key, callback) {
if (key in cache) return callback(null, cache[key]);
getInput(function(err, value) {
if (value) cache[key] = value;
callback(err, value);
});
}
// Cache usage
get('foobar', console.log); // MISS: Sets promise
get('foobar', console.log); // HIT: Returns promise
get('foobar', console.log); // HIT: Returns promise
// Cache expiry
delete cache.foobar;
get('foobar', console.log); // MISS: Sets promiseI don't see how your version is any better. Its significantly harder to understand than my version for the 90% of javascript developers who don't know / use promises.
I'm still not sold on their benefit, and I don't like adding dependancies when they don't have a clear gain. Even in the chained-callbacks use case, how do you move data from one of the next() blocks to the next? Do you have to manually hoist variables? ..........
If you guys are up to the challenge, this is probably the most complicated flow control spaghetti code in the sharejs stack:
https://github.com/share/livedb/blob/master/lib/index.js#L436-L562
Show me what that would look like with promises.
Wait, scratch that. Your version does something completely different than my version - you're actually making a cache of promise objects and then reusing them using next(). Thats not obvious, but actually really clever. It would nicely simplify a bunch of logic in the subscription handing code in session.js. I wonder if some of the client doc.js code could be simplified using that too.
Iiiiinteresting.
@josephg 👉👃 exactly! 😄
