Dealing With Callback Hell
fs.readFile("myFile.txt", function(err, fileContents){
if(!!err) {
throw Error(err);
return;
}
myConvertFile(fileContents, function(err, newContents){
if(!!err){
throw Error(err);
return;
}
fs.writeFile('myNewFile.txt', newFileContents, function(err, saved){
if(!!err){
throw Error(err);
return;
}
console.log("YAY! SAVED FILE!");
}
}
});
Don’t write your code like that.
Just don’t.
Node.js is fairly new, but javascript developers have been dealing with Callback Hell for more than a millennia now. Learning from history and after traversing the far reaches of the inter-webs I have looked for the many ways to deal with the pyramid of doom. Here are my findings:
Past Tense : Named Functions #
Javascript lets you have many anonymous functions. They can be immensely powerful. But, even more importantly, it lets you name functions. It’s an easy and simple solution to Callback Hell. It was fairly popular is the Stone Ages:
fs.readFile("myFile.txt", convertFile);
function convertFile(err, fileContents)){
if(!!err) {
throw Error(err);
return;
}
myConvertFile(fileContents, writeNewFile);
}
function writeNewFile(err, newContents){
if(!!err) {
throw Error(err);
return;
}
fs.writeFile('myNewFile.txt', newFileContents, celebrate);
}
function celebrate(err, saved){
if(!!err){
throw Error(err);
return;
}
console.log("YAY! SAVED FILE!");
}
Simple, yet effective. When working with a small enough project, it is often an elegant solution. You don’t have to keep indenting endlessly, you can re-use functions and you need no new concepts in your head. (Also, as an added benefit, named functions will give you better error stacks)
On the other hand, you have to constantly think of new function names. You still have to check for error in every function and it’s still amazingly painful if you have to wait for two functions to end before you start a third function.
Remember this from my previous blog post?
BinarySearchTree.prototype.breadthFirstLog = function(callback, done){
var that = this;
callback(this.value);
var callDone = (function(obj, done){
var called = 0;
return function(){
called ++;
if(called === 2){
done(obj);
}
};
})(this, done);
setTimeout(function(){
if(!!that.left){
that.left.breadthFirstLog(callback, callDone);
} else {
callDone();
}
if(!!that.right){
that.right.breadthFirstLog(callback, callDone);
} else {
callDone();
}
}, 0)
};
Present Tense: Async.js #
Async.js is a simple and awesome tool. It’s easy to get started, and it has good documentation:
async.waterfall([
function(callback){
fs.readFile("myFile.txt", callback);
},
function(fileContents, callback){
myConvertFile(fileContents, callback);
},
function(newFileContents, callback){
fs.writeFile('myNewFile.txt', newFileContents, celebrate);
},
function(newFileContents, callback){
console.log("YAY! SAVED FILE!");
callback("yay");
},
], function(err){
throw Error(err);
});
It saves us from thinking of new names, and it lets us handle all the errors in one place. Async is small, and powerful tool. It’s also very performant. You can also use Async to solve the problem where a function needs to run after multiple functions are done.
BinarySearchTree.prototype.breadthFirstLog = function(callback, done){
var that = this;
callback(this.value);
setTimeout(function(){
async.parallel([
function(callback){
if(!!that.left){
that.left.breadthFirstLog(callback, callback);
} else {
callback();
}
},
function(callback){
if(!!that.right){
that.right.breadthFirstLog(callback, callback);
} else {
callback();
}
},
], done);
}, 0)
};
If nothing else, just the parallel method alone makes async more than worth it.
Present Continuous Tense: Promises #
Though awesome in most cases, using Async.parallel, Async.series, and Async.waterfall in a single chain can still get pretty messy. Promises are clearly the next step in your journey to salvation. Promises are special objects that behave like events. These objects have a .then method which takes a function and calls it when it can. It is hard to get your head around, but once you understand it, you never want to code without them again.
As an added bonus, Promises are soon going to be native to Javascript.
fs = Promise.promisifyAll(fs);
myConvertFile = Promise.promisify(myConvertFile);
fs.readFileAsync("myFile.txt")
.then(myConvertFile)
.then(function(newFileContents){
return fs.writeFileAsync('myNewFile.txt', newFileContents);
})
.then(console.log.bind(console))
.catch(function(err){
console.err("Error:", err);
});
You can probably see the benefits of using Promises already. Most importantly, Promises are amazing because most Promise libraries (Pick between Bluebird and Q) come with the promisify function that help you convert all your node style functions into promise returning functions. That said, it’s a new concept to wrap your head around, and you should spend some time reading tutorials. I’d start here.
Present Perfect Tense: Reactive Programming #
Once you do get your head around Promises and start loving it, you should probably start learning about Reactive Programming and libraries like Bacon.js and Rx.js
Promises are great for asynchronous functions that call their callback once. But Promise Libraries prevent you from ‘over promising’. Reactive Programing helps you extend this concept over to event listeners. While the code for the problem above looks pretty much the same, let me present this example, straight from the Rx.js that implements a auto-suggest by searching for terms on wikipedia.
var $input = $('#input'),
$results = $('#results');
/* Only get the value from each key up */
var keyups = Rx.Observable.fromEvent($input, 'keyup')
.map(function (e) {
return e.target.value;
})
.filter(function (text) {
return text.length > 2;
});
/* Now throttle/debounce the input for 500ms */
var throttled = keyups
.throttle(500 /* ms */);
/* Now get only distinct values, so we eliminate the arrows and other control characters */
var distinct = throttled
.distinctUntilChanged();
function searchWikipedia (term) {
return $.ajax({
url: 'http://en.wikipedia.org/w/api.php',
dataType: 'jsonp',
data: {
action: 'opensearch',
format: 'json',
search: term
}
}).promise();
}
var suggestions = distinct
.flatMapLatest(searchWikipedia);
suggestions.subscribe( function (data) {
var res = data[1];
/* Do something with the data like binding */
$results.empty();
$.each(res, function (_, value) {
$('<li>' + value + '</li>').appendTo($results);
});
}, function (e) {
/* handle any errors */
$results.empty();
$('<li>Error: ' + error + '</li>').appendTo($results);
});
Future Tense: Generators #
Generators is a new language feature coming in ES6. You can use it today in chrome with some flags turned on and with the --harmony flag in Node 0.11.x
Generators are basically functions that return functions that can pause and resume. The nice part is that your javascript program continues to do other stuff when the function is paused. It’s a whole new awesome concept and once we can all use it you’ll never want to live without it.
This is how we would solve our made up callback problem with generators.
First, we need to define a helper function that we can use all the time:
var sync = function(gen){
var iterator, resume;
resume = function(err, ...args){
// more awesomeness from ES6, splats
if(args.length === 1){
args = args[0];
}
if(!!err){
return iterator.raise(err);
}
iterator.next(args);
};
iterator = gen(resume);
iterator.next();
};
Explaination: iterator is the function returned by our generator gen. iterator can be invoked by calling iterator.next(). The yield keyword acts like return but pauses the function, rather that terminating it. Passing arguments into iterator.next(some argument) replaces the last yield statement with whatever you pass in. And, the resume function can act like a callback that takes error and other arguments and restart the function every time things are ready.
Here’s how we would use the helper function:
sync(function*(resume){
var fileContents = yield fs.readFile("myFile.txt", resume);
fileContents = yield myConvertFile(fileContents, resume);
yield fs.writeFile('myNewFile.txt', fileContents, resume);
console.log("YAY! SAVED FILE!");
});
That’s it. Seriously! If you use a library like CO, with the help of a library called thunkify, you can take this even further and remove resume altogether.
var read = thunkify(fs.readFile);
var write = thunkify(fs.writeFile);
co(function *(){
var fileContents = yield read("myFile.txt");
fileContents = yield myConvertFile(fileContents);
yield write('myNewFile.txt', fileContents);
console.log("YAY! SAVED FILE!");
})()
This feels almost as simple as synchronous code, but it’s not. And it feels and performs better than Promises.
But, you can use generators WITH promises. In fact, co lets you use promise-returning functions in place of thunks, and bluebird.js has an entire section of its documentation talking about how to use it with generators and why its so much more performant.
That’s right!