Before we get into developing our application, we need to take a deeper look at using the Promise class and async functions with Express because Express was invented before these features existed, and so it does not directly integrate with them. While we should be using async functions wherever possible, we have to be aware of how to properly use them in certain circumstances, such as in an Express application.
The rules in Express for handling asynchronous execution are as follows:
- Synchronous errors are caught by Express and cause the application to go to the error handler.
- Asynchronous errors must be reported by calling next(err).
- A successfully executing middleware function tells Express to invoke the next middleware by calling next().
- A router function that returns a result to the HTTP request does not call next().
In this section, we’ll discuss three ways to use Promises and async functions in a way that is compatible with these rules.
Both Promises and async functions are used for deferred and asynchronous computation and can make intensely nested callback functions a thing of the past:
- A Promise class represents an operation that hasn’t completed yet but is expected to be completed in the future. We’ve used Promises already, so we know that the .then or .catch functions are invoked asynchronously when the promised result (or error) is available.
- Inside an async function, the await keyword is available to automatically wait for a Promise to resolve. It returns the result of a Promise, or else throws errors, in the natural location at the next line of code, while also accommodating asynchronous execution.
The magic of async functions is that we can write asynchronous code that looks like synchronous code. It’s still asynchronous code—meaning it works correctly with the Node.js event loop—but instead of results and errors landing inside callback functions, errors are thrown naturally as exceptions and results naturally land on the next line of code.
Because this is a new feature in JavaScript, there are several traditional asynchronous coding practices with which we must correctly integrate. You may come across some other libraries for managing asynchronous code, including the following:
- The async library is a collection of functions for various asynchronous patterns. It was originally completely implemented around the callback function paradigm, but the current version can handle async functions and is available as an ES6 package. Refer to https://www.npmjs.com/package/async for more information.
- Before Promises were standardized, at least two implementations were available: Bluebird (http://bluebirdjs.com/) and Q (https://www.npmjs. com/package/q). Nowadays, we focus on using the standard, built-in Promise object, but both of these packages offer additional features.What’s more likely is that we will come across older code that uses these libraries.
These and other tools were developed to make it easier to write asynchronous code and to solve the pyramid of doom problem. This is named after the shape that the code takes after a few layers of nesting. Any multistage process written as callbacks can quickly escalate to code that is nested many levels deep. Consider the following example:
router.get(‘/path/to/something’, (req, res, next) => {
doSomething(req.query.arg1, req.query.arg2, (err, data1) => {
if (err) return next(err);
doAnotherThing(req.query.arg3, req.query.arg2, data1, (err2, data2) => {
if (err2) return next(err2);
somethingCompletelyDifferent(req.query.arg1, req.query.arg42,
(err3, data3) => {
if (err3) return next(err3);
doSomethingElse((err4, data4) => {
if (err4) return next(err4);
res.render(‘page’, { data1, data2, data3, data4 });
});
});
});
});
});
We don’t need to worry about the specific functions, but we should instead recognize that one callback tends to lead to another. Before you know it, you’ve landed in the middle of a deeply nested structure like this. Rewriting this as an async function will make it much clearer. To get there, we need to examine how Promises are used to manage asynchronous results, as well as get a deeper understanding of async functions.
A Promise is either in an unresolved or resolved state. This means that we create a Promise using new Promise, and initially, it is in the unresolved state. The Promise object transitions to the resolved state, where either its resolve or reject functions are called. If the resolve function is called, the Promise is in a successful state, and if instead its reject function is called, the Promise is in a failed state.
More precisely, Promise objects can be in one of three states:
- Pending: This is the initial state, which is neither fulfilled nor rejected.
- Fulfilled: This is the final state, where it executes successfully and produces a result.
- Rejected: This is the final state, where execution fails.
We generate a Promise in the following way:
function asyncFunction(arg1, arg2) {
return new Promise((resolve, reject) => {
// perform some task or computation that’s asynchronous
// for any error detected:
if (errorDetected) return reject(dataAboutError);
// When the task is finished resolve(theResult);
});
};
A function like this creates the Promise object, giving it a callback function, within which is your asynchronous operation. The resolve and reject functions are passed into that function and are called when the Promise is resolved as either a success or failure state. A typical use of new Promise is a structure like this:
function readFile(filename) {
return new Promise((resolve, reject) => {
fs.readFile(filename, (err, data) => {
if (err) reject(err);
else resolve(data);
});
});
}
This is the pattern that we use when promisifying an asynchronous function that uses callbacks. The asynchronous code executes, and in the callback, we invoke either resolve or reject, as appropriate. We can usually use the util.promisify Node.js function to do this for us, but it’s very useful to know how to construct this as needed.
Your caller then uses the function, as follows:
asyncFunction(arg1, arg2)
.then((result) => {
// the operation succeeded
// do something with the result
return newResult;
})
.catch(err => {
// an error occurred
});
The Promise object is fluid enough that the function passed in a .then handler can return something, such as another Promise, and you can chain the .then calls together. The value returned in a .then handler (if any) becomes a new Promise object, and in this way, you can construct a chain of .then and .catch calls to manage a sequence of asynchronous operations.
With the Promise object, a sequence of asynchronous operations is called a Promise chain, consisting of chained .then handlers, as we will see in the next section.
1. Promises and error handling in Express router functions
It is important that all errors are correctly handled and reported to Express. With synchronous code, Express will correctly catch a thrown exception and send it to the error handler. Take the following example:
app.get(‘/’, function (req, res) {
throw new Error(‘BROKEN’);
});
Express catches that exception and does the right thing, meaning it invokes the error handler, but it does not see a thrown exception in asynchronous code. Consider the following error example:
app.get(‘/’, (req, res) => {
fs.readFile(‘/does-not-exist’, (err, data) => {
if (err) throw new Error(err);
// do something with data, like
res.send(data);
});
});
This is an example of the error indicator landing in an inconvenient place in the callback function. The exception is thrown in a completely different stack frame than the one invoked by Express. Even if we arranged to return a Promise, as is the case with an async function, Express doesn’t handle the Promise. In this example, the error is lost; the caller would never receive a response and nobody would know why.
It is important to reliably catch any errors and respond to the caller with results or errors. To understand this better, let’s rewrite the pyramid of doom example:
router.get(‘/path/to/something’, (req, res, next) => {
let data1, data2, data3, data4;
doSomething(req.query.arg1, req.query.arg2)
.then(_data1 => { data1 = _data1;
return doAnotherThing(req.query.arg3, req.query.arg2, data1);
})
.then(_data2 => { data2 = _data2;
return somethingCompletelyDifferent(req.query.arg1, req.query.arg42);
})
.then(_data3 => {
data3 = _data3;
return doSomethingElse();
})
.then(_data4 => {
data4 = _data4;
res.render(‘page’, { data1, data2, data3, data4 });
})
.catch(err => { next(err); });
});
This is rewritten using a Promise chain, rather than nested callbacks. What had been a deeply nested pyramid of callback functions is now arguably a little cleaner thanks to Promises.
The Promise class automatically captures all the errors and searches down the chain of operations attached to the Promise to find and invoke the first .catch function. So long as no errors occur, each .then function in the chain is executed in turn.
One advantage of this is that error reporting and handling is much easier. With the callback paradigm, the nature of the callback pyramid makes error reporting trickier, and it’s easy to miss adding the correct error handling to every possible branch of the pyramid. Another advantage is that the structure is flatter and, therefore, easier to read.
To integrate this style with Express, notice the following:
- The final step in the Promise chain uses res.render or a similar function to return a response to the caller.
- The final catch function reports any errors to Express using next(err).
If instead we simply returned the Promise and it was in the rejected state, Express would not handle that failed rejection and the error would be lost.
Having looked at integrating asynchronous callbacks and Promise chains with Express, let’s look at integrating async functions.
2. Integrating async functions with Express router functions
There are two problems that need to be addressed that are related to asynchronous coding in JavaScript. The first is the pyramid of doom, an unwieldily nested callback structure. The second is the inconvenience of where results and errors are delivered in an asynchronous callback.
To explain, let’s reiterate the example that Ryan Dahl gives as the primary Node.js idiom:
db.query(‘SELECT ..etc..’, function(err, resultSet) {
if (err) {
// Instead, errors arrive here
} else {
// Instead, results arrive here
}
});
// We WANT the errors or results to arrive here
The goal here is to avoid blocking the event loop with a long operation. Deferring the processing of results or errors using callback functions is an excellent solution and is the founding idiom of Node.js. The implementation of callback functions led to this pyramid-shaped problem. Promises help flatten the code so that it is no longer in a pyramid shape. They also capture errors, ensuring delivery to a useful location. In both cases, errors and results are buried inside an anonymous function and are not delivered to the next line of code.
We’ve already used async functions and learned about how they let us write clean- looking asynchronous code. For example, the db.query example as an async function looks as follows:
async function dbQuery(params) {
const resultSet = await db.query(‘SELECT ..etc..’);
// results and errors land here return resultSet;
}
This is much cleaner, with results and errors landing where we want them to.
However, to discuss integration with Express, let’s return to the pyramid of doom example from earlier, rewriting it as an async function:
router.get(‘/path/to/something’, async (req, res, next) => {
try {
const data1 = await doSomething(req.query.arg1, req.query.arg2);
const data2 = await doAnotherThing(req.query.arg3,req.query.arg2, data1);
const data3 = await somethingCompletelyDifferent(req.query.arg1, req.query.arg42);
const data4 = await doSomethingElse();
res.render(‘page’, { data1, data2, data3, data4 });
} catch(err) {
next(err);
}
});
Other than try/catch, this example is very clean compared to its earlier forms, both as a callback pyramid and as a Promise chain. All the boilerplate code is erased, and the intent of the programmer shines through clearly. Nothing is lost inside a callback function. Instead, everything lands on the next line of code where it is convenient.
The await keyword looks for a Promise. Therefore, doSomething and the other functions are expected to return a Promise, and await manages its resolution. Each of these functions could be an async function, and thereby automatically returns a Promise, or it could explicitly create a Promise to manage an asynchronous function call. A generator function is also involved, but we don’t need to know how that works. We just need to know that await manages the asynchronous execution and the resolution of the Promise.
More importantly, each statement with an await keyword executes asynchronously. That’s a side effect of await—managing asynchronous execution to ensure the asynchronous result or error is delivered correctly. However, Express cannot catch an asynchronous error and requires us to notify it of asynchronous results using next().
The try/catch structure is needed for integration with Express. For the reasons just given, we must explicitly catch asynchronously delivered errors and notify Express with next(err).
In this section, we discussed three methods for notifying Express about asynchronously delivered errors. The next thing to discuss is some architectural choices to structure the code.
Source: Herron David (2020), Node.js Web Development: Server-side web development made easy with Node 14 using practical examples, Packt Publishing.