Express is perhaps the most popular Node.js web app framework. Express is described as being Sinatra-like, which refers to a popular Ruby application framework. It is also regarded as not being an opinionated framework, meaning the framework authors don’t impose their opinions about structuring an application. This means Express is not at all strict about how your code is structured; you just write it the way you think is best.
As of the time of writing this book, Express 4.17 is the current version, and Express 5 is in alpha testing. According to the Express.js website, there are very few differences between Express 4 and Express 5.
Let’s start by installing express-generator. While we can just start with writing some code, express-generator provides a blank starting application, which we’ll take and modify.
Install express-generator using the following commands:
$ mkdir fibonacci
$ cd fibonacci
$ npm install express-generator@4.x
This is different from the suggested installation method on the Express website, which says to use the -g tag for a global installation. We’re also using an explicit version number to ensure compatibility. As of the time of writing, express-generator@5.x does not exist, but it should exist sometime in the future. The instructions here are written for Express 4.x, and by explicitly naming the version, we’re ensuring that we’re all on the same page.
Earlier, we discussed how many people now recommend against installing modules globally. Maybe they would consider express-generator as an exception to that rule, or maybe not. In any case, we’re not following the recommendation on the Express website, and toward the end of this section, we’ll have to uninstall express- generator.
The result of this is that an express command is installed in the ./node_modules/.bin directory:
$ ls node_modules/.bin/
express
Run the express command, as follows:
$ ./node_modules/.bin/express –help
Usage: express [options] [dir]
Options:
–version output the version number
-e, –ejs add ejs engine support
–pug add pug engine support
–hbs add handlebars engine support
-H, –hogan add hogan.js engine support
-v, –view <engine> add view <engine> support
(dust|ejs|hbs|hjs|jade|pug|twig|vash) (defaults to jade)
–no-view use static html instead of view engine
-c, –css <engine> add stylesheet <engine> support
(less|stylus|compass|sass) (defaults to plain css)
–git add .gitignore
-f, –force force on non-empty directory
-h, –help output usage information
We probably don’t want to type ./node_modules/.bin/express every time we run the express-generator application, or, for that matter, any of the other applications that provide command-line utilities. Refer back to the discussion we had in Chapter 3, Exploring Node.js Modules, about adding this directory to the PATH variable. Alternatively, the npx command, also described in Chapter 3, Exploring Node.js Modules, is useful for this.
For example, try using the following instead of installing express-generator:
$ npx express-generator@4.x –help npx: installed 10 in 4.26s
Usage: express [options] [dir]
…
This executes exactly the same, without having to install express-generator and (as we’ll see in a moment) remembering to uninstall it when you’re done using the command.
Now that you’ve installed express-generator in the fibonacci directory, use it to set up the blank framework application:
$ express –view=hbs –git .
destination is not empty, continue? [y/N] y
create : public/
create : public/javascripts/
create : public/images/
create : public/stylesheets/
create : public/stylesheets/style.css
create : routes/
create : routes/index.js
create : routes/users.js
create : views/
create : views/error.hbs
create : views/index.hbs
create : views/layout.hbs
create : .gitignore create : app.js
create : package.json
create : bin/
create : bin/www
install dependencies:
$ npm install
run the app:
$ DEBUG=fibonacci:* npm start
This creates a bunch of files for us, which we’ll walk through in a minute. We asked it to initialize the use of the Handlebars template engine and to initialize a git repository.
The node_modules directory still has the express-generator module, which is no longer useful. We can just leave it there and ignore it, or we can add it to devDependencies of the package.json file that it generated. Most likely, we will want to uninstall it:
$ npm uninstall express-generator
added 62 packages from 78 contributors, removed 9 packages and audited
152 packages in 4.567s
This uninstalls the express-generator tool. The next thing to do is to run the blank application in the way that we’re told. The npm start command relies on a section of the supplied package.json file:
“scripts”: {
“start”: “node ./bin/www”
},
It’s cool that the Express team showed us how to run the server by initializing the scripts section in package.json. The start script is one of the scripts that correspond to the npm sub-commands. The instructions we were given, therefore, say to run npm start.
The steps are as follows:
- Install the dependencies with npm install.
- Start the application by using npm start.
- Optionally, modify json to always run with debugging.
To install the dependencies and run the application, type the following commands:
$ npm install
$ DEBUG=fibonacci:* npm start
> fibonacci@0.0.0 start /Users/David/chap04/fibonacci
> node ./bin/www
fibonacci:server Listening on port 3000 +0ms
Setting the DEBUG variable this way turns on the debugging output, which includes a message about listening on port 3000. Otherwise, we aren’t told this information.
This syntax is what’s used in the Bash shell to run a command with an environment variable. If you get an error when running npm start, then refer to the next section.
We can modify the supplied npm start script to always run the app with debugging enabled. Change the scripts section to the following:
“scripts”: {
“start”: “DEBUG=fibonacci:* node ./bin/www”
},
Since the output says it is listening on port 3000, we direct our browser to http://localhost:3000/ and see the following output:
Cool, we have some running code. Before we start changing the code, we need to discuss how to set environment variables in Windows.
1. Setting environment variables in the Windows cmd.exe command line
If you’re using Windows, the previous example may have failed, displaying an error that says DEBUG is not a known command. The problem is that the Windows shell, the cmd.exe program, does not support the Bash command-line structure.
Adding VARIABLE=value to the beginning of a command line is specific to some shells, such as Bash, on Linux and macOS. It sets that environment variable only for the command line that is being executed and is a very convenient way to temporarily override environment variables for a specific command.
Clearly, a solution is required if you want to be able to use your package.json file across different operating systems.
With this package installed, commands in the scripts section in package.json can set environment variables just as in Bash on Linux/macOS. The use of this package looks as follows:
“scripts”: {
“start”: “cross-env DEBUG=fibonacci:* node ./bin/www”
},
“dependencies”: {
…
“cross-env”: “^6.0.3”
}
Then, the command is executed, as follows:
C:\Users\david\Documents\chap04\fibonacci>npm install
… output from installing packages
C:\Users\david\Documents\chap04\fibonacci>npm run start
> fibonacci@0.0.0 start C:\Users\david\Documents\chap04\fibonacci
> cross-env DEBUG=fibonacci:* node ./bin/www
fibonacci:server Listening on port 3000 +0ms
GET / 304 90.597 ms – –
GET /stylesheets/style.css 304 14.480 ms – –
We now have a simple way to ensure the scripts in package.json are cross- platform. Our next step is a quick walkthrough of the generated application.
2. Walking through the default Express application
We now have a working, blank Express application; let’s look at what was generated for us. We do this to familiarize ourselves with Express before diving in to start coding our Fibonacci application.
Because we used the –view=hbs option, this application is set up to use the Handlebars.js template engine.
Generally speaking, a template engine makes it possible to insert data into generated web pages. The Express.js wiki has a list of template engines for Express (https:// github.com/expressjs/express/wiki#template-engines).
Notice that the JavaScript files are generated as CommonJS modules. The views directory contains two files—error.hbs and index.hbs. The hbs extension is used for Handlebars files. Another file, layout.hbs, is the default page layout.
Handlebars has several ways to configure layout templates and even partials (snippets of code that can be included anywhere).
The routes directory contains the initial routing setup—that is, code to handle specific URLs. We’ll modify this later.
The public directory contains assets that the application doesn’t generate but are simply sent to the browser. What’s initially installed is a CSS file, public/stylesheets/style.css. The package.json file contains our dependencies and other metadata.
The bin directory contains the www script that we saw earlier. This is a Node.js script that initializes the HTTPServer objects, starts listening on a TCP port, and calls the last file that we’ll discuss, app.js. These scripts initialize Express and hook up the routing modules, as well as other things.
There’s a lot going on in the www and app.js scripts, so let’s start with the application initialization. Let’s first take a look at a couple of lines in app.js:
const express = require(‘express’);
…
const app = express();
…
module.exports = app;
…
This means that app.js is a CommonJS module that exports the application object generated by the express module. Our task in app.js is to configure that application object. This task does not include starting the HTTPServer object, however.
Now, let’s turn to the bin/www script. It is in this script where the HTTP server is started. The first thing to notice is that it starts with the following line:
#!/usr/bin/env node
This is a Unix/Linux technique to make a command script. It says to run the following as a script using the node command. In other words, we have Node.js code and we’re instructing the operating system to execute that code using the Node.js runtime:
$ ls -l bin/www
-rwx—— 1 david staff 1595 Feb 5 1970 bin/www
We can also see that the script was made executable by express-generator. It calls the app.js module, as follows:
var app = require(‘../app’);
…
var port = normalizePort(process.env.PORT || ‘3000’);
app.set(‘port’, port);
…
var server = http.createServer(app);
…
server.listen(port);
server.on(‘error’, onError);
server.on(‘listening’, onListening);
Namely, it loads the module in app.js, gives it a port number to use, creates the HTTPServer object, and starts it up.
We can see where port 3000 comes from; it’s a parameter to the normalizePort function. We can also see that setting the PORT environment variable will override the default port 3000. Finally, we can see that the HTTPServer object is created here and is told to use the application instance created in app.js. Try running the following command:
$ PORT=4242 DEBUG=fibonacci:* npm start
By specifying an environment variable for PORT, we can tell the application to listen in on port 4242, where you can ponder the meaning of life.
The app object is next passed to http.createServer(). A look at the Node.js documentation tells us that this function takes requestListener, which is simply a function that takes the request and response objects that we saw previously. Therefore, the app object is the same kind of function.
Finally, the bin/www script starts the server listening process on the port we specified. Let’s now go through app.js in more detail:
app.set(‘views’, path.join( dirname, ‘views’));
app.set(‘view engine’, ‘hbs’);
This tells Express to look for templates in the views directory and to use the Handlebars templating engine.
The app.set function is used to set the application properties. It’ll be useful to browse the API documentation as we go through (http://expressjs.com/en/4x/api.html).
Next is a series of app.use calls:
app.use(logger(‘dev’)); app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join( dirname, ‘public’)));
app.use(‘/’, indexRouter);
app.use(‘/users’, usersRouter);
The app.use function mounts middleware functions. This is an important piece of Express jargon, which we will discuss shortly. At the moment, let’s say that middleware functions are executed during the processing of requests. This means all the features named here are enabled in app.js:
- Logging is enabled using the morgan request logger. Refer to https://www.npmjs.com/package/morgan for its documentation.
- The body-parser module handles parsing HTTP request bodies. Refer to https://www.npmjs.com/package/body-parser for its documentation.
- The cookie-parser module is used to parse HTTP cookies. Refer to https://www.npmjs.com/package/cookie-parser for its documentation.
- A static file web server is configured to serve the asset files in the public directory. Refer to http://expressjs.com/en/starter/static-files.html for its documentation.
- Two router modules—routes and users—to set up which functions handle which URLs.
The static file web server arranges to serve, via HTTP requests, the files in the named directory. With the configuration shown here, the public/stylesheets/style.css file is available at http://HOST/stylesheets/style.css.
We shouldn’t feel limited to setting up an Express application this way. This is the recommendation of the Express team, but there is nothing constraining us from setting it up another way. For example, later in this book, we’ll rewrite this entirely as ES6 modules, rather than sticking to CommonJS modules. One glaring omission is handlers for uncaught exceptions and unhandled Promise rejections. We’ll go over both of these later in this book.
Next, we will discuss Express middleware functions.
3. Understanding Express middleware
Let’s round out our walkthrough of app.js by discussing what Express middleware functions do for our application. Middleware functions are involved in processing requests and sending results to HTTP clients. They have access to the request and response objects and are expected to process their data and perhaps add data to these objects. For example, the cookie parser middleware parses HTTP cookie headers to record in the request object the cookies sent by the browser.
We have an example of this at the end of our script:
app.use(function(req, res, next) {
const err = new Error(‘Not found’);
err.status = 404;
next(err);
});
The comment says catch 404 and forward it to the error handler. As you probably know, an HTTP 404 status means the requested resource was not found.
We need to tell the user that their request wasn’t satisfied, and maybe show them something such as a picture of a flock of birds pulling a whale out of the ocean. This is the first step in doing this. Before getting to the last step of reporting this error, you need to learn how middleware works.
The name middleware implies software that executes in the middle of a chain of processing steps.
Middleware functions take three arguments. The first two—request and response—are equivalent to the request and response objects of the Node.js HTTP request object. Express expands these objects with additional data and capabilities. The last argument, next, is a callback function that controls when the request-response cycle ends, and it can be used to send errors down the middleware pipeline.
As an aside, one critique of Express is that it was written prior to the existence of Promises and async functions. Therefore, its design is fully enmeshed with the callback function pattern. We can still use async functions, but integrating with Express requires using the callback functions it provides.
The overall architecture is set up so that incoming requests are handled by zero or more middleware functions, followed by a router function, which sends the response. The middleware functions call next, and in a normal case, provide no arguments by calling next(). If there is an error, the middleware function indicates the error by calling next(err), as shown here.
For each middleware function that executes, there is, in theory, several other middleware functions that have already been executed, and potentially several other functions still to be run. It is required to call next to pass control to the next middleware function.
What happens if next is not called? There is one case where we must not call next. In all other cases, if next is not called, the HTTP request will hang because no response will be given.
What is the one case where we must not call next? Consider the following hypothetical router function:
app.get(‘/hello’, function(req, res) {
res.send(‘Hello World!’);
});
This does not call next but instead calls res.send. The HTTP response is sent for certain functions on the response object, such as res.send or res.render. This is the correct method for ending the request-response cycle, by sending a response (res.send) to the request. If neither next nor res.send are called, the request never gets a response and the requesting client will hang.
So, a middleware function does one of the following four things:
- Executes its own business logic. The request logger middleware shown earlier is an example of this.
- Modifies the request or response objects. Both body-parser and cookie-parser do this, looking for data to add to the request object.
- Calls next to proceed to the next middleware function or otherwise signals an error.
- Sends a response, ending the cycle.
The ordering of middleware execution depends on the order that they’re added to the app object. The first function added is executed first, and so on.
The next thing to understand is request handlers and how they differ from middleware functions.
4. Contrasting middleware and request handlers
We’ve seen two kinds of middleware functions so far. In one, the first argument is the handler function. In the other, the first argument is a string containing a URL snippet and the second argument is the handler function.
What’s actually going on is app.use has an optional first argument: the path that the middleware is mounted on. The path is a pattern match against the request URL, and the given function is triggered if the URL matches the pattern. There’s even a method to supply named parameters in the URL:
app.use(‘/user/profile/:id’, function(req, res, next) {
userProfiles.lookup(req.params.id, (err, profile) => {
if (err) return next(err);
// do something with the profile
// Such as display it to the user res.send(profile.display());
});
});
This path specification has a pattern, id, and the value will land in req.params.id. In an Express route, this :id pattern marks a route parameter. The pattern will match a URL segment, and the matching URL content will land and be available through the req.params object. In this example, we’re suggesting a user profile service and that for this URL, we want to display information about the named user.
As Express scans the available functions to execute, it will try to match this pattern against the request URL. If they match, then the router function is invoked.
It is also possible to match based on the HTTP request method, such as GET or PUT. Instead of app.use, we would write app.METHOD—for example, app.get or app.put. The preceding example would, therefore, be more likely to appear as follows:
app.get(‘/user/profile/:id’, function(req, res, next) {
// … as above
});
The required behavior of GET is to retrieve data, while the behavior of PUT is to store data. However, as the example was written above, it would match either of the HTTP methods when the handler function is only correct for the GET verb. However, using app.get, as is the case here, ensures that the application correctly matches the desired HTTP method.
Finally, we get to the Router object. This is the kind of middleware used explicitly for routing requests based on their URL. Take a look at routes/users.js:
const express = require(‘express’);
const router = express.Router();
router.get(‘/’, function(req, res, next) {
res.send(‘respond with a resource’);
});
module.exports = router;
We have a module that creates a router object, then adds one or more router functions. It makes the Router object available through module.exports so that app.js can use it. This router has only one route, but router objects can have any number of routes that you think is appropriate.
This one route matches a GET request on the / URL. That’s fine until you notice that in routes/index.js, there is a similar router function that also matches GET requests on the / URL.
Back in app.js, usersRouter is added, as follows:
app.use(‘/users’, usersRouter);
This takes the router object, with its zero-or-more router functions, and mounts it on the /users URL. As Express looks for a matching routing function, it first scans the functions attached to the app object, and for any router object, it scans its functions as well. It then invokes any routing functions that match the request.
Going back to the issue of the / URL, the fact that the router is mounted on the /users URL is important. That’s because the actual URL it considers matching is the mount point (/users) concatenated with the URL in the router function.
The effect is that the mount prefix is stripped from the request URL for the purpose of matching against the router functions attached to the router object. So, with that mount point, an incoming URL of /users/login would be stripped to just /login in order to find a matching router function.
Since not everything goes according to plan, our applications must be capable of handling error indications and showing error messages to users.
5. Error handling
Now, we can finally get back to the generated app.js file, the 404 Error page not found error, and any other errors that the application might show to the user.
A middleware function indicates an error by passing a value to the next function call, namely by calling next(err). Once Express sees the error, it skips over any remaining non-error routings and only passes the error to error handlers instead. An error handler function has a different signature than what we saw earlier.
In app.js, which we’re examining, the following is our error handler, provided by express-generator:
app.use(function(err, req, res, next) {
// set locals, only providing error in development
res.locals.message = err.message;
res.locals.error = req.app.get(‘env’) === ‘development’ ? err : {};
res.status(err.status || 500);
res.render(‘error’);
});
Error handler functions take four parameters, with err added to the familiar req, res, and next functions.
Remember that res is the response object, and we use it to set up the HTTP response sent to the browser; even though there is an error, we still send a response.
Using res.status sets the HTTP response status code. In the simple application that we examined earlier, we used res.writeHead to set not only the status code but also the Multipurpose Internet Mail Extensions (MIME) type of the response.
The res.render function takes data and renders it through a template. In this case, we’re using the template named error. This corresponds to the
views/error.hbs file, which looks as follows:
<h1>{{message}}</h1>
<h2>{{error.status}}</h2>
<pre>{{error.stack}}</pre>
In a Handlebars template, the {{value}} markup means to substitute into the template the value of the expression or variable. The values referenced by this template—message and error—are provided by setting res.locals as shown here.
To see the error handler in action, let’s add the following to routes/index.js:
router.get(‘/error’, function(req, res, next) {
next({
status: 404,
message: “Fake error”
});
});
This is a route handler, and going by what we’ve said, it simply generates an error indication. In a real route handler, the code would make some kind of query, gathering up data to show to the user, and it would indicate an error only if something happened along the way. However, we want to see the error handler in action.
By calling next(err), as mentioned, Express will call the error handler function, causing an error response to pop up in the browser:
Indeed, at the /error URL, we get the Fake error message, which matches the error data sent by the route handler function.
In this section, we’ve created for ourselves a foundation for how Express works. Let’s now turn to an Express application that actually performs a function.
Source: Herron David (2020), Node.js Web Development: Server-side web development made easy with Node 14 using practical examples, Packt Publishing.