The goal we’re working toward is for the Notes home page to automatically update the list of notes as notes are edited or deleted. What we’ve done so far is to restructure the application startup so that Socket.IO is initialized in the Notes application. There’s no change of behavior yet.
What we will do is send an event whenever a note is created, updated, or deleted. Any interested part of the Notes application can listen to those events and act appropriately. For example, the Notes home page router module can listen for events, and then send an update to the browser. The code in the web browser will listen for an event from the server, and in response, it would rewrite the home page. Likewise, when a Note is modified, a listener can send a message to the web browser with the new note content, or if the Note is deleted, a listener can send a message so that the web browser redirects to the home page.
These changes are required:
- Refactoring the Notes Store implementations to send create, update, and delete events
- Refactoring the templates to support both Bootstrap on every page and a custom Socket.IO client for each page
- Refactoring the home page and Notes’ viewing router modules to listen for Socket.IO events and send updates to the browser
We’ll handle this over the next few sections, so let’s get started.
1. Refactoring the NotesStore classes to emit events
In order to automatically update the user interface when a Note is changed or deleted or created, the NotesStore must send events to notify interested parties of those changes. We will employ our old friend, the EventEmitter class, to manage the listeners to the events we must send.
Recall that we created a class, AbstractNotesStore, and that every storage module contains a subclass of AbstractNotesStore. Hence we can add listener support in AbstractNotesStore, making it automatically available to the implementations.
In models/Notes.mjs, make this change:
import EventEmitter from ‘events’;
export class AbstractNotesStore extends EventEmitter {
static store() { }
async close() { }
async update(key, title, body) { }
async create(key, title, body) { }
async read(key) { }
async destroy(key) { }
async keylist() { }
async count() { }
emitCreated(note) { this.emit(‘notecreated’, note); }
emitUpdated(note) { this.emit(‘noteupdated’, note); }
emitDestroyed(key) { this.emit(‘notedestroyed’, key); }
}
We imported the EventEmitter class, made AbstractNotesStore a subclass of EventEmitter, and then added some methods to emit events. As a result, every NotesStore implementation now has an on and emit method, plus these three helper methods.
This is only the first step since nothing is emitting any events. We have to rewrite the create, update, and destroy methods in NotesStore implementations to call these methods so the events are emitted.
Modify these functions in models/notes-sequelize.mjs as shown in the following code:
async update(key, title, body) {
…
const note = await this.read(key);
this.emitUpdated(note);
return note;
…
}
async create(key, title, body) {
…
const note = new Note(sqnote.notekey, sqnote.title, sqnote.body);
this.emitCreated(note);
return note;
}
async destroy(key) {
…
this.emitDestroyed(key);
}
The changes do not change the original contract of these methods, since they still create, update, and destroy notes. The other NotesStore implementations require similar changes. What’s new is that now those methods emit the appropriate events for any code that may be interested.
Another task to take care of is initialization, which must happen after NotesStore is initialized. Recall that setting up NotesStore is asynchronous. Therefore, calling the .on function to register an event listener must happen after NotesStore is initialized.
In both routes/index.mjs and routes/notes.mjs, add the following function:
export function init() {
}
This function is meant to be in place of such initialization. Then, in app.mjs, make this change:
import {
router as indexRouter, init as homeInit
} from ‘./routes/index.mjs’; import {
router as notesRouter, init as notesInit
} from ‘./routes/notes.mjs’;
…
import { useModel as useNotesModel } from ‘./models/notes-store.mjs’;
useNotesModel(process.env.NOTES_MODEL)
.then(store => {
homeInit();
notesInit();
})
.catch(error => { onError({ code: ‘ENOTESSTORE’, error }); });
This imports the two init functions, giving them unique names, then calling them once NotesStore is set up. At the moment, both functions do nothing, but that will change shortly. The important thing is these two init functions will be called after NotesStore is completely initialized.
We have our NotesStore sending events when a Note is created, updated, or destroyed. Let’s now use those events to update the user interface appropriately.
2. Real-time changes in the Notes home page
The Notes model now sends events as Notes are created, updated, or destroyed. For this to be useful, the events must be displayed to our users. Making the events visible to our users means the controller and view portions of the application must consume those events.
At the top of routes/index.mjs, add this to the list of imports:
import { io } from ‘../app.mjs’;
Remember that this is the initialized Socket.IO object we use to send messages to and from connected browsers. We will use it to send messages to the Notes home page.
Then refactor the router function:
router.get(‘/’, async (req, res, next) => {
try {
const notelist = await getKeyTitlesList();
res.render(‘index’, {
title: ‘Notes’, notelist: notelist, user: req.user ? req.user : undefined
});
} catch (e) { next(e); }
});
async function getKeyTitlesList() {
const keylist = await notes.keylist();
const keyPromises = keylist.map(key => notes.read(key));
const notelist = await Promise.all(keyPromises);
return notelist.map(note => {
return { key: note.key, title: note.title };
});
};
This extracts what had been the body of the router function into a separate function. We need to use this function not only in the home page router function but also when we emit Socket.IO messages for the home page.
We did change the return value. Originally, it contained an array of Note objects, and now it contains an array of anonymous objects containing key and title data. We did this because providing the array of Note objects to Socket.IO resulted in an array of empty objects being sent to the browser while sending the anonymous objects worked correctly.
Then, add this at the bottom:
const emitNoteTitles = async () => {
const notelist = await getKeyTitlesList();
io.of(‘/home’).emit(‘notetitles’, { notelist });
};
export function init() {
io.of(‘/home’).on(‘connect’, socket => {
debug(‘socketio connection on /home’);
});
notes.on(‘notecreated’, emitNoteTitles);
notes.on(‘noteupdate’, emitNoteTitles);
notes.on(‘notedestroy’, emitNoteTitles);
}
The primary purpose of this section is to listen to the create/update/destroy events, so we can update the browser. For each, the current list of Notes is gathered, then sent to the browser.
As we said, the Socket.IO package uses a model similar to the EventEmitter class. The emit method sends an event, and the policy of event names and event data is the same as with EventEmitter.
Calling io.of(‘/namespace’) creates a Namespace object for the named namespace. Namespaces are named in a pattern that looks like a pathname in Unix- like filesystems.
Calling io.of(‘/namespace’).on(‘connect’…) has the effect of letting server- side code know when a browser connects to the named namespace. In this case, we are using the /home namespace for the Notes home page. This has the side-effect of keeping the namespace active after it is created. Remember that init is called during the initialization of the server. Therefore, we will have created the /home namespace long before any web browser tries to access that namespace by visiting the Notes application home page.
Calling io.emit(…) sends a broadcast message. Broadcast messages are sent to every browser connected to the application server. That can be useful in some situations, but in most situations, we want to avoid sending too many messages. To limit network data consumption, it’s best to target each event to the browsers that need the event.
Calling io.of(‘/namespace’).emit(…) targets the event to browsers connected to the named namespace. When the client-side code connects to the server, it connects with one or more namespaces. Hence, in this case, we target the notetitles event to browsers attached to the /home namespace, which we’ll see later is the Notes home page.
Calling io.of(‘/namespace’).to(‘room’) accesses what Socket.IO calls a room. Before a browser receives events in a room, it must join the room. Rooms and namespaces are similar, but different, things. We’ll use rooms later.
The next task accomplished in the init function is to create the event listeners for the notecreated, noteupdate, and notedestroy events. The handler function for each emits a Socket.IO event, notetitles, containing the list of note keys and titles.
As Notes are created, updated, and destroyed, we are now sending an event to the home page that is intended to refresh the page to match the change. The home page template, views/index.hbs, must be refactored to receive that event and rewrite the page to match.
2.1. Changing the home page and layout templates
Socket.IO runs on both the client and the server, with the two communicating back and forth over the HTTP connection. So far, we’ve seen the server side of using Socket.IO to send events. The next step is to install a Socket.IO client on the Notes home page.
Generally speaking, every application page is likely to need a different Socket.IO client, since each page has different requirements. This means we must change how JavaScript code is loaded in Notes pages.
Initially, we simply put JavaScript code required by Bootstrap and FeatherJS at the bottom of layout.hbs. That worked because every page required the same set of JavaScript modules, but now we’ve identified the need for different JavaScript code on each page. Because the custom Socket.IO clients for each page use jQuery for DOM manipulation, they must be loaded after jQuery is loaded. Therefore, we need to change layout.hbs to not load the JavaScript. Instead, every template will now be required to load the JavaScript code it needs. We’ll supply a shared code snippet for loading the Bootstrap, Popper, jQuery, and FeatherJS libraries but beyond that, each template is responsible for loading any additional required JavaScript.
Create a file, partials/footerjs.hbs, containing the following code:
<!– jQuery first, then Popper.js, then Bootstrap JS –>
<script src=”/assets/vendor/jquery/jquery.min.js”></script>
<script src=”/assets/vendor/popper.js/popper.min.js”></script>
<script src=”/assets/vendor/bootstrap/js/bootstrap.min.js”></script>
<script src=”/assets/vendor/feather-icons/feather.js”></script>
<script> feather.replace();
</script>
This code had been at the bottom of views/layout.hbs, and it is the shared code snippet we just mentioned. This is meant to be used on every page template, and to be followed by custom JavaScript.
We now need to modify views/layout.hbs as follows:
<html>
<head>…</head>
<body>
{{> header }}
{{{body}}}
</body>
</html>
That is, we’ll leave layout.hbs pretty much as it was, except for removing the JavaScript tags from the bottom. Those tags are now in footerjs.hbs.
We’ll now need to
modify every template (error.hbs, index.hbs, login.hbs, notedestroy.hbs, no teedit.hbs, and noteview.hbs) to, at the minimum, load the footerjs partial.
{{> footerjs}}
With this, every one of the templates explicitly loads the JavaScript code for Bootstrap and FeatherJS at the bottom of the page. They were previously loaded at the bottom of the page in layout.hbs. What this bought us is the freedom to load Socket.IO client code after Bootstrap and jQuery are loaded.
We have changed every template to use a new policy for loading the JavaScript. Let’s now take care of the Socket.IO client on the home page.
2.2. Adding a Socket.IO client to the Notes home page
Remember that our task is to add a Socket.IO client to the home page so that the home page receives notifications about created, updated, or deleted Notes.
In views/index.hbs, add this at the bottom, after the footerjs partial:
{{> footerjs}}
<script src=”/socket.io/socket.io.js”></script>
<script>
$(document).ready(function () {
var socket = io(‘/home’);
socket.on(‘connect’, socket => {
console.log(‘socketio connection on /home’);
});
socket.on(‘notetitles’, function(data) {
var notelist = data.notelist;
$(‘#notetitles’).empty();
for (var i = 0; i < notelist.length; i++) {
notedata = notelist[i];
$(‘#notetitles’)
.append(‘<a class=”btn btn-lg btn-block btn-outline-dark” href=”/notes/view?key=’+ notedata.key +'”>’+
notedata.title +'</a>’);
}
});
});
</script>
This is what we meant when we said that each page will have its own Socket.IO client implementation. This is the client for the home page, but the client for the Notes view page will be different. This Socket.IO client connects to the /home namespace, then for notetitles events, it redraws the list of Notes on the home page.
The first <script> tag is where we load the Socket.IO client library, from /socket.io/socket.io.js. You’ll notice that we never set up any Express route to handle the /socket.io URL. Instead, the Socket.IO library did that for us. Remember that the Socket.IO library handles every request starting with /socket.io, and this is one of such request it handles. The second <script> tag is where the page-specific client code lives.
Having client code within a $(document).ready(function() { .. }) block is typical when using jQuery. This, as the code implies, waits until the web page is fully loaded, and then calls the supplied function. That way, our client code is not only held within a private namespace; it executes only when the page is fully set up.
On the client side, calling io() or io(‘/namespace’) creates a socket object. This object is what’s used to send messages to the server or to receive messages from the server.
In this case, the client connects a socket object to the /home namespace, which is the only namespace defined so far. We then listen for the notetitles events, which is what’s being sent from the server. Upon receiving that event, some jQuery DOM manipulation erases the current list of Notes and renders a new list on the screen. The same markup is used in both places.
Additionally, for this script to function, this change is required elsewhere in the template:
<div class=”col-12 btn-group-vertical” id=”notetitles” role=”group”>
…
</div>
You’ll notice in the script that it references $(“#notetitles”) to clear the existing list of note titles, then to add a new list. Obviously, that requires an id=”notetitles” attribute on this <div>.
Our code in routes/index.mjs listened to various events from the Notes model and, in response, sent a notetitles event to the browser. The browser code takes that list of note information and redraws the screen.
2.3. Running Notes with real-time home page updates
We now have enough implemented to run the application and see some real-time action.
As you did earlier, start the user information microservice in one window:
$ npm start
> user-auth-server@0.0.1 start /Users/david/chap09/users
> DEBUG=users:* PORT=5858 SEQUELIZE_CONNECT=sequelize-sqlite.yaml node
./user-server.mjs
users:service User-Auth-Service listening at http://127.0.0.1:5858
+0ms
Then, in another window, start the Notes application:
$ npm start
> notes@0.0.0 start /Users/david/chap09/notes
> DEBUG=notes:* SEQUELIZE_CONNECT=models/sequelize-sqlite.yaml NOTES_MODEL=sequelize USER_SERVICE_URL=http://localhost:5858 node — experimental-modules ./app
(node:11998) ExperimentalWarning: The ESM module loader is experimental.
notes:debug-INDEX Listening on port 3000 +0ms
Then, in a browser window, go to http://localhost:3000 and log in to the Notes application. To see the real-time effects, open multiple browser windows. If you can use Notes from multiple computers, then do that as well.
In one browser window, start creating and deleting notes, while leaving the other browser windows viewing the home page. Create a note, and it should show up immediately on the home page in the other browser windows. Delete a note and it should disappear immediately as well.
One scenario you might try requires three browser windows. In one window, create a new note, and then leave that browser window showing the newly created note. In another window, show the Notes home page. And in the third window, show the newly created note. Now, delete this newly created note. Of those windows, two are correctly updated and are now showing the home page. The third, where we were simply viewing the note, is still showing that note even though it no longer exists.
We’ll get to that shortly, but first, we need to talk about how to debug your Socket.IO client code.
3. A word on enabling debug tracing in Socket.IO code
It is useful to inspect what Socket.IO is doing in case you’re having trouble. Fortunately, the Socket.IO package uses the same Debug package that Express uses, and we can turn on debug tracing just by setting the DEBUG environment variable. It even uses a variable, localStorage.debug, with the same syntax on the client side, and we can enable debug tracing in the browser as well.
On the server side, this is a useful DEBUG environment variable setting:
DEBUG=notes:*,socket.io:*
This enables debug tracing for the Notes application and the Socket.IO package.
Enabling this in a browser is a little different since there are no environment variables. Simply open up the JavaScript console in your browser and enter this command:
localStorage.debug = ‘socket.io-client:*,socket.io-parser’;
Immediately, you will start seeing a constant chatter of messages from Socket.IO. One thing you’ll learn is that even when the application is idle, Socket.IO is communicating back and forth.
There are several other DEBUG strings to use. For example, Socket.IO relies on the Engine.IO package for its transport layer. If you want debug tracing of that package, add engine* to the DEBUG string. The strings shown were most helpful during the testing of this chapter.
Now that we’ve learned about debug tracing, we can take care of changing the /notes/view pages to react so they changes to the Note being viewed.
4. Real-time action while viewing notes
It’s cool how we can now see real-time changes in a part of the Notes application. Let’s turn to the /notes/view page to see what we can do. What comes to mind is this functionality:
- Update the note if someone else edits it.
- Redirect the viewer to the home page if someone else deletes the note.
- Allow users to leave comments on the note.
For the first two features, we can rely on the existing events coming from the Notes model. Therefore, we can implement those two features in this section. The third feature will require a messaging subsystem, so we’ll get to that later in this chapter.
To implement this, we could create one Socket.IO namespace for each Note, such as /notes/${notekey}. Then, when the browser is viewing a Note, the client code added to the noteview.hbs template would connect to that namespace. However, that raises the question of how to create those namespaces. Instead, the implementation selected was to have one namespace, /notes, and to create one room per Note.
In routes/notes.mjs, make sure to import the io object as shown here:
import { emitNoteTitles } from ‘./index.mjs’;
import { io } from ‘../app.mjs’;
This, of course, makes the io object available to code in this module. We’re also importing a function from index.mjs that is not currently exported. We will need to cause the home page to be updated, and therefore in index.mjs, make this change:
export const emitNoteTitles = async () => { … };
This simply adds the export keyword so we can access the function from elsewhere. Then, change the init function to this:
export function init() { io.of(‘/notes’).on(‘connect’, socket => {
if (socket.handshake.query.key) {
socket.join(socket.handshake.query.key);
}
});
notes.on(‘noteupdated’, note => {
const toemit = {
key: note.key, title: note.title, body: note.body
});
};
io.of(‘/notes’).to(note.key).emit(‘noteupdated’, toemit);
emitNoteTitles();
notes.on(‘notedestroyed’, key => {
io.of(‘/notes’).to(key).emit(‘notedestroyed’, key);
emitNoteTitles();
});
}
First, we handle connect events on the /notes namespace. In the handler, we’re looking for a query object containing the key for a Note. Therefore, in the client code, when calling io(‘/notes’) to connect with the server, we’ll have to arrange to send that key value. It’s easy to do, and we’ll learn how in a little while.
Calling socket.join(roomName) does what is suggested—it causes this connection to join the named room. Therefore, this connection will be addressed as being in the /notes namespace, and in a room whose name is the key for a given Note.
The next thing is to add listeners for the noteupdated and notedestroyed messages. In both, we are using this pattern:
io.of(‘/namespace’).to(roomName).emit(..);
This is how we use Socket.IO to send a message to any browser connected to the given namespace and room.
For noteupdated, we simply send the new Note data. We again had to convert the Note object into an anonymous JavaScript object, because otherwise, an empty object arrived in the browser. The client code will have to use, as we will see shortly, jQuery operations to update the page.
For notedestroyed, we simply send the key. Since the client code will respond by redirecting the browser to the home page, we don’t have to send anything at all.
In both, we also call emitNoteTitles to ensure the home page is updated if it is being viewed.
4.1. Changing the note view template for real-time action
As we did in the home page template, the data contained in these events must be made visible to the user. We must not only add client code to the template, views/noteview.hbs; we need a couple of small changes to the template:
<div class=”container-fluid”>
<div class=”row”><div class=”col-xs-12″>
{{#if note}}<h3 id=”notetitle”>{{ note.title }}</h3>{{/if}}
{{#if note}}<div id=”notebody”>{{ note.body }}</div>{{/if}}
<p>Key: {{ notekey }}</p>
</div></div>
{{#if user }}{{#if notekey }}
<div class=”row”><div class=”col-xs-12″>
<div class=”btn-group”>
<a class=”btn btn-outline-dark” href=”/notes/destroy?key={{notekey}}” role=”button”>Delete</a>
<a class=”btn btn-outline-dark” href=”/notes/edit?key={{notekey}}” role=”button”>Edit</a>
</div></div></div>
{{/if}}{{/if}}
</div>
In this section of the template, we add a pair of IDs to two elements. This enables the JavaScript code to target the correct elements.
Add this client code to noteview.hbs:
{{> footerjs}}
{{#if notekey }}
<script src=”/socket.io/socket.io.js”></script>
<script>
$(document).ready(function () {
let socket = io(‘/notes’, {
query: { key: ‘{{ notekey }}’ }
});
socket.on(‘noteupdated’, note => {
$(‘h3#notetitle’).empty();
$(‘h3#notetitle’).text(note.title);
$(‘#navbartitle’).empty();
$(‘#navbartitle’).text(note.title);
$(‘#notebody’).empty();
$(‘#notebody’).text(note.body);
});
socket.on(‘notedestroyed’, key => {
window.location.href = “/”;
});
});
</script>
{{/if}}
In this script, we first connect to the /notes namespace and then create listeners for the noteupdated and notedestroyed events.
When connecting to the /notes namespace, we are passing an extra parameter. The optional second parameter to this function is an options object, and in this case, we are passing the query option. The query object is identical in form to the query object of the URL class. This means the namespace is as if it were a URL such as /notes?key=${notekey}. Indeed, according to the Socket.IO documentation, we can pass a full URL, and it also works if the connection is created like this:
let socket = io(‘/notes?key={{ notekey }}’);
While we could set up the URL query string this way, it’s cleaner to do it the other way.
We need to call out a technique being used. These code snippets are written in a Handlebars template, and therefore the syntax {{ expression }} is executed on the server, with the result of that expression to be substituted into the template.
Therefore, the {{ expression }} construct accesses server-side data. Specifically, query: { key: ‘{{ notekey }}’ } is a data structure on the client side, but the {{ notekey }} portion is evaluated on the server. The client side does not see {{ notekey }}, it sees the value notekey had on the server.
For the noteupdated event, we take the new note content and display it on the screen. For this to work, we had to add id= attributes to certain HTML elements so we could use jQuery selectors to manipulate the correct elements.
Additionally in partials/header.hbs, we needed to make this change as well:
<span id=”navbartitle” class=”navbar-text text-dark col”>{{ title
}}</span id=”navbartitle”>
We needed to update the title at the top of the page as well, and this id attribute helps to target the correct element.
For the notedestroyed event, we simply redirect the browser window back to the home page. The note being viewed has been deleted, and there’s no point the user continuing to look at a note that no longer exists.
4.2. Running Notes with pseudo-real-time updates while viewing a note
At this point, you can now rerun the Notes application and try the new real-time updates feature.
By now you have put Notes through its paces many times, and know what to do. Start by launching the user authentication server and the Notes application. Make sure there is at least one note in the database; add one if needed. Then, open multiple browser windows with one viewing the home page and two viewing the same note. In a window viewing the note, edit the note to make a change, making sure to change the title. The text change should change on both the home page and the page viewing the note.
Then delete the note and watch it disappear from the home page, and further, the browser window that had viewed the note is now on the home page.
We took care of a lot of things in this section, and the Notes application now has dynamic updates happening. To do this, we created an event-based notification system, then used Socket.IO in both browser and server to communicate data back and forth.
We have implemented most of what we’ve set out to do. By refactoring the Notes Store implementations to send events, we are able to send events to Socket.IO clients in the browser. That in turn is used to automatically update the Notes home page, and the /notes/view page.
The remaining feature is for users to be able to write comments on Notes. In the next section, we will take care of that by adding a whole new database table to handle messages.
Source: Herron David (2020), Node.js Web Development: Server-side web development made easy with Node 14 using practical examples, Packt Publishing.