Daniel GolantI am currently building a side project (GCP, Express, Vue and a Chrome Extension) that I actually want to put in front of other people, rather than just use for my own utility, which is rare for me! That means that I actually need to build in robust error handling and validation, because I no longer have a single, incredibly generous user. A key part of that quality effort is validating the presence and conformance of data in incoming HTTP requests to the definition of the route that is handling those requests.
Pitfalls of Vanilla Validation
This sort of validation handling can be an easy pitfall for code cleanliness in Javascript in particular, where the absence or mis-typing of certain values can't easily be handled through the use of a class constructor. In another language, a value being null could be handled by simply passing that value into the constructor of whatever class would eventually be used by the given route, and if the constructor failed to return an object you could reject the request with a 400 error code. In Javascript, you don't really have the same tools, since the dynamically-typed nature of the language works against you in this instance. The naive approach then, and one I have unfortunately followed at least once in the past, is to manually check that every field in the body that your route is expecting is present. Something like:
app.post('/user/links', function (req, res) {
if (!req.body.important_value || !req.body.data1 || !req.body.data2) {
logger.debug('USER UPDATE POST FAILED: MISSING VALUES', {
request_body: req.body
});
res.status(400).send('Body Properties Missing: ' + req.body);
}
/* actual thing you want to do with this route */
});
What are the issues with this approach? Well first of all, it definitely draws attention away from the actual function of the route. The reader is six lines down (at minimum) before they even see something related to the route operation. When you account for the potential duplication of this sort of logic across many routes, even simple routes can end up comically large, especially if you consider that we are only looking for three values in this case. On top of that, the client doesn't get much information about what expected value is actually missing from the payload. If we wanted to provide more detail, another naive approach might be to split out this logic into multiple conditionals:
app.post('/linksforuser', function (req, res) {
if (!req.body.important_value){
logger.debug('USER UPDATE POST FAILED: MISSING IMPORTANT VALUE', {
request_body: req.body
})
res.status(400).send('Body Important Value Missing: ' + req.body);
}
if(!req.body.data1) {
logger.debug('USER UPDATE POST FAILED: MISSING DATA1 VALUE', {
request_body: req.body
})
res.status(400).send('Body Properties Missing: ' + req.body);
}
if(!req.body.data2){
logger.debug('USER UPDATE POST FAILED: MISSING DATA2 VALUE', {
request_body: req.body
})
res.status(400).send('Body Properties Missing: ' + req.body);
}
});
Perfect, right? Well, yes, you now have more accurate logging and response messaging, but, you have added 18 lines of validation compared to your previous six. On top of that, maybe I am the only person who has ever done this, but copying and pasting log messages usually hurts me at some point. Invariably I copy and paste a message without updating it after, and eventually I try to debug a completely different line or file when an issue comes up. Plus, this payload is still fairly small, and as it grows, so too will your validation. To handle that, you might try to wrap your whole payload in an object, but then you run into the issue of comparing object keys, and we still have not even addressed the actual values of the properties.
So what is a dev to do? We can either add lines and lines of brittle validation logic, or we can write a catch-all function that we have to re-tool every time our payloads change, right? Well, luckily, that is not necessarily the case.
Validation Modules
You see, Express provides us with pre-made middleware modules, which -like any middleware you write yourself- can easily manipulate the request and response objects of a route. If you wanted to, you could attach all of your validation logic as custom middleware functions in order to at least get all of that stuff out of sight. But why would you want to? There are plenty of pre-made, robust, well-tested Express request validation modules.
Having used a few of these modules, something did not sit quite right with me about them. If they were supposed to clean up my code, then I never felt like they went quite far enough. I would still end up with messaging I wasn't quite happy with, or a validation format that didn't click for me. That's all taste, of course.
Joi and Celebration
That's when I found Joi, the built-in schema validation library for Hapi (another Node framework like Express). Something just clicked for me, Joi was exactly what I had been looking for. Once I found Joi, it only took me a short while to discover Celebrate, an Express middleware for Joi validation. You can read a bit about the author's intent and reasoning behind Celebrate here, which I found compelling and well thought-out. I generally err towards tools written by people who take the time to detail, justify, and share their decision-making, because it makes me more confident that they've thought the subject through.
Celebrate allows the user to leverage Joi's simple and powerful object definition and validation capabilities with just one line of code, returns informative error messaging to the client, short-circuits requests that fail validation, and allows the object validator to update the request object itself when the massaging of incoming requests is needed. It is also super simple to get start with.
Using Joi with Celebrate
First, as always:
npm install celebrate
Then, where-ever you need to use Celebrate and Joi, just add the following lines
const express = require('express');
const BodyParser = require('body-parser');
const Celebrate = require('celebrate');
const { Joi } = Celebrate;
const app = express(); // use whatever name you like, I tend to default to app
app.use(BodyParser.json());
To validate a route in that file, you define your route with something similar to the following:
app.post('/user/links', Celebrate({
body: Joi.object().keys({
important_value: Joi.string().required(), // look, type enforcement!
data1: Joi.number().integer(),
data2: Joi.string().default('admin') // hey, and defaults!
}),
query: {
token: Joi.string().token().required() // you can use one object to
// validate body, query,
// and params all at once
}
}), (req, res) => {
/* Here we do whatever the route is
actually supposed to do,
because Celebrate will automatically res.status(400).send()
an informative message
if the validation fails
*/
});
app.use(Celebrate.errors());
// taken and modified from the Celebrate docs
Wow, look at that! We now have type enforcement and default values, way more than the simple validation we had before, and in just 8 lines of code!
"But wait Dan, didn't you say one line of code?"
My Implementation
Well, it's one line of Celebrate code, obviously any object definition's size will be dependent on the size of the object being validated, not to mention that LOC is an arbitrary metric anyway, but we can actually condense this a bit more. The way I handle my validation looks something like this:
const SCHEMA_POST_LINKS = require('./request_schemas/link_collection_routes/links_POST_schema.js');
app.post('/user/links', Celebrate({
body: SCHEMA_POST_LINKS
}), (req, res) => {
logger.info('POST received... \tCreateUser'); // hey look,
// a logging mistake I just
// discovered because
// I forgot to change what I c/p'd
// (I am not kidding);
// Here we add some links to a user object,
// knowing that our validation already happened
});
We've now split out our request schema into a different file, leaving us with a single line of validation (as promised π). The request schema looks like this:
const { Joi } = require('celebrate');
const links_POST_schema = Joi.object().keys({
access_token: Joi.string(),
id_token: Joi.string(),
url: Joi.string().required(),
title: Joi.string().required()
}).xor('access_token', 'id_token');
module.exports = links_POST_schema;
See that xor condition? It's a nifty little convenience that would've been really annoying to add manually. It allows me to get around the limitations of an API that I am using without duplicating my routes, by enforcing that only either an access_token or an id_token can be present in the payload, not both. On top of that, because Celebrate includes its own formal Joi dependency, we can validate objects other than HTTP requests (like responses), using a consistent version of Joi. Here is my validation for a response the server sends in the same route, which adds a layer of protection against sending ugly errors:
Joi.validate(userEntity, SCHEMA_RES_LINKS).then((userEntity) => {
res.send(userEntity);
}).catch((reason) => res.status(400).send(`Something appears to be wrong with this account: ${reason}`));
Joi offers a ton of really neat, helpful utilities around validating and automatically transforming incoming request data, and the API documentation is great. Celebrate is a self-contained and readable middleware wrapper that leverages Joi in a smart way, and makes updating routes a breeze with informative errors. When I started using them, I am going to be totally honest, I was giddy over how much more smoothly my routes worked.
PS: If you'd like to see what I am currently working on with all this, check out LinkMeLater! It's still in testing so expect to get some emails from me π
