Scalable Web API’s with Express and Async-Await

If you’re a C# developer, you’ve most likely fallen in love with async / await. It makes writing asynchronous code as natural as writing non-asynchronous code, resulting in code that is easier to read, debug and maintain.

The exciting news is that async / await goodness has found its way into the world of JavaScript!

async-await-js.png

Currently async / await is a proposed feature of ECMAScript 2017 (ES8) and is moving from stage 3 (candidate) to stage 4 (finished) in the TC39 process.  But there’s no need to wait for the release of ES2017 — with the help of JavaScript transpilers, such as Babel and Typescript, which can translate future versions of JavaScript into downlevel versions compatible with most browsers, you can start using async / await today!

Clone, fork or download the code for this post: https://github.com/tonysneed/Demo.Express.Async

Because of features like type-safety, interfaces and generics, I’m an unabashed TypeScript fanboy, so I’m going to talk about how to use async / await with TypeScript.  But the approach is the same using the async functions Babel plugin.

TypeScript has had support for async / await on the server-side for Node with ES2015 since version 1.7, but it will add support for async / await on the client-side for ES5/ES3 with version 2.1.  In this blog post, I’ll focus on using async / await to create RESTful Web API’s with Express, a minimalist web framework for Node.

The primary motivation for async / await on the server-side is scalability with Web API’s that perform asynchronous I/O operations.  So if you are performing database queries, calling other web services or interacting with the file system, then it’s imperative that you not block the executing thread waiting for I/O operations to complete.  As I demonstrated in a prior post on async functions in C#, blocking threads on a server will result in creating additional threads, each of which carries a great deal of overhead, both in terms of memory and CPU utilization, which can impact application performance.

From CallbackHell to Async-Await Heaven

In the JavaScript world, you’re less likely to use a synchronous API in an Express application, because just about everything is done with callbacks.  The problem is that you can easily find yourself slipping into what is sometimes referred to as callback hell.  Here is a brief example:

try {
    greeter.getGreeting(g => {
        console.log(g);
        try {
            greeter.getGreeting(g => {
                console.log(g);
                try {
                    greeter.getGreeting(g => {
                        console.log(g);
                    });
                } catch (error) {
                    console.log(error.message);
                }
            });
        } catch (error) {
            console.log(error.message);
        }
    });
} catch (error) {
    console.log(error.message);
}

Promises were invented to help prevent callback hell. Using the getGreeting function as an example, you can refactor it to return a promise, in which you call resolve to return a result or reject to throw an error.

getGreeting(): Promise<any> {
    return new Promise((resolve, reject) => {
        try {
            let greeting = generateGreeting();
        } catch (error) {
            reject(error);
        }
        resolve("Hello");
    });
}

The caller of getGreeting can then handle success and failure by calling then or catch on the returned promise, and catch will handle an error from any of the preceeding then handlers.

greeter.getGreeting()
    .then(g => {
        console.log(g);
        return greeter.getGreetingAsync();
    })
    .then(g => {
        console.log(g);
        return greeter.getGreetingAsync();
    })
    .then(g => {
        console.log(g);
    })
    .catch(e => {
        console.log(e);
    });

While an improvement over callback hell, promises still require async code that is quite different than synchronous code.  To help close the gap, async / await makes it possible to write asynchronous code in much the same way as synchronous code.

async function main(): Promise<void> {
    try {
        let g1 = await greeter.getGreeting();
        console.log(g1);
        let g2 = await greeter.getGreeting();
        console.log(g2);
        let g3 = await greeter.getGreeting();
        console.log(g3);
    }
    catch (error) {
        console.log(error);
    }
}

Each call to getGreeting will take place sequentially without any blocking, and a single catch block will handle errors from any awaited method.

Async-Await with Express

When building a Web API that performs I/O against a data store, it’s helpful to use a repository pattern, so that you can mock the repository without external dependencies, as well as swap out one data access API for another should the need arise.  In the code for this post, I created a ProductsRepository class that uses an in-memory collection to simulate a persistent data store.  It imports a Product class with properties for productId, productName and unitPrice.

class Product {
    constructor(
        public productId: number,
        public productName: string,
        public unitPrice: number) { }
}

export { Product };

ProductsRepository has promise-based methods for retrieveAll, retrieve, create, update and delete, which call resolve if successful or reject if the id is not valid.

import { Product } from "../models/product";

// Methods return promises to simulate IO-bound operations

export default class ProductsRepository {

    // Array of products
    private _products = [
        new Product(1, "Chai", 10),
        new Product(2, "Espresso", 20),
        new Product(3, "Capuccino", 30),
        new Product(4, "Macchiato", 40),
        new Product(5, "Americano", 50),
        new Product(6, "Flat White", 60),
    ];

    retrieveAll(): Promise<Product[]> {
        return new Promise((resolve, reject) => {
            resolve(this._products);
        });
    }

    retrieve(id: number): Promise<Product> {
        return new Promise((resolve, reject) => {
            let product = this.getProduct(id);
            if (product === null) {
                reject(`Invalid id: ${id}`);
            }
            resolve(product);
        });
    }

    create(product: Product): Promise<Product> {
        return new Promise((resolve, reject) => {
            if (this.getProduct(product.productId) !== null) {
                reject(`Product exists with id: ${product.productId}`);
            }
            this._products.push(product);
            resolve(product);
        });
    }

    update(product: Product): Promise<Product> {
        return new Promise((resolve, reject) => {
            let existingProduct = this.getProduct(product.productId);
            if (existingProduct === null) {
                reject(`Invalid id: ${product.productId}`);
            }
            let index = this._products.indexOf(existingProduct);
            this._products[index] = product;
            resolve(product);
        });
    }

    delete(id: number): Promise<void> {
        return new Promise((resolve, reject) => {
            if (this.getProduct(id) === null) {
                reject(`Invalid id: ${id}`);
            }
            this._products.splice(id - 1, 1);
            resolve();
        });
    }

    private getProduct(id: number): Product | null {
        let products: Product[] = this._products
            .filter(p => p.productId === id);
        if (products.length > 0) {
            return products[0];
        }
        return null;
    }
}

In Express it’s customary to create a router to handle requests for a path segment, such as api/products.  The idea is similar to controllers in ASP.NET Web API and promotes improved modularity.  Here is a router that calls promise-returning functions on the products repository to perform I/O operations asynchronously.  But rather than resort to Promise’s then and catch methods, we use async / await so that the code looks cleaner and is easier to undersand, including the use of try / catch for error handling.

import * as express from "express";
import { Request, Response } from "express";

import { Product } from "../models/product";
import ProductsRepository from "../services/products-repo";

let router = express.Router();
let productsRepo = new ProductsRepository();

// GET route
router.get("/", async (req: Request, resp: Response) => {
    console.log("Retrieving products");
    try {
        let products = await productsRepo.retrieveAll();
        resp.json(products);
    } catch (error) {
        console.log(error);
        resp.sendStatus(500);
    }
});

// GET route with id
router.get("/:id", async (req: Request, resp: Response) => {
    console.log(`Retrieving product id ${req.params.id}`);
    try {
        let product = await productsRepo.retrieve(+req.params.id);
        resp.json(product);
    } catch (error) {
        console.log(error);
        if (error.indexOf("Invalid id") > -1) {
            resp.sendStatus(404);
            return;
        }
        resp.sendStatus(500);
    }
});

// POST route
router.post("/", async (req: Request, resp: Response) => {
    console.log(`Creating product: ${JSON.stringify(req.body)}`);
    try {
        let product = await productsRepo.create(req.body);
        resp.json(product);
    } catch (error) {
        console.log(error);
        if (error.indexOf("Product exists") > -1) {
            resp.sendStatus(400);
            return;
        }
        resp.sendStatus(500);
    }
});

// PUT route
router.put("/", async (req: Request, resp: Response) => {
    console.log(`Updating product id ${req.body.productId} to: ${JSON.stringify(req.body)}`);
    try {
        let product = await productsRepo.update(req.body);
        resp.json(product);
    } catch (error) {
        console.log(error);
        if (error.indexOf("Invalid id") > -1) {
            resp.sendStatus(404);
            return;
        }
        resp.sendStatus(500);
    }
});

// DELETE route with id
router.delete("/:id", async (req: Request, resp: Response) => {
    console.log(`Deleting product id ${req.params.id}`);
    try {
        await productsRepo.delete(+req.params.id);
        resp.end();
    } catch (error) {
        console.log(error);
        if (error.indexOf("Invalid id") > -1) {
            resp.sendStatus(404);
            return;
        }
        resp.sendStatus(500);
    }
});

// Export products router module
export { router as productsRouter };

To use async / await with Node, you’ll need a tsconfig.json file at the root of your Web API that specifies a target of es2015. Then add a server.ts file to bootstrap Express and plug the products router into the pipeline.  You can smoke test your Web API using a tool such as Fiddler or Postman.

Enjoy!

About Tony Sneed

Sr. Software Solutions Architect, Hilti Global Application Software
This entry was posted in Technical and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.