Speeding up a Node.js server with a Rust addon

For a project running on a small cloud VM, memory is a huge issue. It can get very expensive, very quickly. I was running a personal project on a small EC2 with only 1GB of memory, and that had a database, an nginx reverse proxy and a Node.js app all running on it. Eventually, it got the point where I couldn’t even update the code without the server running out of memory and freezing up.

To reduce memory usage, I decided to rewrite the most memory-intensive parts of the Node.js app in Rust, a low-level compiled language. Fortunately, it is quite painless to write Node addons with Rust thanks to the napi crate. The only major hiccup I had was that I work on a Mac, but the server runs Debian Linux—meaning I needed to cross-compile. (It would be painfully slow, and memory-intensive, to compile on the server.) Normally, this would be easy—Rust cross-compiles well and using the Zig linker works amazingly for cross-compilation.

Setting up the project

Napi has a cli (npm i -g @napi-rs/cli) that comes with a napi new command, but that creates a package for deploying on NPM. For my use case, I just wanted a sub-folder that I could use with all my other code. So, I needed a few files. I started from just a basic cargo setup (cargo new --lib), and then I set up Cargo.toml:

[lib]
crate-type = ["cdylib"]

[dependencies]
napi = { version = "2.10", default-features = false, features = ["napi6", "async", "tokio_fs", "error_anyhow"] }
napi-derive = "2.9"

[build-dependencies]
napi-build = "2.0"

And, I needed a build.rs that called napi_build::setup(), like this:

use std::env;

extern crate napi_build;

fn main() {
    napi_build::setup();
}

Finally, for my use case I needed a napi.json file that serves as a napi config. Normally this would be part of the package.json, but an alternative config can be set by using the -c config.json flag with the napi cli. The napi.json (schema) serves to set the version of the addon, the name and any additional targets. (x86_64-unknown-linux-gnu, x86_64-apple-darwin, x86_64-pc-windows-msvc are enabled by default, but any other targets need to be added).

{
  "name": "native-addon",
  "version": "0.6.0",
  "napi": {
    "name": "native-addon",
    "triples": {
      "additional": [
        "aarch64-apple-darwin",
        "aarch64-unknown-linux-gnu",
        "universal-apple-darwin"
      ]
    }
  }
}

The Rust code

Actually writing the addon was surprisingly easy thanks to the napi crate (napi docs). I just mark anything I want to export with #[napi], for example:

use napi::bindgen_prelude::*;
use napi::tokio::fs;
use napi_derive::napi;

#[napi]
pub fn add_one(n: i32) -> i32 {
    n + 1
}

#[napi]
pub async fn read_file(path: String) -> Result<Vec<u8>> {
    fs::read(path).await
}

Most Rust types can be exported, including structs and enums

use napi::bindgen_prelude::*;
use napi_derive::napi;

// When a struct is just marked #[napi],
// the fields are not set up as Object keys in JavaScript,
// so it can’t be JSON-stringified for example
#[napi]
pub struct Coordinate {
    pub x: f64,
    pub y: f64
}

// When a struct is marked #[napi(object)]
// the fields are readable from JavaScript,
// so this object can be JSON-encoded
#[napi(object)]
pub struct JsCoordinate {
    pub x: f64,
    pub y: f64
}

#[napi]
impl Coordinate {
    // Functions marked #[napi(constructor)] show up as
    // constructors in the generated bindings
    #[napi(constructor)]
    pub fn new(x: f64, y: f64) -> Self {
        Self {
            x,
            y
        }
    }

    // Rust methods are callable as class methods in JavaScript
    #[napi]
    pub fn coords(&self) -> (f64,f64) {
        (self.x, self.y)
    }

    // Functions that don’t have &self show up as associated functions
    // in JavaScript too, i.e. Coordinate.empty() instead of (new Coordinate(0,0)).empty()
    #[napi]
    pub fn empty() -> Self {
        Self {
            x: 0.,
            y: 0.
        }
    }
}

// Enums are converted to TypeScript enums
#[napi]
pub enum ImageFormat {
    Jpeg,
    Png,
    WebP
}

Modifiers can be added to the #[napi] attribute, for example #[napi(js-name = "funcName")], #[napi(constructor)], or #[napi(ts_arg_type = "string | \"all\"")] on a single argument. Entire function types can be set with #[napi(ts_args_type)] and #[napi(ts_return_type)], and the types of fields in structs can be set with #[napi(ts_type)].

Linking to C dependencies

Unfortunately, my project has to deal with HEIF images, based on the HEVC video codec. Previously, I was using the heic-convert npm package, but that runs a version of libheif compiled to JavaScript, so the performance wasn’t very good. Obviously, that was one of the areas I wanted to port over to a compiled binary. While Rust does have a libheif-rs crate, by default that dynamically links to a copy of libheif wherever it’s running, so I couldn’t have a single, self-contained library that I can cross-compile. I needed a static library version of libheif, for both an Arm Mac and an x86 Linux.

As a C program, getting libheif to compile for a different target is significantly more complicated that for a Rust program. Fortunately, Zig has a built-in C compiler for cases just like this. In the end, I created two git submodules in the project repo—one for libheif and one for its dependency libde265 (because HEIF is based on the H.265/HEVC video codec). This was my .gitmodules file:

[submodule "vendors/libheif"]
path = vendors/libheif
url = https://github.com/strukturag/libheif.git
[submodule "vendors/libde265"]
path = vendors/libde265
url = https://github.com/strukturag/libde265
branch = "frame-parallel"

Both these projects have a CMake build system, so I was able to configure everything I needed with environment variables and command line options in a shell script. In the end, this is what that script looked like:

#!/bin/zsh

if [ "$TARGET" = "" ]; then
  TARGET="aarch64-macos"
  SYSLIBROOT="--sysroot;$(xcrun --show-sdk-path)"
fi

export CPATH="$SYSLIBROOT/usr/include:$CPATH"
export CXXPATH="$SYSLIBROOT/usr/include:$CXXPATH"
export CXXFLAGS="-Wno-error=overriding-t-option -Wno-error=unused-command-line-argument -static $CXXFLAGS"
export CFLAGS="-Wno-error=overriding-t-option -Wno-error=unused-command-line-argument -static $CFLAGS"
export LINKERFLAGS="-Wno-error=unused-command-line-argument -static $LINKERFLAGS"

echo "Building for target '$TARGET'"

git submodule update

echo "Building libde265"
cd vendors/libde265
if [ "$CLEAN" = "true" ]; then
  rm -rf build-$TARGET
fi
mkdir -p build-$TARGET
cd build-$TARGET
cmake .. -DCMAKE_C_COMPILER="zig;cc;-target;$TARGET;$SYSLIBROOT" -DCMAKE_CXX_COMPILER="zig;c++;-target;$TARGET;$SYSLIBROOT" \
  -DCMAKE_C_ARCHIVE_FINISH="zig ranlib <TARGET>" -DCMAKE_CXX_ARCHIVE_FINISH="zig ranlib <TARGET>" \
  -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -DVERBOSE=true || exit
VERBOSE=true make -j8 de265 || exit
ln -s ../../libde265/de265.h libde265

echo "Building libheif"
cd ../../libheif
if [ "$CLEAN" = "true" ]; then
  rm -rf build-$TARGET
fi
mkdir -p build-$TARGET
cd build-$TARGET

export CPATH="../../libde265"
cmake .. -DCMAKE_C_COMPILER="zig;cc;-target;$TARGET;$SYSLIBROOT" -DCMAKE_CXX_COMPILER="zig;c++;-target;$TARGET;$SYSLIBROOT" -DCMAKE_C_COMPILER_AR="zig;ar" -DCMAKE_CXX_COMPILER_AR="zig;ar" \
  -DBUILD_SHARED_LIBS=OFF -DWITH_X265=OFF -DWITH_DAV1D=OFF -DWITH_AOM_ENCODER=OFF -DWITH_AOM_DECODER=OFF -DWITH_SvtEnc=OFF -DWITH_RAV1E=OFF -Dpkgcfg_lib_LIBDE265_PKGCONF_de265="../../libde265/build-$TARGET/libde265/libde265.a" \
  -DLIBDE265_INCLUDE_DIR="../../libde265/build-$TARGET" -DCMAKE_C_FLAGS=$CFLAGS -DCMAKE_CXX_FLAGS=$CXXFLAGS \
  -DCMAKE_C_ARCHIVE_FINISH="zig ranlib <TARGET>" -DCMAKE_CXX_ARCHIVE_FINISH="zig ranlib <TARGET>" \
  -DLIBDE265_LIBRARY="../../libde265/build-$TARGET/libde265/libde265.a" -DCMAKE_BUILD_TYPE=Release -DVERBOSE=true || exit
make -j8 heif || exit
ln -s ../../libheif/heif.h libheif

echo "Done building"

I set it up to use zig as the C and C++ compiler, with the -DCMAKE_C_COMPILER="zig;cc;-target;$TARGET" and -DCMAKE_CXX_COMPILER="zig;c++;-target;$TARGET" flags. I also set it to make a static build (LINKERFLAGS="-static" and -DBUILD_SHARED_LIBS=OFF), and to optimize for speed (-DCMAKE_BUILD_TYPE=Release). Because I was compiling from source, I could also turn off all the features I didn’t need, and tell libheif to look for the libde265 files in my local build folder. Then, all I needed to do was TARGET="<ZIG TARGET>" ./build-vendors, with either aarch64-macos, x86_64-linux-gnu, or any other target supported by zig.

I just had one step left before I could use the statically compiled archives. I could still use the Rust libheif-rs crate, but I needed to override the default linking strategy, which I can do from my project’s build.rs. I set it to link statically to de265, heif, and dynamically to (lib)c++, which would be provided by the target platform and properly linked to by the zig linker. This is what build.rs ended up looking like:

use std::env;

extern crate napi_build;

fn main() {
    println!("cargo:rustc-link-lib=static=de265");
    println!("cargo:rustc-link-lib=static=heif");
    println!("cargo:rustc-link-lib=c++");
    let target = env::var("TARGET").unwrap();
    let zig_target = if &target == "aarch64-apple-darwin" {
        "aarch64-macos"
    } else if &target == "x86_64-unknown-linux-gnu" {
        "x86_64-linux-gnu"
    } else if &target == "aarch64-unknown-linux-gnu" {
        "aarch64-linux-gnu"
    } else if &target == "x86_64-unknown-linux-musl" {
        "x86_64-linux-musl"
    } else if &target == "aarch64-unknown-linux-musl" {
        "aarch64-linux-musl"
    } else {
        unimplemented!()
    };
    println!("cargo:rustc-link-search=/absolute/path/to/libde265/build-{}/libde265",zig_target);
    println!("cargo:rustc-link-search=/absolute/path/to/libheif/build-{}/libheif",zig_target);
    napi_build::setup();
}

The rest of the dependencies were either Rust, or statically built by default, so I didn’t have to do anything special other than tweaking one of the bindgen environment variables for a dependency. I could set that in .cargo/config.toml and it would automatically be used whenever I built the project.

[env]
# Becuase it’s just for bindgen, it can use host header files (nothing will actually be compiled for them)
BINDGEN_EXTRA_CLANG_ARGS = "--sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk"

Finally, I could build the project for any platform with napi build --release --strip --target <RUST TARGET TRIPLE> --platform -c napi.json --zig, and it automatically generates the .node file, a loader JS file, and TypeScript types. In the end, I was able to improve my app’s performance and reduce memory usage significantly. It went from around 85% of total memory to the high sixties, a valuable improvement. The server no longer freezes on updates and there is now memory to spare!