XS: Secure, Private JavaScript for Embedded IoT

The latest release of our XS JavaScript engine puts in place fundamental new capabilities to advance Moddable's mission of safely opening IoT products to third party software. These new capabilities are implementations of proposals on track for standardization in Ecma TC39, the JavaScript language committee. The two most significant developments are described in the "Truly Private Implementations" and "Secure ECMAScript" sections below. The release also includes support for many new features in the JavaScript language that are Stage 3 or Stage 4 proposals, as well as a number of bug fixes to improve conformance with the standard.

This release of XS passes 99.84% of the language tests and 99.78% of the tests for built-in objects. The full test262 conformance results are available in the Moddable SDK repository and include detailed notes on the tests run and each failure.

This document describes the additions and changes to the XS engine as they relate to Moddable's work to use JavaScript as the universal language for third party software across IoT products. For a more general understanding of the language features, links are provided to the full proposals.

Truly Private Implementations

JavaScript has never had an easy way to keep the implementation details of a class private. It has been possible to achieve using a variety of techniques that includes closures and WeakMap, but such approaches are neither easy to implement nor maintain. Three proposals make it easy to reliably keep private implementation details of a class truly private.

  1. Class fields - public and private - proposal
  2. Private Methods - proposal
  3. Static class fields and private static methods - proposal

Class Fields

Class fields allow a class to define properties that appears on each instance of the class. Prior to class fields, this is the usual way of defining a property on an instance.

class Digital {
    constructor() {
        this.pin = 0;
    }
}

Using class fields, this code is equivalent.

class Digital {
    pin = 0;
}

The class field is evaluated when the instance is created. The following adds a time stamp property to each instance of the class.

class Digital {
    created = Date.now();
}

The same idea can be applied to static fields. This is how static class fields are usually defined:

class Digital {
    ...
}
Digital.InputPullUp = 1;

Using a static class field, this may now be written:

class Digital {
    ...
    static InputPullUp = 1;
}

While class fields do not bring any new functionality, they do make the code a little more compact and readable. Static class fields help with maintainability as the static field is now initialized inside the class body instead of patching it into the constructor later. Still, what is most important about class fields is that they are the foundation for private class fields.

Private Fields

Private fields are properties of an instance of a class that are hidden from all code outside the class body. Consider the following example:

class Digital {
    created = Date.now();
    get age() {
        return Date.now() - this.created;
    }
}

The created property is intended to be private to the implementation. If not, a script using of the code could change the value of created which would change the value returned by the age getter. JavaScript programmers have long used the convention of a leading underscore on property names to indicate that a field should be considered private by users of the class.

class Digital {
    _created = Date.now();
    get age() {
        return Date.now() - this._created;
    }
}

Unfortunately, this only works if all users of the code follow the convention. That cannot be assumed when building a secure system. JavaScript private fields replace the ad-hoc convention of the leading underscore with a hash (#).

class Digital {
    #created = Date.now();
    get age() {
        return Date.now() - this.#created;
    }
}

The property #created is accessible only to code inside the class body. Any code outside that is unaware of its presence. While some have complained that the # syntax is unfamiliar or even ugly, it has the benefit of being both extremely concise and extremely clear: when reading code, it is always readily apparent if a given property is private or public.

Static private fields follow the same pattern:

class Accelerometer extends I2C {
    static #address = 0x38;
    constructor() {
        super({address: Accelerometer.#address});
    }
}

Private Methods

Private methods (functions) follow the same pattern, allowing a class to have both private methods and private static methods:

class PowerIndicatorLED  {
    static #pin = 5;
    #digital;

    constructor() {
        this.#digital = new Digital({
            pin: PowerIndicatorLED.#pin,
            mode: Digital.Ouptut
        });
    }
    #write(value) {
        this.#digital.write(value);
    }
    #read() {
        return this.#digital.read();
    }
    on() {
        PowerIndicatorLED.#log("on');
        this.#write(1);
    }
    off() {
        PowerIndicatorLED.#log("off');
        this.#write(0);
    }
    toggle() {
        PowerIndicatorLED.#log("toggle');
        this.#write(this.#read() ? 0 : 1);
    }

    static #log(message) {
        trace(message + "\n");
    }
}

The PowerIndicatorLED class keeps all of its implementation information private - the pin number the LED is connected to, the digital instance used to control the LED, its read and write methods, and its static log method. Code using the class sees only the constructor and the on, off, and toggle methods.

Errors Detected at Compile Time

The following code generates an error at compile time because it attempts to access a private field of PowerIndicatorLED:

const power = new PowerIndicatorLED;
power.#digital = 5;
power.on();

.../main.js:2: error: invalid private identifier!

Debugger Support

The JavaScript debugger for XS, xsbug, has been updated to support private fields. Private fields are not private to the debugger, so they may be viewed. They are not displayed by default and are grouped together under the name of the class (since they are private to the class).

Implementation Notes

This is the first implementation in XS of a proposal still working its way through the standard process. As a result, there are some considerations to be aware of:

  1. Private fields cannot be deleted. This means that each private field uses memory on each instance of the class. Some modules in the Moddable SDK delete fields that are not in use to reclaim the memory. That technique does not work with with private fields.

  2. Private fields cannot be frozen. Object.freeze does not apply to private fields, so there is no way to freeze these properties. The XS linker automatically freezes private fields in instances of preloaded objects, but this is non-standard.

  3. Private methods, like private fields, are stored in instances, and therefore use slightly more memory than public methods. While the function objects corresponding to the private methods are shared across instances, the specification requires a reference to each private method be stored in each instance.

Importance

The new support for private fields and methods in JavaScript makes it much easier to create and maintain code that keeps its implementation private. This is not an entirely new capability in JavaScript, but it makes it easier and, often, more efficient to implement this correctly. Over time modules in the Moddable SDK will be updated to use these features. They are already being applied in the work Moddable is doing to implement the Input/Output (IO) proposal being worked on by Ecma TC53, where they allow classes implemented in pure JavaScript the same ability to hide implementation details as native class implementations have always had.

Secure ECMAScript

Secure ECMAScript (SES) is a Stage 1 proposal under consideration by the JavaScript language committee. Unlike the other proposals this document references, SES is still being designed. As a consequence, it is likely that the APIs described in this document will change in the future. The work to implement SES in XS was undertaken to better understand the proposal to be able to contribute to refining its design based on experience implementing and using SES.

The word "secure" is used in IoT too much to have specific meaning, making it necessary to define what is meant in this context. Often security is concerned with protecting executing software from external attacks, for example ensuring that an HTTP server does not leak information in response to an invalid request. SES is different. It addresses internal attacks. For small systems where all the software is written by a small group working together, this is often unnecessary. For large systems authored by a large distributed group, it becomes indispensable. The core problem SES solves is allowing code from different sources to be safely executed in a single JavaScript virtual machine. This ensures each section of code is secure from interference by the others.

SES is not a security model or a security policy. SES is a way to build these using standard JavaScript. By taking this more general approach, SES allows each host to define its own approach to security (including the option to have no restrictions at all). This is important because a security policy that makes sense for scripts running on a lightbulb is unlikely to apply equally well to scripts running on a web server or scripts in a web page. SES provides tools in the language to build secure systems, but it is not a security system.

Moddable's Motivation

Moddable is applying SES to allow safe end-user scripting on IoT products. For example, consider an IoT lightbulb that is powered by JavaScript (if you cannot imagine a JavaScript powered lightbulb, read this). Such a bulb could allow users to install small mods (aka apps) written in JavaScript to implement custom behaviors for the light bulb (if you cannot imagine what those would do or what the code would look like, read these).

The lightbulb manufacturer wants to ensure fundamental aspects of the lightbulb are not interfered with by the mods installed by the user. For example, the manufacturer wants to ensure the Wi-Fi connection is properly managed, the user's private information is protected, and firmware updates are installed to fix security vulnerabilities. The JavaScript code provided by the manufacturer is part of the host and has full privileges. The JavaScript code installed by the user is an app and has limited privileges. This is analogous to the separation between the operating system and app in a mobile phone.

Without something like SES, there is no good way to ensure that the mod's code cannot access information and functionality in the host. SES as implemented in XS combines three mechanisms: freezing all built-in objects, limiting access to globals, and limiting access to modules.

Freezing Built-ins

Because JavaScript is a dynamic language, any script may change the properties of the built-in objects. This powerful feature can easily lead to problems. A malicious script can change the value of Math.PI causing calculations to be incorrect, or patch JSON.stringify to intercept or modify data being serialized for transmission. SES requires that the built-in objects are all frozen, making them read-only so that they cannot be modified by scripts. This ensures that all scripts running in a SES environment have built-ins with exactly the capabilities defined by the JavaScript specification.

By coincidence, this freezing of built-ins is nearly exactly what the XS engine does in the preload phase in order to prepare the JavaScript environment to be stored in the read-only memory of an embedded device.

Limiting Access to Globals and Modules using Compartments

SES requires the host to manage mods' access to globals and modules. By default, only the globals defined by the language specification are available, but the host may add additional globals. By default, all modules are available, but the host may prevent access to some or all of them. The recommended approach to deciding which globals and modules to provide to a mod is to apply the Principle Of Least Authority (POLA). Simply put, that means to give access to the smallest amount of functionality needed for the mod to perform its task.

A host that wants to execute a mod in a secure context creates a new compartment. The Compartment constructor is a global. The first argument is the module specifier for the module to load after creating the compartment. The second argument is a dictionary of globals to make available in compartment in addition those defined by the language. The third argument is a module map, a list of the modules available in the compartment. In this example, the module map only contains the "morsecode" module itself. The lightbulb host creates the compartment and loads the Morse Code mod as follows.

let lightMod = new Compartment("morsecode", {},
        {morsecode: Compartment.map.morsecode});

Module Maps

The above example creates a compartment and loads the module "morsecode" into it. Because a new compartment has access to only standard JavaScript language features, it cannot control the light because it does not have access to an LED driver module. The third argument to the Compartment constructor is a module map, a white list that defines which modules are available to scripts executing in the compartment. Each compartment, including the root compartment where execution begins, accesses its own module map through Compartment.map. For the lightbulb host, the module map might be as follows. The LED driver module here is "my92x1".

{
    "main": "/main.xsb",
    "my92x1": "/drivers/my92x1.xsb",
    "wifi": "/network/wifi.xsb",
    "passwordmanager": "/passwordmanager.xsb",
}

The actual module map uses opaque JavaScript Symbols for the module path (e.g. "/main.xsb" above) so that code running in the compartment cannot change its behavior based on the module path.

Passing this map to the Compartment constructor as the third argument gives the mod access to all modules accessible to the root compartment, including the "my92x1" LED driver module.

let lightMod = new Compartment("morsecode", {},
                    Compartment.map);

However, doing this also gives the Morse Code module the ability to read the user's passwords and control the Wi-Fi connection. To restrict the mod to only controlling the light, use a smaller module map that includes only the LED driver and the mod itself.

let lightMod = new Compartment("morsecode", {},
                     {
                        my92x1: Compartment.map.my92x1,
                        morsecode: Compartment.map.morsecode,
                     });

Any attempt by the mod to load the "wifi" or "passwordmanager" modules fails in a way that appears to the mod as if the module is not installed.

Note: The module map white-list described here was created by Moddable as an extension to the current SES specifications. This extension is necessary because the Moddable SDK runtime is built entirely from modules and the current draft SES specifications do not address handling of modules, only scripts.

Endowments: The Globals White-list

The lightbulb host may define global variables to keep track of state, for example defaultBrightness to keep track of the user's brightness preference. By default this global is unavailable to a mod, but the lightbulb host can make it available by passing it as the second parameter to the Compartment constructor. Such globals are termed endowments and are present in the mod's execution environment when execution begins.

let lightMod = new Compartment("morsecode", {defaultBrightness});

Creating a Communication Channel

Compartments exist to isolate code. Still, some interactions between compartments are usually needed. Endowments are one way to create a limited communication channel across the compartment boundary. For example, the lightbulb host does not want the mod to have unrestricted access to the network, but it may want the mod to be able to deliver error reports to a cloud service or a local log file. In the following example, the host creates a global reportError function in the mod's compartment to deliver error reports.

globalThis.log = new Log;
globalThis.cloud = new Cloud("http://wwww.lightbulb.com/service");

function hostReportError(message) {
    message = message.toString();
    log.write(message);
    if (cloud)
        cloud.reportError(message);
}

let lightMod = new Compartment("morsecode", {reportError: hostReportError});

When the mod calls reportError the host's hostReportError function is called. At that time, the execution context changes from the lightMod compartment to the root compartment. When hostReportError returns, the context changes back to the lightMod context. This automatic and lightweight context switching is a key characteristic of SES that makes it well suited to execution on less powerful hardware. Also, note that the reportError global exists only the mod's compartment, not in the host's compartment.

SES Runtime Efficiency

Not only are execution context switches (e.g. switching between compartments) in SES efficient as described above, but the runtime memory use is also relatively light.

Without SES, one common approach used to isolate JavaScript is to use a worker (e.g. Web Worker), a completely separate JavaScript virtual machine. This virtual machine requires its own stack, storage, globals, etc. A minimal virtual machine in XS when using web workers, such as the basic web worker example, requires 6 to 7 KB of RAM. A virtual machine of that size is minimally functional, but unable to do much meaningful work. The actual size is almost always larger.

Creating a new compartment in the Moddable SDK uses about 2 KB of RAM (as measured using the compartment examples). Because the compartment runs in the same JavaScript virtual machine, there is no need for a separate stack, memory heap, or virtual machine data structures. This makes compartments practical on a much more constrained device than using a worker for code isolation.

As an added benefit, function calls between compartments are made directly whereas using web workers requires marshalling and unmarshalling of arguments and a message dispatcher, all of which takes more time, requires more code, and uses more memory.

Finally, XS is able to garbage collect compartments, allowing the resources used by a compartment to be reclaimed when there are no references to it. Because compartments are relatively lightweight to create and that they are automatically garbage collected, they can be used for transient purposes.

Development and Debugging

SES is part of XS. There is no special flag required to turn it on. If a script in a project accesses the Compartment global, SES is automatically built into the XS engine. If SES is unused, most of the code that implements it is automatically stripped from the build.

The xsbug JavaScript debugger has been enhanced to support SES debugging. By way of an example, consider the classic Piu balls app extended with a "test" module that consists solely of a debugger statement. The following line at the end of main.js:

let testMod = new Compartment("test", {Application},
        {test: Compartment.map.test});

This creates a compartment to contain the module test. The module is given access to the Piu Application global and to its own module, but no other modules. When executed, the debugger statement causes it to stop in the compartment. Notice on the left column that the only module visible is test and that the globals are only JavaScript built-ins with the addition of Application.

Without stepping, selecting the stack frame for the call to the compartment constructor shows all the Piu and Commodetto module and globals are available to the root compartment.

OCAP and Further Reading on SES

Using the basic SES building blocks of frozen built-ins, endowments (the global map), and a module map, it is possible to build a variety of simple and complex security models. The design of SES is well-suited to implementing the Object Capabilities (OCAP) model, which is what Moddable is exploring with this implementation of XS.

A more detailed set of examples that walks through various scenarios using Compartments is included in the Moddable SDK. The compartments document describes those examples.

The details of each SES are complex. This document is a high-level, informal introduction, not a detailed specification. To understand SES in more detail, the draft standalone specification prepared by Mark Miller of Agoric is an excellent place to start.

Seven More New JavaScript Language Features

This section describes additional JavaScript language standard proposals now fully supported in XS.

Numeric separators

This proposal adds the ability to use an underscore character in numeric literals to improve readability. For example:

const oneBillion = 1000000000;
const oneBillionSeparated = 1_000_000_000;

Separators work for hexadecimal and binary number as well, which is convenient when coding hardware related constants:

Sleep.BrownOutReset = 0b0001_0000; // power dipped too low
Red = 0xFF_00_00;

Dynamic Import

This proposal allows JavaScript modules to be imported dynamically by (what appears to be) a function call to import. Previously JavaScript modules could only be imported statically through import statements.

const Digital = await import("builtin/digital");
let pin = new Digital({});

The dynamic import is an asynchronous operation because the result may not be available immediately. For example, on the web the module source code may first need to be obtained from a remote network server.

The XS engine has long supported dynamic import using a non-standard global require function loosely modeled on the function of the same name in Node.js. The require function has been removed in favor of the standard dynamic import functionality. As few projects used require directly, this has little impact. Some parts of the set-up code in the Moddable SDK used require, for example to bind to the display and touch drivers specified in the project manifest. The set-up code has been updated to work with the changes to XS.

The "XS in C" API defined in the xs.h header file now includes the xsAwaitImport function which native code may use to perform a synchronous import. This is primarily used to bootstrap the runtime of the Moddable SDK. Hosts are discouraged from using this native function to provide synchronous import to scripts.

Import Metadata

This proposal provides a standard way for a module to obtain information about itself at runtime. The proposal only defines a single piece of metadata: the URL the module was loaded from. This is useful for loading additional modules or assets from the same path. It is expected that additional properties will be added in the future. This proposal has no practical use in the Moddable SDK at this time.

globalThis

This proposal provides access to the JavaScript global scope through globalThis. An earlier version of the proposal used global to access the global scope, but this name had compatibility issues with certain web sites.

The Moddable SDK has supported the name global for some time. This release adds support for globalThis without removing support for global. Both will co-exist for some time. It is recommended that code migrate to using globalThis.

String.prototype.matchAll

This proposal adds the ability to access all matches of a regular expression through a JavaScript iterator. This capability is valuable for some uses of regular expressions. Most projects built with the Moddable SDK do not use regular expressions. However, it is appreciated that the design returns an iterator rather than an array of results as this approach allows memory use to be minimized for large result sets making this feature more practical for memory constrained devices.

Promise.allSettled

This proposal adds a new function for working with promises. It reports when all promises in a group have been either resolved or rejected. This operation simplifies working with groups of promises, when the result of all promises is needed before proceeding. Projects built with the Moddable SDK make sparing use of asynchronous functions and seldom have multiple promises pending simultaneously. Still, on more powerful targets such as the ESP32, promises work well and often do not tax the available resources so this capability may well be useful to some.

Hashbang Grammar

This proposal extends the JavaScript language grammar to treat a "hashbang" (e.g. #!) that appears as the first two characters of the first line of a JavaScript code file as a comment. 

#!/usr/bin/env node
export {};
...

This has been added to make standard a common practice in some parts of the JavaScript ecosystem. It has no practical impact on the Moddable SDK.

Bug Fixes

This release of XS contains a number of bug fixes, many of which are to improve conformance with the test262 suite. This section describes two fixes made that are outside the test262 tests.

Writable Globals in Frozen Modules

The XS preload feature loads designated modules at build time to reduce RAM usage and start-up time on the target device. The mechanism is documented in detail. The original implementation of this feature made the top level declarations of a module read-only. The following module code would fail in the constructor at runtime if the module was preloaded:

let count = 0;
export class Counter {
    constructor() {
        count++;
    }
    static get count() {
        return count;
    }
}

XS now aliases closures which allows such top level declarations in a module to be writable. Note that there is some runtime memory and execute cost to this, so it should be used sparingly. However, it is more efficient than existing workarounds, so code should be updated to eliminate the workarounds.

When declaring top level constants in a module be sure to use const instead of let or var. Using const tells XS the declaration can never change, allowing it to save an entry in the alias table (four bytes on most microcontrollers).

RegExp Benchmark

It was recently observed that XS fails when attempting to run a RegExp benchmark in the V8 repository. This has been fixed and may suggest an additional test case to be added to test262.

Conclusion

With this update, XS again delivers the latest advances in the JavaScript language to developers working on embedded systems. While the evolution of JavaScript is still guided by the needs of the web platform, a remarkable number of the recent proposals implemented in this update benefit embedded development of IoT products.

The most important language work from Moddable's perspective are Secure ECMAScript, and private fields and methods. That is why Moddable has invested in making XS the first JavaScript engine to implement Secure ECMAScript. Viewed from a purely language perspective, these features provide strong encapsulation with the benefit of simplifying some aspects of software development, particular in large systems.

Viewed more broadly, having strong, reliable encapsulation that is straightforward to use creates entirely new opportunities. Agoric is using Secure ECMAScript for secure, smart contracts. Salesforce is using Secure ECMAScript in its Lightning Locker service to provide enterprise grade security for scripts running the browser. And Moddable is applying Secure ECMAScript to empower users to install the software they choose on their IoT products, just as they do today on computers and mobile devices.