XS Update

The latest update to the XS JavaScript engine continues to focus on adapting and optimizing JavaScript for embedded systems. In this release, the majority of these improvements take place during the preload phase, part of the build process. Additionally, this release adds support for top-level await and WeakRef, two new JavaScript features expected to be part of a future standard. Finally, this release includes a set of bug fixes to improve conformance with the JavaScript language specification as verified by Test262.

Preload Improvements

The preload feature of XS is one of the key reasons it is able to support modern JavaScript on inexpensive microcontroller hardware. There are three major improvements to the preload process included in this release of XS.

For an introduction to the XS preload feature, see the Preload documentation.

Store Instances of Most Built-in Objects in ROM

XS now defines the behavior of all built-in objects when they are stored in firmware. For example, what should happen when writing to an indexed value of a typed array stored in firmware? The JavaScript language standard assumes all typed arrays are writable, so it provides no guidance. The previous version of XS attempted to write to the typed array, which caused a hardware fault. The current version of XS detects the attempt to write to an object stored in firmware and generates a TypeError. This release of XS defines the behavior for all operations that modify instances of built-in objects stored in a device's firmware image.

It is now is possible to create instances of most built-in objects during preload time. They continue to work when stored in the firmware. Most objects are immutable, however, so they cannot be changed. XS supports aliasing for instances of Object and Array which allows instances stored in the firmware to be modified at runtime.

The following built-in objects may be instantiated at preload time and used at runtime: ArrayBuffer, Boolean, DataView, Date, Error, FinalizationGroup, Function, Map, Number, Promise, Proxy, Set, String, Symbol, TypedArray, WeakMap, WeakRef, and WeakSet. XS generates an exception when the following instances are created during preload: AsyncGenerator, Generator, and RegExp. It is possible that a future release of XS may support RegExp instances created during preload.

The job queue is emptied before the preload phase completes. This means that there are never any unsettled promises in the firmware image created following preload.

For further details on the behavior of these instances when stored in firmware, see the Exceptions section of the new XS linker warnings document.

The behavior implemented in XS to make built-in objects safe to store in firmware images is an extension to the JavaScript language. JavaScript language development has focused on execution from RAM, as that is how it is used on the web. The work done in this release of XS may be of interest as a proof-of-concept for future work on the language standard. Related work on Read-only collections is at Stage 1.

Warning on Mutable Module State

The ability to make changes to objects stored in firmware is a feature of XS. That feature has some costs:

• Memory -- Each object stored in flash that may be modified at runtime requires XS to allocate an alias pointer in RAM. Since most microcontrollers are 32-bit, this requires four bytes of memory. While that may not seem like much, they quickly add up in deeply nested JavaScript data structures. Consider that XS runs on microcontrollers with as little as 32 KB of total RAM, so every byte matters.
• Performance -- When getting and setting the properties of an object stored in flash that may also be modified at runtime, XS must do additional work. It must first check the alias pointer before checking the object in firmware. While this extra check is not very heavy, every cycle matters on a microcontroller running at 80 MHz or less.
• Security -- There is nothing inherently wrong with JavaScript modules maintaining state at runtime. Many modules would be unable to perform their work if they could not maintain their own module state. Still, such state is a potential security vulnerability because it creates a potential communications side-channel in systems that host untrusted code. Identifying mutable state in modules by source code inspection is unreliable.

Because there are potential memory, performance, and security costs associated with having objects in firmware that may be modified at runtime, this release of XS reports a warning at build time when it detects such objects. This warning is generated at the end of the preload phase. The following sections give examples of the warnings and techniques for eliminating them.

Use const

Using the let statement at the top-level of a module is effectively modifiable state. Consider the following excerpt from an example sensor driver in the file "sensorexample.js".

let DATARATE_10_HZ = 0b0010;

class Sensor extends I2C {
    constructor() {
        ...
        this.write(DATARATE_10_HZ);
    }
}

export default Sensor;

This assignment generates a warning because it uses a let statement.

### warning: "sensorexample": default() DATARATE_10_HZ: no const

This warning indicates that DATARATE_10_HZ requires XS to create an alias to allow it to be modified at runtime. The solution is to define to use const instead, which tells XS the value can never change.

const DATARATE_10_HZ = 0b0010;

Note that XS does not generate a warning for top-level let assignments which are not used at runtime. If the call to this.write in the example is removed, no warning is generated because DATARATE_10_HZ is no longer referenced at runtime.

Use Object.freeze()

The following object defines a set of constants for an accelerometer sensor.

export const Range = {
    VALUE_16_G: 0b11,
    VALUE_8_G: 0b10,
    VALUE_4_G: 0b01,
    VALUE_2_G: 0b00
};

When built, it generates the following warning.

### warning: "sensorexample": Range: not frozen

While the Range variable itself is a constant, the object it references may be modified, for example by adding or removing properties. To make the object immutable, use Object.freeze:

export const Range = Object.freeze({
    VALUE_16_G: 0b11,
    VALUE_8_G: 0b10,
    VALUE_4_G: 0b01,
    VALUE_2_G: 0b00
});

Additional Techniques

The XS linker warnings document explains these warnings in detail along with techniques for resolving them.

Keep in mind that these are warnings, not errors. There are modules in the Moddable SDK that generate these warnings. These include the TLS cache, which maintains cache state across all TLS connections, and the Piu user interface Sound class, which manages a single AudioOut instance to play all user interface sounds.

Stripping Initialization Byte Code

After the preload phase is complete, some of the JavaScript code is no longer needed because it is not used at runtime. Consider the following example module.

functions makeSquares(limit) {
    let result = new Uint32Array(limit);
    for (let i = 0; i < limit; i++)
        result[i] = i ** 2;
    return result;
}

const squares = Object.freeze(makeSquares(100));
export default squares;

When this module is preloaded, the function makeSquares is run and returns an array. This array is exported from the module. The function makeSquares is never called at runtime (e.g. after the preload phase completes). In previous version of XS, the byte-code for makeSquares is present in the firmware on the device. With the latest version of XS, this function is automatically eliminated from the firmware image.

The savings from stripping unused code after preload varies significantly depending on the scripts. For example, even defining what appears to be a simple class requires executing JavaScript code, and that code is eliminated now. In some large projects that take full advantage of preload, stripping unused code has recovered over 100 KB of flash space. For even small projects built with Piu (e.g. cards) the stripping of unused code saves about 11 KB.

New JavaScript Language Features

Thanks to the work of Ecma TC39, the JavaScript language is constantly evolving. This release of XS adds full implementations of two Stage 3 proposals.

XS is able to integrate support for these, and other new language features for microcontrollers despite their limited capability to store code. XS achieves this by tailoring the build of the XS engine itself to the language features used by the scripts it will run. As a result, for many new features such as WeakRef, if your scripts do not use the feature, the support for the feature is automatically excluded from the XS engine build.

Top-level await

The top-level await feature allows modules to use the await keyword outside of functions. This is useful when paired with dynamic import (already supported in XS) as it allows a module to manage its imports dynamically when loaded.

const strings = await import(`/i18n/${navigator.language}`);

For large, complex systems this can be useful, even necessary. For embedded systems, top-level await is not of immediate value. The preload mechanism used to build most embedded modules executes the module body at build time rather than on the target device. Therefore, the top-level await for such modules is resolved fully at build time. The feature does work on embedded devices and, in time, may prove useful.

WeakRef and FinalizationGroup

The WeakRef and FinalizationGroup are a pair of related new features that provide a new tool for managing resources. A WeakRef allows one object to refer to another in such a way that the referenced object may be garbage collected. This feature is designed for use by authors of frameworks to allow them to more efficiently manage resources.

The behavior of WeakRef depends on the implementation of the garbage collector in the JavaScript engine. TC39 has taken great care to define a safe behavior that can work across all engines.

Because WeakRef is designed to help scripts better manage runtime resources, they have the potential to benefit the resource constrained devices that XS targets. The support in XS allows the exploration of this potential to begin.

Bug Fixes

XS is regularly tested against Test262, the standard test suite for the JavaScript language. The suite itself continues to evolve, both to accommodate new language features and to add more precise tests for existing features.

This version of XS has the best results yet. The details of what is tested and instructions for reproducing the results are in the XS Conformance document.

• 99.74% on language tests (38857 / 38959)
• 99.76% on built-ins tests (29925 / 29998)

The Test262 Report website hosts a summary of Test262 results for major JavaScript engines, including XS. The results on Test262 Report and the XS Conformance document have not matched. Test262 Report indicates failures with tests that check the "ticks" behavior around how asynchronous functions and asynchronous generators invoke their promises. These inconsistencies are the result of a bug in the test harness used to generate the XS Conformance results. This update to XS includes the test harness fix together with fixes for the relevant test cases. The results reported in XS Conformance should now be consistent with results with Test262 Report for these features (once Test262 is run against the latest XS). Many thanks to the Test262 Report team for their work to independently test JavaScript engines, which made it possible to find this bug.

Try It Out

This release of XS is a big step forward for the use of JavaScript in embedded firmware. For the first time, the behavior of all built-in instances stored in a firmware image is well defined. This work is also beneficial to the Secure ECMAScript effort, which makes use of immutable objects.

You can try out the latest XS. The code is available in the Moddable SDK repository. There are build instructions for ESP8266, ESP32, QCA 4020, and Silicon Labs microcontrollers as well as macOS, Windows, and Linux computers. Prebuilt binaries of the engine may be installed using jsvu.