XS Engine Updates for ECMAScript 2020

The XS JavaScript engine brings the many benefits of modern JavaScript to embedded systems like those that power IoT products. The latest release of XS is a major step forward. It delivers new language features, performance optimizations, improved conformance with the ECMAScript specification, and updated support for the Secure ECMAScript proposal.

This blog post summarizes the changes with a focus on their impact for developers working with JavaScript on embedded systems.

ECMAScript 2020 Features

XS supports all the new language features included in the current draft of the ECMAScript 2020 specification.

Note: As of this writing the ECMAScript 2020 specification is available from the Ecma TC39 committee web site. Once it is officially approved, the standard will be available on the Ecma International web site.

Nullish Coalescing

JavaScript now supports a nullish coalescing operator, using ??. The operator is a convenient shortcut to test if a value is equal to undefined or null. It is common in JavaScript to write the following to provide a default value for a missing value:

let x = (value !== undefined) ? value : defaultValue;

Using the nullish coalescing operator simplifies the code:

let x = value ?? defaultValue;

The two are not entirely identical as ?? checks for both undefined and null while the original code checks for undefined only. In practice, JavaScript code uses both null and undefined for missing values, so the behavior of the nullish coalescing operator gives the correct result in both cases.

Developers have long used the logical or operator to achieve a similar result:

let x = value || defaultValue;

This often works, but it is imperfect. The logical or operator treats many values as false including 0, the empty string, null, and undefined. This makes it unsafe to use the logical or operator in all circumstances. Therefore, it is recommended to use the nullish coalescing operator instead, now that it is available.

Optional Chaining

The optional chaining operator is an efficient way to confirm a value is an object before getting a property or calling a method. For example. the MQTT client in the Moddable SDK has code like this:

if (this.onClose)
    this.onClose();

Using the optional chaining operator, this can be written as:

this?.onClose();

This operator is also convenient when working with data structures, such as objects instantiated from JSON. Code often needs to check if a property exists in JSON before accessing its values.

let record = JSON.parse(text);
let zip;
if (record.address)
    zip = record.address.zip;

Using optional chaining this can be written more compactly.

let record = JSON.parse(text);
let zip = record.address?.zip;

Both optional chaining and nullish coalescing extend the language syntax to ease checking for missing values, a common task in JavaScript code. These two features may be combined. The following example retrieves the brightness property of this.light. If this.light is missing or the brightness value is undefined, a default value of zero is used.

let brightness = this?.light.brightness ?? 0;

In addition to being more compact, the byte code generated using optional chaining is smaller and runs faster. Consider this code fragment that does not use optional chaining:

if (this.#resolver)
    this.#resolver.close();

It generates the following ten byte codes.

this
get_property [resolver]
branch_else
this
get_property [resolver]
dub
get_property [toString]
call
run_1 0
pop

Rewriting the code with optional chaining...

this.#resolver?.close();

...generates only eight byte codes.

this
get_property [resolver]
branch_chain
dub
get_property [toString]
call
run_1 0
pop

While saving two byte codes may not appear important, in an embedded system where every byte of flash storage and every CPU cycle are precious resources, this is a welcome improvement. Further, note that the resolver property is only retrieved once in the optional chaining version. Property look-up is not always fast, so eliminating the unnecessary look-up helps.

String.prototype.replaceAll

JavaScript has long had a replace method for strings. Using a Regular Expression with replace supports replacing multiple occurrences of a string with a single call to replace. However, when using string, only the first occurrence is replaced. The behavior of replace cannot be safely changed as it would likely break existing web sites. Therefore, a new replaceAll method has been added to provide the ability to replace all occurrences of a string.

To implement the equivalent of replaceAll the Moddable SDK sometimes uses split followed by join. For example, the following fragment replaces all colons with an empty string.

let mac = Net.get("MAC").split(":").join("");

Using replaceAll is more clear, concise, and efficient.

let mac = Net.get("MAC").replaceAll(":", "");

Using Regular Expressions with replace the same result is possible. However, not only are Regular Expressions more difficult to understand, they are seldom used in embedded systems as they are a relatively heavy feature of JavaScript. The replaceAll function is a welcome addition that is easy for developers to understand while allowing for efficient implementation on embedded systems.

Promise.any and RegExp Match Array Offsets

This release of XS also implements new features for Promises and Regular Expressions. Neither is commonly used in embedded systems, so they are only described here briefly.

Promise.any is a useful helper function when resolving a group of promises simultaneously. It completes the suite of promises helper functions that includes Promise.allSettled, Promise.all, and Promise.race.

Regular Expressions have been extended to provide additional information about captured substrings. It adds an indices property to the regular expression that contains the start and end indices of captured substrings.

Performance Optimization

The ability of XS to execute JavaScript efficiently on resource constrained embedded systems is essential to enable developers to use JavaScript in their products. XS has long had optimizations which allow the JavaScript Number type to be implemented using integers when possible. This optimization is especially important for embedded systems where floating point is implemented in software.

To help developers better understand when their code is using floating point numbers, the Instruments pane in the xsbug debugger has been enhanced with a count of the number of floating point operations performed by XS. This value is updated once a second. The image below shows the Floating Point instrumentation when running the Piu Cards example.

Because Cards makes extensive use of Timeline for animations, there are a large number of floating point operations. The Piu Balls example, on the other hand, has no floating point operations after its initial set-up. There is nothing inherently wrong with floating point operations. They are useful, even essential. However, they can have a performance cost so it is helpful to measure and visualize when they occur.

Monitoring of the floating point instrumentation in xsbug brought to light some places where XS used floating point numbers where an integer was sufficient. Two of these are worth noting. The first is the length property of the built-in JavaScript Array. For conformance reasons, there are situations where the length needs to be a non-integer value. However, this is not usually the case. The XS implementation now returns an integer in most cases, speeding array iteration. The second case is TypedArray objects, such as Uint8Array and Int32Array. Getting and setting values into TypedArrays was using floating point operations because the code was generalized to also work with floating point arrays like Float32Array. The code has been modified to allow integer-only operations to bypass unnecessary floating point conversion. Because TypedArrays are commonly used in embedded development to work with binary data, this behind-the-scenes improvement benefits many different modules.

These floating point optimizations are a good example of the challenges faced when implementing modern JavaScript on microcontrollers. XS must balance multiple factors in its implementation: execution performance, code size, memory use, code complexity, and conformance with the language specification.

Language Conformance

During development, XS is regularly run against Test262, the conformance test suite for the JavaScript language. The latest version of XS shows improved conformance. The test results based on execution of Test262 by Moddable shows 99.9% (39039/39065) of the language tests passing and 99.8% (30164/30234) of the built-in tests passing. Each of these results surpass the most conformant JavaScript engine available in a web browser. Notes on the failures are available.

Bocoup independently runs Test262 on all major engines and posts the results at the Test262 Report web site. The results differ in places from Moddable's run of the same tests, primarily because Test262 Report terminates some tests before they complete because they require more time to execute with XS. Still, the Test262 report results show XS is the most conformant engine in the language tests and second most conformant in built-in tests.

This version of XS contains many changes, large and small, to improve conformance. Of particular note are changes to the function calling conventions. This is the first time in the development of XS that these have changed. The changes were necessary to support certain required behaviors and to efficiently implement optional chaining.

Secure ECMAScript Update

Secure EcmaScript (SES) is a proposal to add secure virtualization support to JavaScript. It is an early stage proposal that addresses the challenge of making it safe to run untrusted JavaScript code on embedded systems. Ecma TC53, the ECMAScript Modules for Embedded Systems technical committee, is considering requiring SES as a baseline for its work to ensure a secure software foundation for consumer and industrial IoT products. Moddable has chosen to implement SES in XS at this stage to help validate and refine SES.

XS has included built-in support for SES since July 2019. The "XS: Secure, Private JavaScript for IoT" blog post introduces SES and the implementation in XS. The basic unit of virtualization in SES is the compartment. This release of XS updates the implementation of the Compartment class to match the latest draft of the proposal presented to TC39.

The Compartment API design in the current SES proposal is the result of initial work by Agoric on their SES shim and the original XS implementation of SES. Those two efforts were reconciled and combined into the current design by JF Paradis. The result is an API that is remarkable both for its simplicity and power.

The Moddable SDK contains a small suite of SES examples that exercise the features of Compartments. The documentation for the examples provides detailed walkthroughs of the code.

Conclusion

Moddable continues to invest in enhancing the XS JavaScript engine to deliver the best possible tools for developers building embedded products. The latest release of XS reflects the many dimensions of that effort -- ongoing work to give developers native support for the latest JavaScript language, optimizations that improve performance and reduce code size, ongoing work to ensure XS is able to pass the rigorous conformance requirements of Test262, and an investment in validating and refining emerging language features like SES that are poised to deliver a significant benefit to embedded software developers.