Moddable SDK Updates

There's been a lot of good work in the Moddable SDK in recent months. This blog post highlights a some of the recent additions and improvements.

mDNS

The mDNS protocol is arguably the most robust solution available for IoT devices connected to the same local network to discover one another. It is a mature technology initially deployed by Apple over fifteen years ago and since adopted by countless others.

We've implemented mDNS support in the Moddable SDK. We chose to make it part of the Moddable SDK rather than relying on platform implementations because many platforms, such as the ESP8266, do not have a strong mDNS implementation. We also believe that it is appropriate, even beneficial, to implement network protocols for embedded devices in JavaScript.

Our implementation supports three major features of mDNS. First, it supports claiming and defending a local name for a device. This means that your IoT product can be accessed as "mydevice.local" instead of "10.0.1.5," creating a better experience for users and ensuring a name that remains consistent independent of IP address distribution. Second, it supports advertising DNS-SD services, so that other devices can discover the network services provided by your device. Finally, it supports discovering the DNS-SD services advertised by other devices. This allows your device to discover and use the network services of other devices.

The code for our mDNS implementation is available for your use and review. We intend to continue to enhance and improve the implementation. The mDNS class documentation includes simple examples of using the API to claim a name, advertise services, and discover services.

DNS parser and serializer

Our mDNS implementation uses new DNS parser and serializer modules to work with the binary DNS packet data. The DNS parser module extracts resource records from a binary DNS packet. The DNS serializer module combines one or more resource records into a binary DNS packet.

Web Things API

The Mozilla Project Things initiative shares with Moddable the goal of putting users in control of their IoT products. We undertook an exploration of the Web Things API, part of the Project Things effort, with the goal of achieving interoperability with Mozilla's home gateway. The Moddable SDK now has a Web Things API module which makes it easy to make a device compatible with the Mozilla gateway.

There are several examples of Web Thing compatible devices available. These include a simple on/off light, an off/off switch, a remote controlled power outlet, a sign that displays scrolling messages, and a thermostat. All of these examples have a graphical simulator built using Piu, so they run on a Moddable Zero without requiring any additional hardware. The documentation provides a reference on using the WebThings module in your own projects. The code is available to review and improve.

BLE Human Interface Device (HID) client

In June we added support for BLE to the Moddable SDK. Since then, we have been exploring uses of BLE. We recently added support for working with BLE Human Interface Devices (HID), for example a keyboard or mouse. These allow ESP32 and Gecko MCUs to use a keyboard or mouse as a peripheral. Some, but not all, keyboard and mouse products support the standard BLE HID protocol.

The HID support is implemented in a module that is not specific to any particular type of a HID device. The module can be used to build support for different type of devices. The hid-keyboard example includes a module that implements the HID keyboard protocol and a simple application that echoes key presses to the device console. Similarly, the hid-mouse includes a module that implements the HID mouse protocol and an application that outputs mouse coordinates and clicks to the device console. The hid-mouse-gui reuses the HID mouse module to implement a mouse pointer on a display, building on the Piu drag example.

Easier builds for device targets

One challenge in embedded development is properly configuring a build for a particular device target. The Moddable SDK uses the mcconfig command line tool together with JSON manifest documents to manage this process. Originally this mechanism targeted a build based on platforms, for example macOS, Windows, ESP8266, ESP32, and Gecko.

This works well when all devices in a platform have similar hardware characteristics. An ESP8266 for example can be connected to many different kinds of hardware. The build configuration varies depending on what is connected, for example a screen or speaker. Linux, macOS, and Windows solve this problem by building in support for a wide variety of drivers. For embedded devices, this is impractical as there isn't enough flash storage space for unnecessary drivers.

For the Gecko family of devices, we added support for sub-platforms to manifests and mcconfig to more easily support hardware variations. Last month, we extended sub-platform support to ESP8266 and ESP32 targets. These device targets bring together in one place native configuration, JavaScript configuration, modules, and include code. This is done in an extensible way, so it is easy to add support for a new sub-platform device target.

On ESP8266, the esp platform target is useful when working with a bare board, such as a NodeMCU board. The esp/moddable_zero target for the Moddable Zero board includes the ILI9341 display driver, XPT2046 touch driver, and configures the I2C pins to match the board pin outs.

The ESP32 has spawned a relatively large number of development boards, with a variety of different peripherals. The esp32 device target is used for bare boards, such as the NodeMCU ESP32 board. Additional targets are provided for M5Stack (esp32/m5stack), M5Stack Fire (esp32/m5stack_fire), Oddwires (esp32/oddwires), LilyGO TAudio (esp32/lilygo_taudio), LilyGo T5S (esp32/lilygo_ts5), and the Moddable Zero style display wiring for ESP32 (esp32/moddable_zero).

TinyInt

With BigInt on track to become part of the JavaScript language standard, now seems like a good time to introduce TinyInt. There are many situations in embedded development where a large number of integer values needs to be stored in memory, for example collecting sensor readings over an extended period of time. TinyInt is a tool to pack small integers efficiently into an array.

In JavaScript, the TypedArray subclasses (Uint8Array, Int16Array, etc.) store integer data in a more compact representation than the built-in Array. This is useful when working with large arrays, as the memory savings can be significant. In XS on an embedded device, each Array element requires a slot, which is 16 bytes. Using a Uint8Array instead to store an integer 0 or 1 for a boolean uses only one byte, or sixteen times less memory. However, that is still eight times more memory than needed.

To create an array of tiny integers, invoke the constructor in one of two ways. The first provides the element count and the number of bits for the array elements. This example creates an array of 1024 elements, each one bit in size:

let flags = new UintBitsArray(1024, 1);

The second way to invoke the constructor allows the caller to allocate the buffer that stores the bits. This example creates an array of 512 2-bit elements:

let flags = new UintBitsArray(new ArrayBuffer(128), 2);

As with TypedArray, the length and buffer of the array are available as properties:

flags.length
flags.buffer

Elements of the array are accessed with the usual brackets notation:

flags[3] = 1;
flags[4] += 2;
if (flags[5])
    trace("true");

TinyInt supports field bit sizes from 1 to 32. Temperature sensors, for example, typically provide readings as 12 bit fixed pointed values, which can be stored 25% more compactly using a 12-bit TinyInt array than a Uint16Array.

TinyInt extends DataView with getUintBits and setUintBits to allow arbitrary bitfield access to the buffer. Here are the function signatures:

getUintBits(bitsOffset, bitsSize [, endian])
setUintBits(bitsOffset, bitsSize, value [, endian])

The optional endian argument is for consistency with other getters and setters in DataView. They are defined so that 16 and 32-bit values written to a a byte aligned byte offset give the same result as setUint16 and setUint32.

The code for our initial implementation is available on GitHub. It should work on all device targets supported by the Moddable SDK. The implementation contains stubs for signed integer support as well, which may also be useful.

We expect TinyInt to be useful in a variety of situations. Our focus is on its use in embedded JavaScript applications. We are interested in hearing feedback on the design as well as how you might use it on embedded or elsewhere. TinyInt seems like the kind of feature that is well suited to inclusion in the JavaScript Standard Library, an idea recently reactivated in a proposal by Apple.

Crashing exploits

A properly functioning JavaScript engine should never crash, no matter what a script does. This is one reason that, in principle, JavaScript is a good choice for implementing robust IoT products. Of course, software has bugs and our XS JavaScript engine is no exception. We recently received reports of several crashing bugs from Jake Miller of Bishop Fox. Our thanks to Jake for reporting these together with simple scripts below that reliably reproduced the issues. Our most recent push includes fixes for these issues.

If you encounter a crash in XS in your work with the Moddable SDK, we would very much appreciate you reporting that. A crash in the engine is a potential security exploit. We want to identify and resolve those as quickly as practical. You may email us with the issue at info@moddable.com or open an issue on GitHub.

heap-overflow

let someArray1 = [];
someArray1[4000000000] = 1;

//Fill triggers overflow
someArray1.fill('A');

heap-overflow2

let someArray1 = [];
i1 = 0;

// Control the amount of overflow with loop
for (; i1 < 4000000000; i1 += 1500000000) { //96 gb
    someArray1[i1 + 3] = 1;
}

//triggers heap overflow
someArray1.fill('A');

out-of-bounds-read

// 2-byte Out-of-bounds read
let someArray1 = [1,2,3,4,5,6,7,8,9];

someArray1.sort(function(a, b) {
    return true;
});

stack-overflow

let someArray1 = Array(63);

someArray1.fill(1);
someArray1.unshift(1, -1);

someArray1.sort(function(a, b) {
    return a;
});

JavaScript language conformance

The JavaScript language specification is well over 1500 pages. We regularly run test262, the JavaScript language conformance test suite, against XS to help ensure strong conformance with the standard. While test262 is an amazing resource, it does not catch all issues.

Kevin Gibbons of Shape Security, and a member of TC39, reported two issues via GitHub.

Michael Hunter observed, while reviewing the XS source code, that the increment and decrement operators would generate an incorrect result on the boundary where an integer is no longer large enough to hold the result.

All of these issues are fixed in the current repository. While these issues may seem to be obscure edge cases, we want to ensure that the XS engine's implementation of JavaScript is as consistent as possible with the language standard to provide developers a consistent experience everywhere. Our thanks to Kevin and Michael for the reports.

Contributions

The Moddable SDK has a new folder for contributions from the community. We created it to share code that we believe will be useful to developers working with the Moddable SDK, but that may not yet be fully integrated into the SDK.

The first contributor is Diogo Vianna who provided a Serial module for use on ESP32. Check it out here.