NeoPixels, Moddable style
NeoPixels are strings of bright LEDs that have been popular within the maker community for adding animated illumination to objects. Several recent ESP32 based development boards have built-in NeoPixels, including the M5Stack Fire and Lily Go TAudio. Both of these boards drive NeoPixels using MicroPython. This article introduces a new module for the Moddable SDK to control the NeoPixels using standard modern JavaScript. It explains how to use the API as well as the API design, which aims to balance the goals of simplicity, speed, and portability.
Background
A strand of NeoPixels is driven by a single Digital pin (GPIO). There is no single controller for the pixels. Instead, each pixel contains a tiny controller which receives the command to set the pixel's color. The tiny controller on each pixel requires very precise timing in the signal. Consequently, the biggest obstacle to getting NeoPixels to work reliably has been the code on the microcontroller (MCU) to transmit color data to the pixels.
More modern MCUs, including the ESP32, contain specialized hardware support that makes this task simple. On the ESP32, a technique has been developed that uses the Remote Control (RMT) hardware module designed to generate signals for infrared remotes. This approach uses the ESP32 hardware to time transmission of the pixel color data, ensuring reliable timing.
For the NeoPixel module in the Moddable SDK, we use the RMT code used in the MicroPython ESP32 implementation. Created by Boris Lovosevic, this small C library cleanly encapsulates the code to control the RMT module for NeoPixels. The module is used unchanged, although the code for setting individual pixel values is not used for performances reasons. More on this below.
Note: DotStar is an alternative to NeoPixel that provides similar capabilities with the advantage of not being sensitive to small timing variations. DotStar is expensive than NeoPixel so cost-sensitive hardware manufacturers continue to favor NeoPixel for many applications. The Moddable SDK also supports DotStar.
Programming
The JavaScript code to work with a NeoPixel strand uses a small API that provides access to individual pixels and fast fill operations. The following section introduces the APIs using examples.
Configuring a NeoPixel instance
To use the NeoPixel module in an app, first import it:
import NeoPixel from "neopixel";
The NeoPixel module must be configured to work with the strand attached to the MCU. The following properties must be configured:
- pin - the Digital pin that transmits the pixel color data signal
- length - the number of pixels in the strand. Memory is required for each pixel, making it essential this number of property configured to avoid allocating memory that will not be used
- order - Some pixels have only red, green, and blue LEDs while others add a white LED. Additionally, the order that the color components are arranged in varies. The order property is a string that indicates both the number of color components and their ordering, for example "RGB" or "BGRW". The default is "GRB".
Pass the configuration properties to the NeoPixel constructor in a dictionary:
let np = new NeoPixel({length: 19, pin: 22, order: "GRB"});
The constructor resets the pixels in the strand to all off.
Calculating RGB colors
The next step is creating some colors values. This is a two step process for efficiency reasons (this will be explained below). To create a color, use the makeRGB method, passing the red, green, blue, and white (if the strand has the optional white LED) color components as values from 0 to 255:
const red = np.makeRGB(255, 0, 0);
const green = np.makeRGB(0, 255, 0);
const blue = np.makeRGB(0, 0, 255);
const white = np.makeRGB(255, 255, 255);
const black = np.makeRGB(0, 0, 0);
The value returned by makeRGB is a a JavaScript number, internally represented as an integer, with the color components in the order required by the NeoPixel strand in use.
Setting pixel color
Individual pixels on the strand are set to a color using setPixel. The call to setPixel does not transmit the pixel to the strand to allow multiple pixels to be updated with a single transmission. Once all pixels have been set, call the update method to transmit the color values to the NeoPixel strand.
The following code sets the strand to alternating blue and white pixels. The loop uses the length property of the NeoPixel instance to know the number of pixels in the strand.
for (let i = 0; i < np.length; i++)
np.setPixel(i, (i & 1) ? white : blue);
np.update();
Filling with colors
To fill the entire strand with a single color, use the fill method to set the pixel colors, again followed by update to transmit the updated pixels.
np.fill(blue);
np.update();
The fill method may also be used to set a subset of consecutive pixels using the optional start and length arguments. The following sets the pixels 2, 3, 4, 5, and 6 to white:
np.fill(white, 2, 5);
If the optional length argument is omitted, pixels from the index of the start argument to the end of the strand are filled. The following sets pixels from index 10 to the end of the strand to green:
np.fill(green, 10);
Adjusting brightness
The low level NeoPixel driver code implements a brightness feature in software to fade the entire strand without modifying each individual pixel value. The following code fades the strand from full brightness (255) to off (0) in one second at 16 ms intervals (60 FPS).
import Timer from "timer";
let step = 0;
Timer.repeat(id => {
np.brightness = ((60 - ++step) / 60) * 255;
np.update();
if (60 === step)
Timer.clear(id);
}, 16);
Calculating HSB colors
A popular use of NeoPixel strands is to display animated rainbows of colors. The most convenient way to calculate rainbow colors is using the HSB (Hue - Saturation - Brightness) color representation, instead of the more common RGB.
The makeHSB method calculates a pixel color value from hue, saturation, brightness, and white (when the strand has the optional white LED) components values. The hue value has a range of 0 to 359, saturation and brightness range from 0 to 1000, and white is 0 to 255. The following example displays a simple rainbow across the length of a strand:
for (let i = 0; i < np.length; i++) {
let h = i / np.length) * 360;
let color = np.makeHSB(h, 1000, 1000);
np.setPixel(i, color);
}
np.update();
Direct pixel access in JavaScript
The pixels on a NeoPixel strand are represented by an array of bytes with either 3 or 4 bytes per pixel depending on whether the NeoPixel strand uses RGB or RGBW LEDs. The NeoPixel instance provides direct access to this underlying array of bytes. The NeoPixel instance is also a host buffer, a reference to a buffer of native memory, which can be used as an array buffer. For example, for an RGBW NeoPixel strand each pixel is 4 bytes, which can be represented by a Uint32:
let pixels = new Uint32Array(np);
pixels[0] = white;
pixels[1] = ~pixels[0];
Direct pixel access in C
For long NeoPixel strands that involve complex calculations for each pixel, it is more efficient to implement the color calculations in C. The buffer property provides direct access to the pixels.
The following example extends subclasses the NeoPixel class to add a native invert method that performs a bitwise NOT on the entire contents of the pixel buffer:
class NeoPixelPlus extends NeoPixel {
invert() @ "xs_invert";
}
The function is invoked as follows:
np.invert();
np.update();
Here's the corresponding C function:
void xs_invert(xsMachine *the) {
uint8_t *buffer = xsGetHostData(xsThis);
int byteLength = xsToInteger(xsGet(xsThis, xsID_byteLength));
while (byteLength--) {
*buffer = ~(*buffer);
buffer++;
}
}
Note: Native code is fast. Often, JavaScript is fast enough. Because JavaScript is also easier to correctly implement and more portable, it is a good idea to try a JavaScript implementation first to see if it meets the performance goals.
API design
The design of the Moddable NeoPixel class went through a few iterations before arriving at the API shown above. Since the implementation uses the NeoPixel driver from MicroPython the earliest designs reflected the MicroPython class.
Note: This section describes different choices made in the Moddable JavaScript and MicroPython NeoPixel APIs, and explains the design motivation for the JavaScript API. It is not intended as a critique of the MicroPython API which works well, served as an excellent learning tool for our work, and even provides the driver layer for the Moddable implementation.
Consistent with the principles of the Extensible Web Manifesto, this NeoPixel API for JavaScript aims to be as small as practical. The focus of the low level APIs should be to provide an efficient encapsulation of core capabilities in a way that can be extended to support specific situations, needs, and programming styles. This keeps the API as small as practical which, in turn, minimizes the implementation footprint, an important consideration on resource constrained devices. The class is designed to be subclassed to add additional capabilities. It is free of any convenience functions (except perhaps makeHSB).
The MicroPython API constructor has a field to set the number of color components but not the order of the components. The order is configured by making an additional function call after instantiating the object. In the JavaScript API, the order is an optional property in the dictionary and the length of the order string determines whether there are 3 or 4 color components.
The MicroPython API does not have makeRGB or makeHSB methods. Instead it has several setPixel methods (set, setWhite, setHSB, setHSBInt) which have color component arguments in a canonical (fixed) order. This is a little easier for client programmers but at the expense of performance because each time a color is passed to the NeoPixel API it must be converted to the strand's native pixel order. This takes time for each pixel of each step of an animation. The JavaScript API provides makeRGB and makeHSB so colors can be reordered once and then set directly.
The ability to use the NeoPixel instance as an ArrayBuffer (or, more precisely, host buffer) was a late addition to the JavaScript API. This was added primarily to allow native code to control the pixels. Most JavaScript APIs are designed with the expectation that they will only be used by other JavaScript code. With the Moddable SDK's focus on embedded development, often a class will contain native methods so it is natural to consider this in the design. Making the instance available as an ArrayBuffer gives scripts the ability to use TypedArray operations on the pixels which is more natural in some cases, and perhaps more efficient too.
Advanced configuration
The NeoPixel constructor accepts a dictionary to configure the NeoPixel strand. Here's the configuration for the NeoPixels on the M5Stack Fire board:
let np = new NeoPixel({length: 10, pin: 15, order: "GRB"});
It is a little absurd that every developer building JavaScript software on the M5Stack Fire should need to configure these parameters. This ability to set the configuration at runtime is useful during development but adds complexity to users of a particular device where hardware on each unit shares the same configuration. It is preferable to simply write:
let np = new NeoPixel;
The defines block is a feature of the manifest used to build applications in the Moddable SDK which allows native C language #defines to be set. The NeoPixel class uses these defines to fix constructor properties at build time. The manifest for the M5Stack Fire firmware can configure the defines block as follows:
"defines": {
"neopixel": {
"length": 10,
"pin": 15,
"order": "GRB",
}
}
Doing this eliminates the need to pass the corresponding properties to the NeoPixel constructor. It also makes the native code a little smaller and faster by removing the code which checks for the corresponding properties in the dictionary.
#ifdef MODDEF_NEOPIXEL_LENGTH
length = MODDEF_NEOPIXEL_LENGTH
#else
xsmcGet(xsVar(0), xsArg(0), xsID_length);
length = xsmcToInteger(xsVar(0));
if (!length)
xsRangeError("no pixels");
#endif
This build-time configuration, in turn, has a security advantage for real products. Imagine a consumer product that incorporates NeoPixels. The forward-thinking manufacturer also incorporates the ability for the consumer to install apps written in JavaScript on the device. If the NeoPixel API allows the JavaScript code to configure the pin property it can be used to write to any GPIO pin on the device. If the device is a door lock which is opened by setting the GPIO to a high or low, or even transmitting in a particular pattern, it may be possible to subvert the NeoPixel object to unlock the door. By fixing the value of the pin property at build time, the script loses the ability to select the pin thereby eliminating the potential security hole.
This approach to limiting configurability of hardware components is found in the W3C Sensor work as well. The Generic Sensor API, as well as the various specific sensor classes like Proximity, provides no means to configure the connections of the hardware, with the assumption that the underlying platform (e.g. browser and/or operating system) have already properly configured the driver. The difference here is that, in some circumstances, it is reasonable and convenient for embedded development to provide such configurability.
Conclusion
Integrating support for new capability into an embedded development environment involves providing a small, but still useful, set of functionality though a clear API that performs well and anticipates extensibility. The NeoPixel class is an example of a capability that takes these factors into account.
To try out the NeoPixel module yourself, use an ESP32 board with integrated NeoPixels such as the M5Spark Fire or Lily Go TAudio or combine an ESP32 board (like the NodeMCU ESP32) with a NeoPixel board from AdaFruit or Sparkfun. The Moddable SDK contains a NeoPixel example to get you started.