Getting the Most from ePaper Displays

One of the defining features of the Moddable SDK is its outstanding support for displays in IoT products. Much of that work has focused on LCD touch screens, but ePaper displays have also been supported from the beginning. Recent work has brought broader and deeper support for ePaper displays, making the Moddable SDK the best choice for building devices with ePaper displays.

This article introduces the new APIs, device support, and programming techniques used to make working with ePaper displays as simple and consistent as possible.

Characteristics of ePaper Displays

ePaper displays are a unique display technology that can maintain the image on a display without using any power. This makes them ideal for signs that are displayed for an extended period of time, such as retail price tags.

ePaper displays also don't usually have a built-in backlight, so they don't glow in a dark room like an LCD. Reading an ePaper display is more like reading a piece of paper, which is why they are often used in eBook products. Direct sunlight, which makes an LCD more difficult to read, actually makes ePaper displays easier to read.

ePaper displays are available from many different manufacturers, the most well-known being eInk. Displays are available in a variety of sizes, resolutions, and with support for different numbers of colors. In smaller sizes they can be quite inexpensive, though larger size are often remarkably expensive.

ePaper displays use unique technology to achieve their low power and high contrast. This creates challenges for products with an ePaper display:

  • They are only able to display a limited number of colors, often just black and white.
  • They take a relatively long time to update, making the highest practical frame rate around four frames per second.
  • They also can show a faint ghost of the previous image, which can only be erased by periodically refreshing the full screen by flashing all pixels between black and white.

The Moddable SDK has features to take full advantage of the benefits of ePaper and tools to help you manage the challenges of ePaper in the best way possible for your product.

Hardware Products

The Moddable SDK now supports three very different development boards that incorporate ePaper displays. Each hardware product uses a different ePaper display with a distinct display controller. These products are all great starting points to learn about building products with ePaper displays with the Moddable SDK, even if you will eventually select a different ePaper display for your product.

Product Size Resolution Colors Display Controller CPU
Moddable Three 2.13" 250 x 122 Black & White DESTM32-S ESP8266
M5Core Ink 1.54" 200 x 200 Black & White GDEW0154M09 ESP32
M5 Paper 4.7" 960 x 540 16 grays IT8951 ESP32

DIY

You aren't limited to using just these hardware products. You can create your own display driver to integrate with the Moddable SDK (more on that below) or adapt an existing driver to work with other compatible displays. The prolific Shinya Ishikawa (@meganetaaan on Twitter) reconfigured the DESTM32-S used in Moddable Three for use with a much larger display to create a photo viewer. Ishikawa-san used a display that displays black, white, and red pixels, making very clever use of the red pixels.

Examples

The unique display properties of ePaper displays require a different kind of user interface design. That's because designs for an LCD often don't make sense or don't perform well on an ePaper display. For instance, Moddable uses the piu/balls application to demonstrate a new display driver, but its high frame rate and use of color make it a poor choice for ePaper displays.

The Moddable SDK contains several new example applications designed specifically for ePaper displays. All of these examples were built by Lizzie Prader, with the exception of the Clock.

Photos

The ePaper Photos example displays a slideshow of photographs. The example runs on Moddable Three, M5Core Ink, and M5 Paper – and looks great on each of them.

Given that these displays have very different sizes, aspect ratios, and color capabilities, that's quite a feat. Here are some highlights of how it achieves that:

  • To provide optimal photos for each screen size, ePaper Photos includes photos cropped and scaled for each display size. Only the version of the image that matches the display is downloaded to the device. The selection is performed by the application manifest through entries for each supported device.
  • Two of the hardware devices support only black and white pixels. Ordinarily, that's not enough color to display a photo. But the photos look great on those because they are automatically dithered by the ePaper drivers in the Moddable SDK. This gives the appearance of several levels of gray, even with only black and white pixels. The M5 Paper display is able to show 4-bit gray images which is enough colors to eliminate the need for dithering. For each device, the ePaper Photos app delivers the best quality image from the display.
  • A photo viewer designed for an LCD uses transitions to move from one photo to the next (and the Moddable SDK has an collection of built-in transitions that could be used for that). Because of the slow update rate of ePaper displays, the ePaper Photos app does not attempt to apply transitions.

Moddable Three Window Display

The Moddable Three Window Display was created to show off techniques for creating attractive ePaper displays – using non-rectangular shapes and animations designed to look good at low frame rates.

The Moddable Three Window Display example is a tour-de-force of design that shows how some clever thinking and asset design can achieve unexpected results. By contrast, the code that implements it is a straightforward Piu application.

The designer of the Moddable Three Window Display example, Moddable's Chris Krueger, shared the following notes on how he approached the design:

  • The ePaper display on Moddable Three is monochrome with a fairly low frame rate but reasonably fast refresh rate. We worked with those hardware characteristics when designing this animation.
  • The screen refresh has a visual blending effect that can be used to advantage. By moving one image off screen with another following in behind with a dithered blending segment the low frame rate is less of a visual liability.
  • Since the display is only white or black the best changes are from largely white images to black background images. This makes the screen change more deliberate and less visually confusing.
  • Even though the Moddable SDK can do the graphic dithering conversion of color graphics to 1-bit images we did that conversion prior to building the application so we could control the dithering process and remove stray pixels that showed up in the process that were not desired.

Clock (Battery-powered)

The ePaper Clock example draws a basic digital clock on the ePaper display that updates once a minute. The Clock only runs on M5 Paper and M5Core Ink because they are battery powered. What is special about the Clock is its power management techniques, including one to eliminate drawing flicker on the M5Core Ink.

After drawing the current time, the Clock tells the device to turn off until the minute changes. The ESP32, display controller, and all other components are disconnected from power to preserve battery power. In this state, only the Real-Time Clock component (RTC) is using battery power (and it uses very little). The RTC continues to operate because it is responsible for restarting the ESP32 to draw the next time.

On the M5 Paper, when the Clock wakes up there is a flicker on the screen as the display controller refreshes its state. This is typical on most ePaper displays. But, on the M5Core Ink, there is no flicker when the device wakes up and refreshes the screen. Why not?

The GDEW0154M09 display controller in the M5Core Ink paper contains two frame buffers, one for the frame to draw next and one for the previous frame. It uses these two frame buffers to optimize how the ePaper is redrawn so that it can eliminate flicker. The problem is that both frame buffers are lost when power is turned off. The Moddable SDK GDEW0154M09 display driver solves this by allowing the script to provide both the current and previous frames. If the previous frame isn't provided, there will be a flicker. But, if the previous frame is provided, the driver can instruct the display controller to achieve a flicker-free update. Not only does this look better, but it uses less power. As a result, the ePaper Clock on the M5Core Ink hardware runs for at least 10 days on battery, updating once a minute. That's over 14,400 full boots of the ESP32 and Moddable SDK plus screen updates. Not bad.

Travel Time

The Travel Time example displays the travel time between your home and work. It uses the Google Maps Web API to determine the current travel time, taking into account current traffic conditions. When running on the M5 Paper, the Travel Time app uses the touch screen to swap the travel between home to work and work to home.

The Travel Time example shows how to bring together a web service (here Google Maps) with an ePaper display.

The Travel Time example runs on all three hardware devices. On ESP32-powered devices, the application is written using the Piu user interface framework, which has layout features that make it easy to adapt the UI to different screen sizes.

On the larger M5 Paper display, Travel Time also displays the current date and time along with the current temperature taken from the built-in thermometer.

On Moddable Three, memory was a little tight. This is mostly because the Google Maps Web API returns a large JSON response. Instead of leaving Moddable Three unsupported, there's a different version of the example that makes room for the JSON response by using the Poco graphics renderer directly instead of using the Piu user interface framework. The Google Maps Web API code remains the same with the ESP32 version, but the UI is rendered using a lighter-weight, lower-level API. If you are interested in comparing how the same app is structured with and without Piu, this is a great example to take a look at.

Flash Cards

The Flash Cards example takes advantage of the large M5 Paper ePaper display to show a sequence of flash cards. The big screen leaves plenty of space to show the question and answer on the same screen. The answer is displayed when tapped. A swipe left or swipe right moves to the previous or next card.

When the Flash Card example reveals the answer, there's no flicker. The example uses a feature of the display driver to control how the update is applied. The display controller supports eight different update modes, each optimized for updating a particular kind of image. Here the application is drawing white text on a black background, where the A2 update mode gives the best result. The display driver's configure API allows an application to control the update mode (more on that below).

The Flash Card example is pre-loaded with a series of questions about the Moddable SDK. Do you know all the answers?

APIs

Building applications for ePaper displays with the Moddable SDK uses the same APIs as building for an LCD. That's great news. The experience you have using the Moddable SDK with an LCD applies to an ePaper display. All that knowledge you have about the Moddable SDK, perhaps from reading IoT Development for ESP32 and ESP8266 with JavaScript, works with ePaper too.

Products with ePaper displays can use the Piu application framework to build complex, scalable products with responsive layouts that adapt to different display sizes. They can also use the Commodetto graphics framework and Poco renderer to have complete control over graphics.

While the APIs to build user interfaces are the same regardless of what device target you are using, there are differences that need to be taken into consideration. The most important is designing the user interface to take advantage of ePaper's benefits and to avoid its challenges.

The "Display API Extensions" section below describes APIs provided by the display drivers for ePaper.

Dithering

Many ePaper displays have only black and white pixels, making dithering essential. The Moddable SDK has long had dithering support in some display drivers. It now provides a module to dither to 1-bit pixels. The API supports both Atkinson and Burkes dithering algorithms, and can be extended to support others.

To use the module, first import it and then initialize it by providing the line width and dithering algorithm to use:

import Dither from `commodetto/Dither`

let dither = new Dither({
    width: 200,
    algorithm: "atkinson"
});

To dither a bitmap, first call reset() to reinitialize the internal error accumulation buffers. Then call send() one or more times, passing one or more complete scan lines.

dither.reset();

dither.send(numberOfScanLines,
    src, srcOffset,
    dst, dstOffset);

The GDEW0154M09 display driver uses the Commodetto Dither module to implement its dithering support. See its source code for a complete example.

Display API Extensions

ePaper displays are different from an LCD and so, naturally, require some additional APIs. All ePaper displays are the same in some ways – for example, each ePaper display must deal with ghosting that occurs in some update modes. On the other hand, many ePaper display drivers provide differentiating features – for example, update modes tuned to different kinds of content – that applications need to access to take full advantage of the hardware.

Refresh

All ePaper display controllers have some variation of a refresh operation, which quickly switches all pixels between black and white to clear the previous image and any ghosting. This is implemented by the refresh method. The Moddable SDK puts the display driver instance on the screen global variable, so projects can refresh the screen at any time:

screen.refresh();

By default, the display drivers perform a reset prior to the first update. On the GDEW0154M09, this refresh is not necessary if the application can provide the previous display image. This capability is used in the ePaper Clock example to eliminate all flicker from periodic updates. To disable the initial refresh, call configure with refresh set to false:

screen.configure({refresh: false});

Dither

Both the GDEW0154M09 and DESTM32-S drivers support dithering. On both, those can be turned on and off using the dither property:

screen.configure({dither: true});

The GDEW0154M09 also allows configuration of the dither algorithm to use, by naming it with a string.

screen.configure({dither: "atkinson"});
screen.configure({dither: "burkes"});

Draw Previous

When an application wants to update the previous image on the GDEW0154M09, it first calls

screen.configure({previous: true});

It then draws the previous frame using Piu or Poco. Finally, it disables drawing to the previous frame to return to drawing to the current frame:

screen.configure({previous: false});

Then it can draw as usual.

Filter

The M5 Paper supports 16 levels of gray, which is just about good enough to show photo-realistic images without significant artifacts. However, the contrast of the display isn't optimal for all photos. This can be compensated for in software by applying a filter. The IT8951 ePaper display driver in the Moddable SDK adds a suite of filters, some to improve the quality of images and others just for fun.

Here's examples of setting each available filter:

screen.configure({filter: "negative"});
screen.configure({filter: "posterize"});
screen.configure({filter: "monochrome"});
screen.configure({filter: "bright"});
screen.configure({filter: "contrast"});
screen.configure({filter: "none"});

Once the filter is set, it is applied to all images drawn afterwards.

Update Mode

The IT8951 driver in the M5 Paper supports several different update modes, each optimized for displaying a particular kind of content. Finding the optimal mode requires some experimentation, but it can make a real difference so it is worth exploring. The modes are (lightly) documented in the driver source code.

Here's examples of setting each available mode:

screen.configure({updateMode: "DU"});    // Monochrome menu, text input, and touch screen input
screen.configure({updateMode: "GC16"});  // High quality images
screen.configure({updateMode: "GL16"});  // Text with white background
screen.configure({updateMode: "GLR16"}); // Text with white background
screen.configure({updateMode: "GLD16"}); // Text and graphics with white background
screen.configure({updateMode: "DU4"});   // Fast page flipping at reduced contrast
screen.configure({updateMode: "A2"});    // Anti-aliased text in menus / touch and screen input

Implementing Drivers

The three ePaper drivers described in this article all implement the same Moddable SDK PixelsOut API. This allows them to integrate directly with the Piu user interface framework and Poco graphics engine.

The Moddable Three (DEST32M-S) is the first ePaper driver in the Moddable SDK. It is implemented entirely in C, as were most of the early drivers. Over time, it has become practical to implement more drivers in JavaScript – without sacrificing performance. The adoption of the Ecma-419 IO Class Pattern in the Moddable SDK has helped with this significantly.

The JavaScript drivers are easier to write and far more portable. The GDEW0154M09 driver for the M5Core Ink is implemented entirely in JavaScript. It imports the native Commodetto dither module to perform efficient dithering. The IT8951 driver for M5 Paper is entirely implemented in JavaScript, except for a native function to apply filters. While the filters could be applied in JavaScript (and originally were) there is a significant performance boost for doing this in native code.

That's an important point. When working in the Moddable SDK, the preference is to work in JavaScript whenever practical in order to benefit from the language's simplicity, portability, accessibility, and security. But, there are times when C or C++ is a better choice. A common example is when raw performance matters. The Moddable SDK lets implementations invoke native code wherever they want. That blending of JavaScript and C makes programmers more efficient and products more robust.

Acknowledgement

The IT8951 and GDEW0154M09 ePaper display drivers in the Moddable SDK began as near literal JavaScript translations of the C++ Arduino implementation provided by M5. Once they were working, the code was significantly reorganized, enhanced, and optimized for use in the Moddable SDK. The Moddable SDK implementations would have been much more difficult to create without the M5 code as a starting point. Many thanks to the M5 folks for making their work freely available.

Memory Use

The three ePaper drivers not only deliver excellent visual results, they are also fast and memory efficient. That's important because microcontrollers have limited performance and memory. The ESP32 has about 512 KB of RAM, though once the ESP-IDF, Wi-Fi and BLE are in use, that drops to about 200 KB. An ESP8266 has just 80 KB of RAM, with about 45 KB available. Any memory used by the display driver is memory that is unavailable for your product.

A typical ePaper driver, outside the Moddable SDK, requires buffering the entire image in RAM to perform a full screen update. Because rendering to 1-bit pixels is challenging, those are often 4- or 8-bit pixels. To allow full screen dithering, the Moddable Three renders at 8-bits per pixel and then dithers to 1-bit. A full-screen 8-bit screen buffer would use about 30 KB of RAM, or 66% of all free memory. But the Moddable Three ePaper driver uses just a few KB of RAM. The Arduino implementation of the M5 Paper driver uses an 8-bit offscreen buffer of the full screen, which requires 518 KB of memory, which is more than the ESP32 has built-in. Consequently, the M5 Paper includes an external RAM chip. While that's convenient, it adds cost and increases the energy use on the battery. The Moddable SDK ePaper driver for the ePaper display uses just few KB of memory, and runs comfortably without any external memory. How is that possible?

The main reason isn't the display driver, but the graphics engine. The graphics engine in the Moddable SDK – the Poco renderer – is able to render the display in bands, rather than rendering the full frame at once. This allows Poco to operate with a buffer of just a handful of scan lines. As each band is rendered, it is sent to the display and then the next band is rendered. The rendering and transmission time are about the same as operating on a full screen buffer, but the memory used is just a tiny fraction.

Implementing an ePaper Display Driver

It is entirely possible to add another ePaper display driver to the Moddable SDK. The basic driver code is just a few hundred lines of code. It is reasonably straightforward to take a native implementation (for example, an Arduino driver) and convert it to JavaScript. Once the driver is implemented, it connects automatically to Poco for graphics and Piu for an application framework.

ePaper displays with differentiating features can make them available through the display driver using the two techniques shown in the Display API Extensions section above:

  1. Add new methods to its display class. This is a good choice for new actions, like refresh().
  2. Add a new property to the configure method. This is a good choice for a value that changes the how the display driver operates, such as dithering or update modes.

Note: The Moddable SDK's PixelsOut API is very close to, but not the same as, the Display Class Pattern API defined in the ECMAScript Embedded Systems API standard. They will be updated to conform to Ecma-419 as the Moddable SDK completes its transition to Ecma-419 standard APIs.

Conclusion

ePaper displays have benefits over LCDs for IoT products looking to reduce energy consumption, eliminate screen glow, or be viewable in sunlight. Many ePaper displays are cost-competitive with LCDs.

The challenge – until now – is that software support for ePaper displays has been minimal and uneven. While there is often sample code for a driver provided, production quality drivers are rare and graphics engines are often ad-hoc or quick adaptations. The Moddable SDK changes that by connecting ePaper displays to the full power of the Commodetto graphics framework, the Poco renderer, the Piu user interface framework, the XS JavaScript engine, and countless other integrated features including QR Code rendering, direct PNG and JPEG display, and animated GIF rendering.

Creating products that incorporate ePaper displays has never been easier or more powerful. Get started with one of the supported hardware boards – Moddable Three (now available from Digi-Key!), M5Core Ink, or M5 Paper. Use the latest ePaper examples to start exploring how to build the software you need for your project. Once you are confident in a direction, choose a compatible display (or make an incompatible display compatible by writing the display driver).

ePaper displays have great potential for IoT products. This article may have given you some new ideas about how ePaper displays could fit into your product plans. When you start exploring, Moddable is here to help.

  • To get some first-hand experience working with ePaper displays, get one of the hardware products described above.
  • Think about how to optimize the user experience to work comfortable with the constraints of ePaper display.
  • If you run into technical problems in your explorations, reach out to us on GitHub. Post your questions to Discussions. If you've found a bug, report an Issue.
  • Once you've built a great new example (or made a cool improvement to an existing one), tell your friends and consider sharing it by opening a Pull Request.
  • If you'd like help designing or developing your product, send an email to info@moddable.com.