Moddable SDK Improvements for ESP32 Projects

Today, we released a major update to improve our support for the popular ESP32 microcontroller. This update includes four major new features:

  • Faster, more reliable ESP32 build system
  • Upgrade to ESP-IDF v3.3.2, a Long Term Support release from Espressif
  • Lightweight NimBLE Bluetooth LE stack for ESP32
  • Execution of individual build steps — clean, build, deploy, debug — directly

We are continuously updating the build and deployment tools in the Moddable SDK to make them more useable and more useful. To help you stay up-to-date with those, we've added a new document focused on applying updates.

Fast, Native ESP32 Builds on Windows

The Moddable SDK now uses CMake to build the ESP-IDF to improve build performance and reliability. With today's update, all three host platforms — macOS, Windows, and Linux — support native builds using CMake and Espressif's idf.py utility.

The benefits of the new build system are meaningful on every platform. On macOS and Linux, the new build is slightly faster and eliminates a race condition issue in the ESP-IDF build process that required manually running clean builds twice to successfully complete the build.

The changes are most significant on Windows, where ESP32 builds were previously slow and somewhat awkward. On a 2019 Surface Laptop 3 the new build system completes a clean build in approximately one minute, while clean builds with the previous MinGW-based build system took about four and a half minutes. Further, the new build system is a true native cross-compilation on Windows, eliminating the need to manually switch back and forth between a virtualized Linux environment and the Windows command prompt during the build process.

ESP-IDF Long Term Support Release v3.3.2

Today's update to the Moddable SDK brings support for ESP-IDF v3.3.2 as the foundation for ESP32 projects. This is the current Long Term Support release of the ESP-IDF and will be supported by Espressif through February 2022. Prior to today's update, our builds used ESP-IDF version 3.2.2.

ESP-IDF v3.3.2 comes with significant improvements to system security, stability, and performance. Details are available on the ESP-IDF releases page, but highlights of the improvements between v3.2.2 and v3.3.2 include:

  • Dozens of security, stability, and performance fixes
  • Fixes for two critical Wi-Fi reliability issues
  • Major improvements to the Wi-Fi stack's SoftAP mode
  • More refined OTA support
  • A new CMake-based build system
  • NimBLE Bluetooth stack

By incorporating ESP-IDF v3.3.2 into builds with the Moddable SDK, we are bringing all of these improvements to our developers.

More Efficient Bluetooth LE with NimBLE

Today's update to the Moddable SDK adds support for the Apache NimBLE Bluetooth LE 5.1-certified open-source host on ESP32. The NimBLE stack is new to ESP-IDF v3.3.x and is recommended by Espressif for projects using only BLE (i.e. not using classic Bluetooth). Compared to the dual-mode Bluedroid stack, also included in the ESP-IDF and previously used in the Moddable SDK, NimBLE provides several benefits:

  • Smaller Flash footprint. The NimBLE stack has a smaller binary code size than Bluedroid. In a typical BLE project, for example the heart-rate-server, the firmware is about 254 KB smaller. Not only does this free-up limited firmware space for your projects, but reduces the time needed to deploy builds when developing.
  • Smaller RAM footprint. The NimBLE stack also uses less memory than Bluedroid. The same BLE heart-rate-server example, after establishing a connection, has 33 KB more system memory free when using NimBLE. As the ESP32 has only about 250 KB of RAM available to your project, this is a significant improvement.
  • Support for up to nine simultaneous connections, compared to seven with Bluedroid.
  • Fewer buffer copies. NimBLE provides a unified buffering scheme that eliminates buffer copies from the BLE stack to the Moddable SDK.
  • Faster builds. The NimBLE code builds approximately 907 ESP-IDF source files compared to 1061 ESP-IDF sources required for Bluedroid.

NimBLE is enabled by default by the Moddable SDK in ESP32 builds. Therefore, Moddable SDK developers working on ESP32 with our BLE JavaScript API automatically benefit from these enhancements without needing to change their code.

Execution of Individual Build Steps

Today's update includes an improvement to mcconfig, our command line tool that generates make files based on a manifest and then builds and launches Moddable SDK projects on microcontrollers or in the simulator.

By default, mcconfig performs a complete build, deploy, and debug sequence for the selected platform. For instance, the command mcconfig -d -m -p esp32 will build your project, flash it to the ESP32 device, and start serial2xsbug to connect the device to the xsbug debugger on your host machine.

Today's update adds a new optional flag for mcconfig: -t <target>. By selecting a target, you configure mcconfig to perform only a single step.

The options for the <target> argument are:

  • clean: removes build outputs for the project
  • build: builds the project, but does not deploy it
  • deploy: deploys a previously built project without rebuilding
  • xsbug: connect the xsbug debugger to a previously-deployed project
  • all: performs all steps (This is the default value when the -t flag is omitted)

These changes to the ESP32 build were so great to have that we applied them across our other target platforms as well. The all, clean, and build options are applicable across all target platforms (i.e. ESP8266, ESP32, Gecko, QCA4020, and the desktop simulators). The xsbug option is available for ESP8266, ESP32, and QCA4020. The deploy option is available for the ESP8266 and ESP32.

This update allows for more flexible development workflows and interoperability with other tools. For instance, a continuous integration tool could use the -t build option to rebuild a project after each change without attempting to deploy it. Developers can use the -t deploy option to switch between deploying multiple projects to a device without rebuilding. And it is often useful to restart the debugging process without redeploying by using the -t xsbug option.

Keeping Up To Date with Moddable SDK Improvements

It is important to maintain an up-to-date development environment to get security and reliability fixes and new features, including those described in this article. The Moddable SDK tools and the utilities used to build for target platforms need to be occasionally updated or rebuilt.

To help you with that process, we have added a new Keeping Up To Date guide to our documentation. Follow along with that document to update your Moddable SDK repository, rebuild the Moddable SDK tools, and install and update the utilities you will need in your build environment for the updated ESP32 build system.

Support for the older ESP32 build systems remains in the Moddable SDK for now (set the environment variable ESP32_CMAKE=0). But we hope that everyone will move up to the new build system soon to enjoy its many benefits.

Conclusion

Today's Moddable SDK update brings a pile of huge improvements to our developer experience for the wildly-popular ESP32 microcontroller.

ESP32 Moddable SDK developers now have access to:

  • Much faster builds on Windows and a consistent ESP32 build experience across host platforms
  • Espressif's Long Term Support ESP-IDF v3.3.2, which arrived with a number of critical bug fixes
  • Dramatically improved NimBLE Bluetooth LE stack
  • Individually-selectable build targets for the mcconfig build tool

We recommend that all of our developers upgrade their Moddable SDK today using our newly published Keeping Up To Date document. If you have any issues—or if you just love today's update—please reach out to us on GitHub or Twitter.