Building BLE Keyboards & Media Controllers with the Moddable SDK
With just a little software, you can quickly turn an inexpensive microcontroller into a powerful custom remote control for many of the devices you already own. Devices like televisions, media players, mobile phones, and computers support external keyboards and remote controls connected using Bluetooth Low Energy (BLE). Many microcontrollers, including the ESP32 found in Moddable Two, also support BLE. By implementing keyboard or remote control communication on the microcontroller, you can send commands to control any device. You can pair that with a custom touch screen user interface to create exactly the experience you want.
This article describes a new contribution to the Moddable SDK that demonstrates how to build keyboards and media controllers using BLEs's Human Interface Device (HID) profile. A BLE HID device is a peripheral that accepts user input and communicates it to another (typically more capable) device called a BLE Central. There are many types of HID input devices, but the focus of this article is keyboards and media controllers.
The Moddable SDK now includes a new BLE HID Host project that makes it easy to create new keyboards and media controllers. There are also four new examples that demonstrate the kinds of exciting and useful applications that can be built on the BLE HID Host:
Running the Examples
These four examples are all available in the Moddable SDK. All four examples share a common project, the BLE HID Host which contains the RTOS, XS JavaScript engine, BLE modules, and more. You'll first build and install the BLE HID Host. Each example is a relatively small mod (a bundle of JavaScript code and graphical assets) that is installed on top of the BLE HID Host. Because mods are small, they install very quickly which is convenient for development.
To deploy the BLE HID host to your device, use mcconfig with the platform specifier for your target device. The following shows how to build and install on Moddable Two:
cd $MODDABLE/contributed/ble/hid-peripheral/host
mcconfig -d -m -p esp32/moddable_two
Once installed, the BLE HID Host waits for a mod to be installed. To install a mod, use mcrun. For example, to try out the Media Controller mod on Moddable Two:
cd $MODDABLE/contributed/ble/hid-peripheral/mod-media
mcrun -d -m -p esp32/moddable_two
Because the BLE HID host takes care of many details, all of these examples can be run and experimented with without having to learn about the underlying BLE HID technology. For details about how the BLE HID host works and more about HID devices, see the BLE HID Host documentation.
The user interfaces of all of these examples are created using the Piu user interface framework. To learn more about Piu, see the Piu documentation and examples.
Pairing with a Computer or Television
After installing a mod, you need to pair your Moddable Two with your BLE-capable computer or television. This is the process on macOS Monterey:
This is the pairing process on Windows 11:
Photoshop Tools Palette
The Photoshop Tools Palette mod lets you quickly switch between editing tools in Photoshop using the Moddable Two touchscreen. This video shows Moddable Two paired with a MacBook to control Photoshop.
This mod works by sending single keystrokes to the connected computer. But there's no reason it couldn't do more: entire macros or sentences could be transmitted using the BLE HID Host. Many different applications, games, and tools could be controlled this way. Start from the Photoshop Tools Palette example and see what you can build!
Media Controller
The Media Controller mod lets you control media playback on your computer or television with buttons for play/pause, previous/next track, and volume up/down. The following image shows the Media Controller paired with a Windows 11 laptop to control Spotify.
This mod was designed by the Moddable team to be elegant and general-purpose. But, you could build a more specific UI design made for interacting with a specific media player. Start from the Media Controller example and give it a try!
Keyboard
The Keyboard mod presents the Moddable Expanding Keyboard for text entry. While perhaps of limited utility on a computer, this app can be used paired to a television to search for content on YouTube or other services.
The following image shows Moddable Two paired with a Samsung television running the TIZEN operating system.
There is considerable room for creativity with both this example and the Combination Keyboard & Media Controller that follows. Where the Photoshop Tools Palette and Media Controller examples are useful, fully-realized product concepts, the final two examples are more of a starting point for your design. Many modes of text and media control input are possible on a touch screen. It's even possible to use the other hardware capabilities of the Moddable Two to build a keyboard or media controller using external buttons or sensors. What applications can you imagine controlling with a custom hardware design?
Combination Keyboard & Media Controller
Finally, the Combination Keyboard & Media Controller mod combines the expanding keyboard with a very basic set of media controls. This is intended to demonstrate how to build a mod that blends the keyboard and media functionalities of the BLE HID Host.
Exploring Further
HID is particularly flexible because it allows an input device to communicate its own input capabilities and how it will report changes of state to the BLE Central in a standardized way. Major operating systems like Windows, macOS, Android, iOS, and Linux all have built-in support for interfacing with HID Keyboards and Media Controllers. This capability is also becoming common on modern media systems, televisions, game consoles, and more. This arrangement allows for the creation of infinite permutations of input device types that will all "just work" when connected to a computer.
Support for BLE in the Moddable SDK is remarkably powerful. Our blog post Bluetooth Low Energy Support Now Available in Moddable SDK introduces many of the key concepts.
- Develop using a high-level JavaScript API that is considerably easier to use than the underlying native APIs.
- Use JSON files to describe GATT services rather than building them in code, simplifying device definition.
- A single JavaScript application built on the Moddable SDK is portable across several unrelated native BLE stacks.
If you want to learn more about BLE and the Moddable SDK, a great starting point is the book IoT Development for ESP32 and ESP8266 with JavaScript which has a chapter dedicated to BLE.
Conclusion
The BLE HID Keyboard and Media Controller specifications are a flexible platform on top of which countless unique input peripherals can be built. The examples now included with the Moddable SDK only begin to scratch the surface, but are a compelling starting point. Indeed, multiple members of the Moddable team have had dedicated Moddable Media Controllers on their desks in recent weeks!
Try out these examples paired to your own PCs and media devices. If you see room for improvement or want a different type of user experience on top of the BLE HID Host, check out the BLE HID Host documentation for instructions on how to build your own mod.
Do you have ideas for additional input capabilities that should be included in the BLE HID Host? Reach out to us on Gitter or GitHub Discussions. The BLE HID Host was designed to encourage exploration of new kinds of inputs. We'd love to hear from you!