Antonio Gomes, Calvin Rubens, Sean Braley, and Roel Vertegaal. 2015. BitDrones: Towards Using 3D Nanocopter Displays as Interactive Self-Levitating Programmable Matter. To appear In Proceedings CHI'16 [Paper]
BitDrones is a toolbox for building interactive real-reality 3D displays that use nano quadcopters as self-levitating tangible building blocks. Our prototype is a first step towards interactive self-levitating programmable matter, in which the user interface is represented using Catomic structures. We discuss three types of BitDrones: PixelDrones, equipped with an RGB LED and a small OLED display; ShapeDrones, augmented with an acrylic mesh spun over a 3D printed frame in a larger geometric shape; and DisplayDrones, fitted with a thin-film 720p touchscreen.
BitDrones (2015): PixelDrone with dot-matrix display.
BitDrones (2015): Cube-shaped ShapeDrone with RGB LED (red)
BitDrones (2015): DisplayDrone with 720p F-OLED Display
PixelDrones can fly in a group that forms a menu structure shaped as a cone tree. Our cone trees begin with a top PixelDrone node representing the root folder (see Figure below). When touched, this expands into a submenu of PixelDrones flying underneath it in a circular arrangement. The name of the folder or file is displayed on the corresponding PixelDrone’s display. Submenus can be browsed by gentle unimanual horizontal throws of a single drone inside the submenu. This causes the entire submenu wheel to rotate with simulated momentum and friction. When a folder in a submenu is touched, a third level of submenus can be displayed. Other drones automatically make space for this layer, while drones are automatically recruited to display its contents. When a file node is touched, drones are recruited to show the object inside the file
BitDrones (2015): User opening a file in a PixelDrone cone tree.
BitDrones (2015): User interacting with group of ShapeDrones.
We present a number of unimanual and bimanual input techniques, including touch, drag, throw and resize of individual drones and compound models, as well as user interface elements such as self-levitating cone trees, 3D canvases and alert boxes. We describe application scenarios and depict future directions towards creating high-resolution self-levitating programmable matter.
BitDrones (2015): Resizing a group of ShapeDrones with a Pinch gesture.
BitDrones (2015): Placing a ShapeDrone in mid-air.
Compound objects typically consist of ShapeDrones or PixelDrones acting as sparse voxels representing some physical 3D data structure. The advantage of using a ShapeDrone is that larger, pre-shaped 3D content can be added to the canvas without requiring PixelDrones to render that shape. Resulting designs are automatically communicated to a Unity 3D engine, from where they can be texture mapped and exported, or 3D printed
BitDrones (2015): ShapeDrones hover together to form a structure.
We also developed a remote telepresence application in which users embody themselves via a remote DisplayDrone that displays a Skype videoconference. When reciprocated, this gives users the ability to interact with remote participants at eye level without requiring a robotic structure
BitDrones (2015): TelePresence through a DisplayDrone with Skype.
