MIThril project overview

MIThril, a borglab production. Richard W. DeVaul, Jonathan Gips, Michael Sung, Sandy Pentland
MIThril home

MIThril FAQ

MIThril wiki

Anduin window manager

MIThril Code

MIThril
hardware design

MIThril
project overview

The MIThril vision

MIThril diagrams

MIThril people

MIThril photos

Memory Glasses

MIThril Enchantment Software

MIThril
Real-Time Contxt Engine

MIThril IAP

Wearables Home Page

MIThril is a next-generation wearables research platform developed by researchers at the MIT Media Lab. The goal of the MIThril project is the development and prototyping of new techniques of human-computer interaction for body-worn applications. Through the application of human factors, machine learning, hardware engineering, and software engineering, the MIThril team is constructing a new kind of computing environment and developing prototype applications for health, communications, and just-in-time information delivery.

The MIThril hardware platform combines body-worn computation, sensing, and networking in a clothing-integrated design. The MIThril software platform is a combination of user interface elements and machine learning tools built on the Linux operating system.

This document provides a brief overview of some of the major components of the MIThril system; It is necessarily incomplete and somewhat out-of-date. If you have questions that are not answered by this document or the MIThril FAQ, contact the MIThril team at MIThril@media.mit.edu.

MIThril hardware may be broken down into three classes of components: computing cores, networking and body bus, and sensors and peripherals. MIThril software may be broken down four categories: OS, HCI components, machine learning/classification tools, and prototype applications.

Hardware

MIThril computing cores
The MIThril hardware platform uses one or more low-power computing cores. There are four types of computing cores presently in use; Two of these are Linux-based single-board computers, one is a special-purpose wireless bridge, and one is a light-weight microcontroller-based data-gathering unit (the SAK board.) A fifth core, the WearARM, is still in development - see below for details.

The Linux cores provide substantial on-body computation, signal processing/classification, and networking resources to the user. They communicate with each other and the outside world via on-body 10 Mbit Ethernet. The wireless bridge provides a high-bandwidth connection to off-body resources. The SAK board provides a self-contained, inexpensive light-weight data acquisition system that is compatible with the MIThril body bus.

BSEV The Linux-based BSEV core combines the Brightstar Engineering ipEngine-1 (based on the MPC823 and an Altera FPGA) with the MIThril BSE dock. The BSE dock provides the hardware driver for the MicroOptical QVGA clip-on head-mounted display and the MIThril body network and body bus connectors, as well as separate USB and serial port connections.
CerfBoard The Linux-based Intrinsyc CerfBoard is a low-power high-performance SA 1110 device, similar in performance to the Compaq iPaq. It provides 10 Mbit Ethernet and a CF (Compact Flash) interface. We modify the CerfBoard slightly to run off of power provided by the MIThril body network. The CerfBoard, combined with a 1GB IBM Microdrive acts as the on-body file server and primary computational resource.
802.11 bridge The wireless bridge is a repackaged Lucent product (the ORiNOCO/EC) that is connected directly to the body network via 10 Mbit Ethernet. This is both large and power-hungry, and we are working to replace the Lucent bridge with another CerfBoard and the Symbol 24 Compact Flash 802.11 card.
SAK The SAK boardSAK board is a microcontroller-based data acquisition platform suitable for a wide range of low-bandwidth biomedical and environmental/user sensing applications. Developed in collaboration with Vadim Gerasimov, the general-purpose SAK "motherboard" is customized for particular applications through specially designed daughter boards, which provide sensing and interaction hardware. The SAK board is also able to talk to MIThril body-bus peripherals via the I2C protocol.
WearARM We are collaborating with the ETH Wearable Computing Laboratory, (part of the Electronics Laboratory (IfE) of the Electrical Engineering Department at the Swiss Federal Institute of Technology) to develop the WearARM modular low-power wearable computing core, which is based on the StrongARM processor. The WearARM consolidates the functionality of the two existing Linux cores and the wireless bridge, provides a full VGA head-mount display driver, and employs a modular design with components broken out on flex connectors for easy integration into clothing-based packaging. As of late January 2002, the WearARM is nearing completion. It successfully runs Linux, talks to the wireless interface and provides other standard functionality. We are waiting on the completion of the HMD driver.
MIThril network and body bus

MIThril employs two methods of getting information around the body: the MIThril Body Network and the MIThril Body Bus. The Body Network connects the Linux-based computing cores to each other and the wireless bridge. The MIThril body bus connects peripherals and sensors (devices which do not require Ethernet networking) to the BSEV core. The MIThril Body Network and MIThril Body Bus simplify the physical on-body networking problem by providing a reliable single-cable power/data connection between each device on the body.

  • MIThril Body Network
    The MIThril Body Network is the Ethernet/power network which connects the MIThril cores, providing peer-to-peer TCP/IP networking and power. The current design employs standard 4-twisted pair Category 5 Ethernet cable and a modified Ethernet hub to distribute both power and data.

  • MIThril Body Bus
    The MIThril body bus is a branching single-cable power/data network design that provides regulated 5V and unregulated 12V power, USB and I2C through a 10 wire cable. The USB and I2C protocols were selected to provide maximal compatibility with current COTS (cheap, off-the-shelf) USB devices like cameras and microphones, and hackability, since it is relatively easy to build simple microcontroller-based I2C sensors and peripherals. The connectors and cables were chosen to be mechanically and electrically stable, robust, and locking. All components are flat and small, making it easy to integrate the MIThril body bus with existing clothing or new designs.

    MIThril body bus junctions are small, flat hubs that act as 3-way branches for the bus and tie the bus to the physical garment substrate. At present the hubs are passive and do not support USB directly, although USB devices may be plugged into the BSEV core itself. We have a nearly complete design for a USB-enabled body-bus junction, but it is not yet tested.

MIThril sensors and peripherals

One important goal of the MIThril architecture is to make it as easy as possible to connect a wide range of sensor and I/O devices to the system. Some of these sensors and peripherals are off-the-shelf USB devices, such as CCD cameras and audio input/output devices. Some are custom sensors that speak I2C or other experimental protocols. Once USB enabled body bus hubs are working, all of these sensors and peripherals will plug directly into the body bus, providing a single, unified means of connecting sensor and peripherals to MIThril cores.

Working MIThril body-bus sensors include:

  • High-precision three-axis accelerometer (custom I2C device).
  • IR Active tag reader (custom I2C device).
  • Integrated accelerometer/IR active tag reader/microphone/headphone audio board for SAK core.
  • USB microphone and headphone driver.
  • USB CCD camera
  • .

The USB and I2C protocols span a wide range of useful sensors and peripherals, but the 12Mbit bandwidth constraint of USB and 400Kbit constraint of I2C do impose some limitations on the type and number of devices which may share this bus. Likewise, debugging console connections should not be routed over the body bus for practical reasons. Thus, it makes sense to connect some devices directly to a MIThril core. These devices include:

  • A re-cabled Twiddler One chording keyboard and modified Palm folding keyboard, which are hot-swappable through an exterior plug and connect to the same serial port on the Brightstar.
  • The debugging console serial ports on the Brightstar and CerfBoard.
  • Earthmate GPS receiver (serial)
  • the MicroOptical Clip-on, which is based on the Kopin CyberDisplay 320 Color chip, is one such peripheral. The Kopin display signal is produced through a combination of the MPC823's on-board LCD driver, timing code for the Brightstar's FPGA, and some additional D to A and analog circuitry on a companion board which plugs into the Brightstar IPEngine.

MIThril software

MIThril software includes FPGA code, Linux OS/driver code, UI code, sensing/signal processing/statistical machine learning code, and prototype applications.

The current FPGA core for the BSEV core generates timing signals for the HMD and provides great flexibility and control over the display. For instance, a combination of the FPGA core and Linux driver allows us to switch from "standard" field-sequential color to high-brightness 180Hz gray scale to low-power monochrome (red "gray scale") display modes on the fly.

We have functioning Linux distributions for both of our current Linux computing cores (Debian on the StrongARM, a Hard Hat Linux derivative on the Brightstar).

We have functioning Linux I2C and USB master drivers for the MPC823, which we will publish in our CVS repository shortly. We also have a field-sequential-color frame-buffer driver for the BSEV core, allowing us to use 24-bit color and X-windows.

We have made substantial progress in implementing the Enchantment user interface, as described in the Enchantment Whitepaper. This includes the Anduin window manager, which provides a mouseless interface to X applications (Anduin is in our MIThril Code CVS repository but not documented elsewhere yet) and the Enchantment IPC whiteboard context server (also in the CVS repository, but not yet documented elsewhere).

The MIThril Inference Engine provides a set of tools for applying statistical machine learning techniques to classifying sensor data in real-time. The inference engine works with the Enchantment IPC whiteboard to provide real-time classification and regression to context-aware applications. (This code can also be found in the CVS repository). We have successfully used the MIThril Inference Engine to implement a sophisticated activity classification system based on accelerometer data; see the Motion Classification Project Paper (gzip compressed postscript) for more details.

At present we are working on three major applications; the Memory Glasses project, the Context-Aware Cell Phone Project and a biomedical patient functional assessment project utilizing the SAK platform (a collaboration with researchers at the Rochester Center for Future Health).

Last updated Thu Jan 31 05:13:16 EST 2002