Anduin window manager
The MIThril vision
MIThril Enchantment Software
Real-Time Contxt Engine
Wearables Home Page
MIThril is a next-generation wearables research platform developed by
researchers at the MIT Media Lab. The goal of the MIThril project is
the development and prototyping of new techniques of human-computer
interaction for body-worn applications. Through the application of
human factors, machine learning, hardware engineering, and software
engineering, the MIThril team is constructing a new kind of computing
environment and developing prototype applications for health,
communications, and just-in-time information delivery.
The MIThril hardware platform combines body-worn computation, sensing,
and networking in a clothing-integrated design. The MIThril software
platform is a combination of user interface elements and machine
learning tools built on the Linux operating system.
This document provides a brief overview of some of the major
components of the MIThril system; It is necessarily incomplete and
somewhat out-of-date. If you have questions that are not answered by
this document or the MIThril FAQ, contact the
MIThril team at MIThril@media.mit.edu.
MIThril hardware may be broken down into three classes of components:
computing cores, networking and body bus, and sensors and peripherals.
MIThril software may be broken down four categories: OS, HCI
components, machine learning/classification tools, and prototype
HardwareMIThril computing cores
The MIThril hardware platform uses one or more low-power computing
cores. There are four types of computing cores presently in use;
Two of these are Linux-based single-board computers, one is a
special-purpose wireless bridge, and one is a light-weight
microcontroller-based data-gathering unit (the SAK
board.) A fifth core, the WearARM, is still in development -
see below for details.
The Linux cores provide substantial on-body computation, signal
processing/classification, and networking resources to the user.
They communicate with each other and the outside world via on-body
10 Mbit Ethernet. The wireless bridge provides a high-bandwidth
connection to off-body resources. The SAK board provides a
self-contained, inexpensive light-weight data
acquisition system that is compatible with the MIThril body bus.
The Linux-based BSEV core combines the Brightstar Engineering
ipEngine-1 (based on the MPC823 and an Altera FPGA) with the
MIThril BSE dock. The BSE dock provides the hardware driver for
the MicroOptical QVGA clip-on head-mounted display and the MIThril
body network and body bus connectors, as well as separate USB and
serial port connections.
The Linux-based Intrinsyc CerfBoard
is a low-power high-performance SA 1110 device, similar in
performance to the Compaq iPaq. It provides 10 Mbit Ethernet and
a CF (Compact Flash) interface. We modify the CerfBoard slightly
to run off of power provided by the MIThril body network. The
CerfBoard, combined with a 1GB IBM Microdrive acts as the on-body
file server and primary computational resource. |
The wireless bridge is a repackaged Lucent product (the
ORiNOCO/EC) that is connected directly to the body network via 10
Mbit Ethernet. This is both large and power-hungry, and we are
working to replace the Lucent bridge with another CerfBoard and
the Symbol 24 Compact Flash 802.11 card.
boardSAK board is a microcontroller-based data acquisition
platform suitable for a wide range of low-bandwidth biomedical and
environmental/user sensing applications. Developed in
collaboration with Vadim Gerasimov, the general-purpose SAK
"motherboard" is customized for particular applications through
specially designed daughter boards, which provide sensing and
interaction hardware. The SAK board is also able to talk to
MIThril body-bus peripherals via the I2C protocol.
We are collaborating with the ETH Wearable Computing
Laboratory, (part of the Electronics Laboratory (IfE) of
the Electrical Engineering
Department at the Swiss Federal
Institute of Technology) to develop the WearARM
modular low-power wearable computing core, which is based on the
StrongARM processor. The WearARM consolidates the functionality of
the two existing Linux cores and the wireless bridge, provides a full
VGA head-mount display driver, and employs a modular design with
components broken out on flex connectors for easy integration into
clothing-based packaging. As of late January 2002, the WearARM is
nearing completion. It successfully runs Linux, talks to the wireless
interface and provides other standard functionality. We are waiting on
the completion of the HMD driver.
|MIThril network and body bus
MIThril employs two methods of getting information around the body:
the MIThril Body Network and the MIThril Body Bus. The Body Network
connects the Linux-based computing cores to each other and the
wireless bridge. The MIThril body bus connects peripherals and
sensors (devices which do not require Ethernet networking) to the BSEV
core. The MIThril Body Network and MIThril Body Bus simplify the
physical on-body networking problem by providing a reliable
single-cable power/data connection between each device on the body.
- MIThril Body Network
The MIThril Body Network is the Ethernet/power network which connects
the MIThril cores, providing peer-to-peer TCP/IP networking and power.
The current design employs standard 4-twisted pair Category 5 Ethernet
cable and a modified Ethernet hub to distribute both power and data.
- MIThril Body Bus
The MIThril body bus is a branching single-cable power/data network
design that provides regulated 5V and unregulated 12V power, USB and
I2C through a 10 wire cable. The USB and I2C
protocols were selected to provide maximal compatibility with current
COTS (cheap, off-the-shelf) USB devices like cameras and microphones,
and hackability, since it is relatively easy to build simple
microcontroller-based I2C sensors and peripherals. The
connectors and cables were chosen to be mechanically and electrically
stable, robust, and locking. All components are flat and small,
making it easy to integrate the MIThril body bus with existing
clothing or new designs.
MIThril body bus junctions are small, flat hubs that act as 3-way
branches for the bus and tie the bus to the physical garment
substrate. At present the hubs are passive and do not support USB
directly, although USB devices may be plugged into the BSEV core
itself. We have a nearly complete design for a USB-enabled body-bus
junction, but it is not yet tested.
sensors and peripherals
One important goal of the MIThril architecture is to make it as easy
as possible to connect a wide range of sensor and I/O devices to the
system. Some of these sensors and peripherals are off-the-shelf USB
devices, such as CCD cameras and audio input/output devices. Some are
custom sensors that speak I2C or other experimental
protocols. Once USB enabled body bus hubs are working, all of these
sensors and peripherals will plug directly into the body bus,
providing a single, unified means of connecting sensor and peripherals
to MIThril cores.
Working MIThril body-bus sensors include:
- High-precision three-axis accelerometer (custom I2C device).
- IR Active tag reader (custom I2C device).
Integrated accelerometer/IR active tag reader/microphone/headphone
audio board for SAK core.
- USB microphone and headphone driver.
- USB CCD camera.
The USB and I2C protocols span a wide range of useful sensors and
peripherals, but the 12Mbit bandwidth constraint of USB and 400Kbit
constraint of I2C do impose some limitations on the type and number
of devices which may share this bus. Likewise, debugging console
connections should not be routed over the body bus for practical
reasons. Thus, it makes sense to connect some devices directly to a
MIThril core. These devices include:
- A re-cabled Twiddler One chording keyboard and modified Palm
folding keyboard, which are hot-swappable through an exterior plug and
connect to the same serial port on the Brightstar.
- The debugging console serial ports on the Brightstar and CerfBoard.
- Earthmate GPS receiver (serial)
- the MicroOptical
Clip-on, which is based on the Kopin CyberDisplay
320 Color chip, is one such peripheral. The Kopin display signal
is produced through a combination of the MPC823's on-board LCD driver,
timing code for the Brightstar's FPGA, and some additional
D to A and analog circuitry on a companion board which plugs
into the Brightstar IPEngine.
MIThril software includes FPGA code, Linux OS/driver code, UI code,
sensing/signal processing/statistical machine learning code, and
The current FPGA core for the BSEV core generates timing signals for
the HMD and provides great flexibility and control over the display.
For instance, a combination of the FPGA core and Linux driver allows
us to switch from "standard" field-sequential color to high-brightness
180Hz gray scale to low-power monochrome (red "gray scale") display
modes on the fly.
We have functioning Linux distributions for both of our current Linux
computing cores (Debian on the StrongARM, a Hard Hat Linux derivative
on the Brightstar).
We have functioning Linux I2C and USB master drivers for
the MPC823, which we will publish in our CVS repository shortly. We
also have a field-sequential-color frame-buffer driver for the BSEV
core, allowing us to use 24-bit color and X-windows.
We have made substantial progress in implementing the Enchantment user
interface, as described in the Enchantment Whitepaper. This
includes the Anduin window manager, which provides a mouseless
interface to X applications (Anduin is in our MIThril Code CVS
repository but not documented elsewhere yet) and the Enchantment
IPC whiteboard context server (also in the CVS repository, but not yet
The MIThril Inference Engine provides a set
of tools for applying statistical machine learning techniques to
classifying sensor data in real-time. The inference engine works with
the Enchantment IPC whiteboard to provide real-time classification and
regression to context-aware applications. (This code can also be found
in the CVS
repository). We have successfully used the MIThril Inference
Engine to implement a sophisticated activity classification system
based on accelerometer data; see the Motion Classification Project Paper
(gzip compressed postscript) for more details.
At present we are working on three major applications; the Memory Glasses project, the Context-Aware Cell Phone
Project and a biomedical patient functional assessment project
utilizing the SAK platform (a collaboration with researchers at the
Rochester Center for Future Health).