Anduin window manager
The MIThril vision
MIThril Enchantment Software
Real-Time Contxt Engine
Wearables Home Page
The MIThril Real-Time Context Engine (formerly named the MIThril
Inference Engine) is an light-weight modular architecture for the
development and implementation of real-time context classifiers for
We find that a systematic approach to activity classification based on
modern machine learning techniques can greatly simplify the process of
developing and implementing real-time activity classification models.
The MIThril Real-Time Context Engine was developed to provide this
systematic framework in a flexible, modular, open-source form.
The MIThril Real-Time Context engine is a combination of methodology
and supporting software tools. To train our models we use Kevin
Murphy's Bayes Net Toolkit and Matlab. (We have a Gaussian Mixture Modeling Tutorial
available online -- download the tutorial here in tar.gz format or zip format.).
In addition to the tutorial, there is Real-Time Context
Engine whitepaper explaining the real-time context engine and
describing in some detail the development of a three-class
accelerometer-based activity model.
Real-time classification is implemented as Linux C/C++ code on top of
the Enchantment Whiteboard and Signal systems -- see the inference
section of the MIThril CVS
One of the first projects implemented using the MIThril Real-Time
Context engine was a
five-state single-accelerometer based activity
This page provides a simplified, non-technical overview of the
real-time modeling and classification side of the MIThril Real-Time
Context engine. For a detailed discussion of modern statistical
classification techniques, see R.O. Duda, P.E. Hart, and D.G. Stork,
Pattern Classification. Wiley-Interscience, 2nd edition,
2000. (Available from Amazon, among other sources.)
The MIThril Real-Time Context Engine architecture is divided into four distinct processes or stages:
The arrows in the diagram represent information flow, starting with
the "World" and ending with the application. At each stage a distinct
operation is performed that enables the next.
- Feature Extraction
The user's world is complex. The only things we can know
are what we measure thorough sensors or are explicitly told.
Measuring something in the "analog" world produces a sequence of
discrete digital observations, or digital signal. The amount
of information in the digital signal is a function of the complexity of
the original analog process, the sensitivity and precision of the
sensor, and the resolution and rate of the sampling process.
For any particular modeling task, this digital signal contains some
useful information and some extraneous information. For example, a
three-axis accelerometer produces measurements that combine "true"
acceleration, some systematic bias, and some random noise.
In the feature extraction stage, a raw
sensor signal is transformed into a feature signal more suitable for
a particular modeling task. For our activity classifier we compute
short-time-windowed power spectrum features
We can often simplify a modeling task by transforming raw observations
into a form that is more appropriate for a particular modeling task.
For example, features that use a pitch/energy representation might be
more appropriate for a speaker-identification modeling task than raw
In the modeling stage, a statistical model (such as a Gaussian mixture
model, Hidden Markov Model, etc) is used to classify a
feature signal in real-time.
For each context we want to model, a statistical model is created.
These models allow us to answer the question "which state is the user
in," or "how likely is state X?"
In the inference stage, the results of the modeling stage, possibly
combined with other information, are fed into a Bayesian inference
system for complex interpretation and decision-making. The inference
stage is only implemented under Matlab, at present.
At present, our applications use only the output of the modeling
stage. However, we foresee the need for more complex applications in
which the output of multiple modeling stages provides the input into a
Bayesian inference system.
Action, or what to do with the output MIThril Real-Time Context
Engine, is left up to the application.
An important class of action is one that provokes a user interaction,
which in turn effects the user's state and hence the "world."