Research Group Projects and Descriptions

The goal of the Ambient Intelligence research group is to radically rethink the ways we interact with the digital world by designing interfaces that are more intuitive, more intelligent, and better integrated in our daily physical lives. We investigate ways to augment the everyday objects and spaces around us, making them responsive to our attention and actions. The resulting augmented environments offer opportunities for learning and interaction and ultimately for enriching our lives.

The Aesthetiscope is an interactive art installation with a wall of color that reacts to portray the relationship between an idea (a word, a poem, a song) and a person (a realist, a dreamer, a neurotic) standing before it. The Aesthetiscope analyzes each idea through a multi-perspectival linguistic analysis of connotation, through the realms of "think," "culturalize," "see," "intuit," and "feel," and then maps the ideas to a world of colors. With these different vocabularies of aesthetic, we can try to make sense of a "sunset." A sunset may be "seen," revealing the dark purple swatches with splashes of warm hues that characterize the visual remembrance of a sunset. But there is also an inner sunset. A sunset "felt" and "intuited" recalls warmth, beauty, and serenity, and these will bring about brighter, warmer, and more intense colors than the outer sunset. The Aesthetiscope encourages us to experience and reflect on aesthetic in a new way.

Meaning is all around us—it's just waiting to be found. An enormous wealth of digital information exists about the people and things in our daily lives. We exploit this digital information to enrich a person's everyday experience. Using RFID tags and a wireless, small RFID reader worn like a watch, a person is presented with meaningful information about the things he touches or the people whose hand he shakes. We are humanizing this information and making it meaningful, not just relevant. When you take "The Great Gatsby" from your bookshelf, the system reminds you of your father, who loved this book very much. When meeting a new person, the system tells you which interests and friends you have in common.

A display embedded in a mirror allows users to reflect on their recent activities and use of their time. The mirror recognizes its users and fetches data regarding their activities from their mobile device. The system also demonstrates the use of the mirror as a financial aid, helping users understand their spending. An alternative application which is based on the same platform is a clothing store shopping scenario, in which the mirror acts as a portal to the user's instant messager buddy list.

Blossom is a multiperson awareness system that uses ioMaterials-based techniques to connect distant friends and family. It provides an awareness medium that does not rely on the attention- and reciprocity-demanding interfaces that are provided by digital communication media such as mobile phones, SMS and email. Combining touch-based input with visual, haptic, and motile feedback, Blossoms are created as pairs that can communicate over the network, echoing the conditions of each other and forming an implicit, always-there link that physically express awareness, while retaining the instantaneous capabilities that define digital communication.

A mapping technology that transports one into the world of another, Cherry Blossoms gives witness to the tragedy of war. A backpack is outfitted with a small microcontroller and a GPS unit. Recent news of bombings in Iraq are downloaded to the unit every night, and their relative locations are superimposed on a map of Boston. If the wearer walks in a space in Boston that correlates to a site of violence in Baghdad, the backpack detonates and releases a compressed air cloud of confetti, looking for all the world like smoke and shrapnel. Each piece of confetti is inscribed with the name of a civilian who died in the war, and the circumstances of their death. A technology for empathy, Cherry Blossoms honors the human cost of war.

Simple, localized interactions between elements of a system can sometimes result in the emergence of unexpected and surprising patterns. This project explores how we can leverage these patterns to create visual and sonic textures using tangible nodes that can be directly manipulated by the end users.

Siftables, Quickies, and Cherry Blossoms are projects that embody digital information as physical tokens, opening up the world of physical affordances and the connection that human beings have to material objects. By leveraging the most powerful aspects of digital information management and physical form, we gain both the flexibility, instant access, and random access that digital information provides, and the physical presence of embodied tokens.

This project explores how programmable surfaces in an urban context can be shaped and textured in more flexible ways than traditional LED displays. Our screen consists of modular LED tiles that can be bent and rearranged to create curved or 3-D display surfaces. The tiles are textured using sillicon, which provides waterproofing as well as light diffusion.

Alumni Contributor(s): David Bouchard

What if you could look in the mirror and see not just what you look like, but also who you are? The Identity Mirror is an augmented evocative object. Looking into it, the viewer's face is painted over with identity keywords and interest keywords, sourced from a deep model of the viewer's identity. By gazing into Identity Mirror, a viewer can glean his identity-situation. Is his hair out of place? Are one of his interests out of place? How do his facial features combine to compose a gestalt? How do his various interests come together to compose an identity or aesthetic gestalt?

While most recommender systems continue to gather detailed models of their “users” within their particular application domain, they are, for the most part, oblivious to the larger context of the lives of their users outside of the application. What are they passionate about as individuals, and how do they identify themselves culturally? As recommender systems become more central to people’s lives, we must start modeling the person, rather than the user. We harvested 100,000 social network profiles, in which people describe themselves using a rich vocabulary of their passions and interests; from this, we built a map of how interests and identities link together semantically to form a fabric. Using this “InterestMap,” it is possible to make inferences about people's identities and what two strangers meeting for the first time might have in common.

ioMaterials is an umbrella project encompassing a variety of collocated sensing-actuation platforms looking at various aspects of dense sensing for humane communication, memory, and remote awareness. Using dense collocated sensing actuation and sensing, we can change common objects into an interface capable of hiding unobtrusively in plain sight. Relational Pillow and TextureWall are instantiations of this ideal.

Using several technologies, we are investigating how to provide mobile users with information that is highly relevant to their current contexts. In one project, ReachMedia, we built an RFID-reading wristband that offers users information and services related to the object they last touched. Another project, Invisible Media, is a ring that users can use to point at shelves, objects, or parts of a complex object in order to receive personally relevant information. Additional projects explore novel visual tag technology for authoring and consuming information about a tagged object, as well as means of accessing information via touch, presence in a particular space, or interaction with a surface. Each system pairs its input modality with a matched output that maximally leverages the unique capabilities of the system.

Alumni Contributor(s): Enrico Costanza and Xinyu H. Liu

Can static paint be programmable? Can a watercolor comic change? We explore these options by using special ink and embedded LEDs, and through the design of watercolor and paper paint that visually change and tell a story.

Programming is the art of constructing a story about objects and what they do in various situations, expressed in programming languages which are easy for the computer to accurately convert into executable code, but difficult for people to write and understand. This project explores using descriptions in a natural language like English as a representation for programs. We cannot convert arbitrary English descriptions to fully specified code, but we can use an expressive subset of English as a visualization tool. Simple descriptions of program objects and their behavior are converted to scaffolding (underspecified) code fragments that can be used as feedback for the designer and then elaborated. Our parser can infer a surprising amount of information about program structure from relations implicit in the linguistic structure; we call this "programmatic semantics." Our program editor, Metafor, dynamically converts a user's stories into program code. Users found it useful as a brainstorming tool.

Midas is a touch-based personalization system that bridges the gap between personal area networks (PANs) and explicit authentication. It is designed to interact with ubiquitous interfaces in the objects and environment surrounding us in order to achieve seamless access to information via public and pseudo-public surfaces and environments that modify themselves to the needs to the users on the fly and on contact. By confining the communication to touched object while avoiding the need to use an explicit authentication token such as a RFID, we enable new means of customizing our surroundings to our needs.

Using the built-in camera of a mobile phone and inexpensive printed tags, users can leave and retrieve video messages in specific locations or associated to specific objects. At the heart of the project is d-touch: an open-source system for computer vision recognition of markers. D-touch's peculiarity, compared to other marker systems, is that its recognition mechanism is based on topology rather than geometry. As a consequence, the system allows considerable freedom in the visual design of the markers, so that they can be "hidden" or designed to fit the specific aesthetic requirements of different applications.

Alumni Contributor(s): Enrico Costanza

The moving portrait is based on a set of black and white portraits, comprising a rich library of photographic sequences. It resides in a picture frame and interacts with its viewers using a variety of sensing techniques. The sensing architecture enables the portrait to be aware of viewers’ presence, identity, distance, speed, and body movements. The cognitive architecture controls the portrait’s reaction, taking into account the viewer’s behavior, the portrait’s mood, and memory of previous interactions, all of which contribute to complex, believable behavior. A portrait represents a part of our lives and reflects our feelings, but it is completely oblivious to the events that occur around it or to the people who view it. By adding interaction, dynamics, and memory to a familiar portrait we create a different and more engaging relationship between it and the viewer.

The most common functions of mobile devices involve alerting the user; whether notifying of incoming calls and messages or reminding of calendar events, the system has to catch the user's attention. This causes undesired interruptions to those surrounding the mobile device user, and in some circumstances the notification can be disruptive even for the user himself. A wearable peripheral display, embedded in a pair of eyeglasses, was built to deliver notification cues in a private, subtle, and non-obtrusive way. The display is composed of arrays of small LEDs of different colors, each placed at the end of the arms near the lens. The LEDs are lit (at very dim intensity) to display moving patterns in the wearers' peripheral vision. The display has a Bluetooth interface, so it can be controlled by standard mobile devices.

Alumni Contributor(s): Enrico Costanza and Elan Pavlov

Imagine taking a digital picture with your new camera-phone and the picture is able to annotate itself. For example, if you are at Fenway Park and take a picture, the picture will automatically be annotated with keywords such as "Fenway Park," "Baseball," or "Red Sox." This project takes advantage of a GPS receiver and a geographical search engine called MetaCarta to annotate pictures in real time with relevant keywords. These keywords help users to make sense of their surroundings when they take the picture, as well as to organize and retrieve pictures afterwards.

How can traditional values can be embedded into a digital object? We explore this concept by implementing a special guitar that combines physical acoustic properties with virtual capabilities. The acoustical values will be embodied by a wooden heart--a unique, replaceable piece of wood that will give the guitar an unique sound. The acoustic signal created by this wooden heart will be digitally processed in order to create flexible sound design.

Audiopint, subTextile, Wordplay, inktuitive, Sound of Touch, PureJoy, and SoundMites are enabling platforms for creative expression. These systems are construction kits, instruments, and toolkits, permitting the rapid and accessible construction of sonic, visual, and fabric-based interactive devices, embodying novel interfaces for expression, and enabling prototyping of new interactions.

The goal of ‘Quickies’ is to bring one of the most useful inventions of the twentieth century into the digital age: the ubiquitous sticky note. Sticky (aka Post-it) notes help us manage our to-do lists and capture short reminders and information needed in the near future, but keeping track of these Post-its can be a difficult task in itself. Quickies enrich the use of Post-it notes so that they can be tracked and managed more effectively. We give these stickies intelligence and the ability to remind us at the relevant time about the task we ought to perform. The project explores how the use of RFID and ink-recognition technologies can make it possible to create intelligent sticky notes that can be searched, send reminders and messages, and, more broadly, help us to connect seamlessly our physical and informational experiences.

ReachBand is a wireless wristband with an RFID reader that can detect user interactions with everyday objects, such as holding a book or opening a door. ReachBand enables the delivery of personalized services and just-in-time information to the user. It is used by various projects at the Lab, such as ReachMedia and Ambient Semantics. ReachBand v2 is being further developed, and will include updated electronics and communcation capabilities, as well as reduced dependence on a PC.

ReachMedia is an interface for on-the-move interaction with information and services for everyday objects. The purpose of this project is to blend the affordances of the physical world with the strengths of the wealth of related digital information, leveraging the particular strengths of each. The system uses ReachBand, a wireless wristband with an RFID reader and accelerometers to support hand- and eye-free interaction by using a unique combination of input and output methods such as gestural input and audio output.

Digital design for acoustic instrument. The idea is to enable guitar players to customize their own sound by assembling different sound cells instead of a single large sound box. each string will have it’s own bridge, each bridge will be connected to a different cell. changing the cell size, material or structure will allow sound design innovations, re-designing acoustic musical instrument according to the abilities and characteristics of rapid prototype matarials. open source and shared files environment can lead to a reality in which a player can download or design his own sound cells and add them (as a patch) to his instrument.

With the Relational Pillow project, we are trying to provide a simple, intimate, and personable communication medium between loved ones. The pillows are capable of sensing touch information, and displaying incoming touch data as a pattern of lights that show the outline "drawn" upon the remote pillow. Pillows can connect to each other over the network so that this sense of touch can be shared across long distances. The physical sensation of holding a pillow and interacting with it builds upon the idea of using the natural features of the object in order to acheive a deeper connection between the users, without interfering in the communication process itself.

We are exploring how to integrate information and communication capabilities into the everyday objects around us. Using smart materials, we are making objects such as paper-based interfaces, tables, mirrors, portraits, and pillows, and even flowers into dedicated interfaces to the digital functionality upon which we rely (such as SMS messaging, news, scheduling, or weather reports), thereby providing a more pleasant and integrated experience.

Shutters is a soft kinetic membrane for environmental control and communication. It is composed of actuated louvers (or shutters) that can be individually addressed for precise control of ventilation, daylight incidence, and information display. By combining smart materials, textiles, and computation, Shutters builds upon other façade systems to create living environments and work spaces that are more energy efficient, while being aesthetically pleasing and considerate of its inhabitants’ activities.

Siftables are active manipulatives that combine interaction affordances from tangible user interfaces, graphical user interfaces, and wireless sensor networks. They embody the information they represent in a direct manner that allows for physical manipulation. Siftable platform elements are small, battery-powered devices with inertial sensing, wireless communication, and graphical display capabilities. They can be manipultated gesturally, individually, or as a group, as a coordinated interface to computational processes and media editing or organization. Siftables builds on a body of work in tangible tabletop interfaces that offers physical "handles" to digital entities, but allows for graphical display on each manipulative, and removes the requirement for externally installed infrastructure, freeing the resulting interactions from the two-dimensional surface of the table. Application areas for Siftables include organizing and associating entities like conference papers, airline flights, or ideas in a brainstorming session, expressive and creative activities such as musical creation, or playful multi-person games.

Alumni Contributor(s): Jeevan James Kalanithi

Spotlight is about an artist's ability to create a new meaning using the combination of interactive portraits and diptych or polyptych layouts. The mere placement of two or more portraits near each other is a known technique to create a new meaning in the viewer's mind. Spotlight takes this concept into the interactive domain, creating interactive portraits that are aware of each other's state and gesture. So not only the visual layout, but also the interaction with others creates a new meaning for the viewer. Using a combination of interaction techniques, Spotlight engages the viewer at two levels. At the group level, the viewer influences the portraits "social dynamics". At the individual level, a portrait's "temporal gestures" expose much about the subject's personality.

Sprout I/O is a kinetic fur that can capture, mediate, and replay the physical impressions we leave in our environment. It combines embedded electronic actuators with a texturally rich substrate that is soft, fuzzy, and pliable to create a dynamic structure where every fur strand can sense physical touch and be individually moved. By developing a composite material that collocates kinetic I/O, while preserving the expectations that we normally have from interacting with physical things, we can more seamlessly embed and harness the power of computation in our surrounding environments to create more meaningful interfaces for our personal and social activities.

subTextile is a toolkit for behavioral textiles: the intersection of on-body computation and electronic textiles focusing on the interactive capability of electronic textiles. It provides a powerful visual programming language and hardware platform specifically designed to create complete behavioral textile systems. Using a rich, goal-oriented interface, subTextile makes it possible for technical novices to explore electronic textiles, while providing open-ended expandability to experts. As e-textiles mature, better tools and techniques are needed by artists and designers experimenting with these new materials. The subTextile project was created to support cross-pollination between technical and design disciplines in the hopes of fostering greater creativity and depth within the field.

Surflex is a programmable surface for the design and visualization of physical objects and spaces. It combines the different memory and elasticity states of its materials to deform and gain new shapes, providing a novel alternative for 3D fabrication and the design of physically adaptable interfaces.

This work explores a technological answer to the perennial question "What's for dinner?" Few of us know with great certainty the exact food we crave; instead, we stew on the question and explore the nature of cravings with imaginative descriptions: "I feel like something light, fresh, sophisticated, not too mushy—something influenced by Thai or Indian, something aromatic." Synaesthetic Recipes is a visual search program that allows imaginative textual descriptions, using them to drive recipe recommendations. A database of 100,000 recipes is automatically annotated with common sense about food. An artificial intelligence robotic reader reads each recipe, and based on flavors of the ingredients and cooking procedures predicts how a food will look, taste, and smell. We are translating recipes into the rich descriptive vernacular of how people naturally conceptualize their cravings for food.

taggyMedia is an online interactive system that provides easier and more interesting ways of media search and exploration. By constucting a semantic network using a social tagging scheme, we are able to design a system that provides better tagging interface, and a variety of ways of exploration based on concepts, emotions, scenarios, etc. The system improves the current "keyword-matching" searching scheme to a much more intuitive and explorative one.

People's tastes are their systems of aesthetic judgment and preference. Taste systems are generative—understanding the tastes of friends makes it possible to anticipate what book or movie they would like, or how they might feel about a particular news topic. We are pioneering a computational approach to automatically modeling the tastes of individuals and whole communities by mining their texts—blogs, social network profiles, and homepages—and analyzing the attitudes and preferences contained therein. By also applying machine-learning techniques, we generalize these models so that they can predict how a person would feel about hypotheticals, and even forecast tomorrow's trends.

Transitive Materials is an umbrella project encompassing novel materials, fabrication technologies, and traditional craft techniques which can operate in unison to create objects and spaces that realize truly omnipresent interactivity. We are developing interactive textiles, ubiquitous displays, and responsive spaces that seamlessly couple input, output, processing, communication, and power distribution, while preserving the uniqueness and emotional value of physical materials and traditional craft. Life in a Comic, Physical Heart in a Virtual Body, Augmented Pillows, Flexible Urban Display, Shutters, Sprout I/O, and Pulp-Based Computing are current instantiations of these technologies.

The system consists of virtual representations of a handful of people that react to what a person is currently reading, writing, or talking about. These "digital personas" are constructed automatically by analyzing personal texts (such as interviews/blogs/messages posted by the person being modeled) using natural language processing techniques and the Open Mind Common Sense knowledge base. The "What Would They Think?" system has several applications. For example, it can help a person form a deep understanding of a community that is new to them by constantly showing them the attitudes and disagreements of strong personalities of that community. Other applications include virtual "mentors" or "guides," and knowledge-sharing in communities and workgroups.