BrainVR is a learning experience for neuroscience that leverages motion-tracked virtual reality to convey cutting-edge knowledge in neuroscience. In particular, an interactive 3D model of the retina illustrates how the eye detects moving objects. The goal of the project is to explore the potential of motion tracked virtual reality for learning complex concepts, and build reusable tools to maximize this potential across knowledge domains.

We are developing a new and exciting tool for expression in paint, combining technology and art to bring together the physical and the virtual through the use of robotics, artificial intelligence, signal processing, and wearable technology. Our technology promotes expression in paint not only by making it a lot more accessible, but also by making it flexible, adaptive, and fun, for everyone across the entire spectrum of abilities. With the development of the technology, new forms of art also emerge, such as hyper, hybrid, and collaborative painting. All of these can be extended to remote operation (or co-operation) thanks to the modular system design. For example, a parent and a child can be painting together even when far apart; a disabled person can experience an embodied painting experience; and medical professionals can reach larger populations with physical therapy, occupational therapy, and art therapy, including motor/neuromuscular impaired persons.

FingerReader is a finger-worn device that helps the visually impaired to effectively and efficiently read paper-printed text. It works in a local-sequential manner for scanning text that enables reading of single lines or blocks of text, or skimming the text for important sections while providing auditory and haptic feedback.

As part of a Google-sponsored Glass developer event, we created a Glass-enabled improv comedy show together with noted comedians from ImprovBoston and Big Bang Improv. The actors, all wearing Glass, received cues in real time in the course of their improvisation. In contrast with the traditional model for improv comedy, punctuated by "freezing" and audience members shouting suggestions, using Glass allowed actors to seamlessly integrate audience suggestions. Actors and audience members agreed that this was a fresh take on improv comedy. It was a powerful demonstration that cues on Glass are suitable for performance: actors could become aware of the cues without having their concentration or flow interrupted, and then view them at an appropriate time thereafter.

HRQR is a visual Human and Machine Readable Quick Response Code that can replace usual 2D barcode and QR Code applications. The code can be read by humans in the same way it can be read by machines. Instead of relying on a computational error correction, the system allows a human to read the message and therefore is able to reinterpret errors in the visual image. The design is highly inspired by a 2,000 year-old Arabic calligraphy called Kufic.

JaJan! is a telepresence system wherein remote users can learn a second language together while sharing the same virtual environment. JaJan! can support five aspects of language learning: learning in context; personalization of learning materials; learning with cultural information; enacting language-learning scenarios; and supporting creativity and collaboration. Although JaJan! is still in an early stage, we are confident that it will bring profound changes to the ways in which we experience language learning and can make a great contribution to the field of second language education.

LuminAR reinvents the traditional incandescent bulb and desk lamp, evolving them into a new category of robotic, digital information devices. The LuminAR Bulb combines a Pico-projector, camera, and wireless computer in a compact form factor. This self-contained system enables users with just-in-time projected information and a gestural user interface, and it can be screwed into standard light fixtures everywhere. The LuminAR Lamp is an articulated robotic arm, designed to interface with the LuminAR Bulb. Both LuminAR form factors dynamically augment their environments with media and information, while seamlessly connecting with laptops, mobile phones, and other electronic devices. LuminAR transforms surfaces and objects into interactive spaces that blend digital media and information with the physical space. The project radically rethinks the design of traditional lighting objects, and explores how we can endow them with novel augmented-reality interfaces.

Move Your Glass is an activity and behavior tracker that also tries to increase wellness by nudging the wearer to engage in positive behaviors.

PsychicVR integrates a brain-computer interface device and virtual reality headset to improve mindfulness while enjoying a playful immersive experience. The fantasy that any of us could have superhero powers has always inspired us, and by using Virtual Reality and real-time brain activity sensing we are moving one step closer to making this dream real. We non-invasively monitor and record electrical activity of the brain and incorporate this data in the VR experience using an Oculus Rift and the MUSE headband. By sensing brain waves using a series of EEG sensors, the level of activity is fed back to the user via 3D content in the virtual environment. When users are focused, they are able to make changes in the 3D environment and control their powers. Our system increases mindfulness and helps achieve higher levels of concentration while entertaining the user.

Remot-IO is a system for mobile collaboration and remote assistance around Internet-connected devices. It uses two head-mounted displays, cameras, and depth sensors to enable a remote expert to be immersed in a local user's point of view, and to control devices in that user's environment. The remote expert can provide guidance through hand gestures that appear in real time in the local user's field of view as superimposed 3D hands. In addition, the remote expert can operate devices in the novice's environment and bring about physical changes by using the same hand gestures the novice would use. We describe a smart radio where the knobs of the radio can be controlled by local and remote users. Moreover, the user can visualize, interact, and modify properties of sound waves in real time by using intuitive hand gestures.

Screen interactions have been shown to contribute to increases in stress, anxiety, and deficiencies in breathing patterns. Since better respiration patterns can have a positive impact on wellbeing, ScreenSpire improves respiration patterns during information work using subliminal biofeedback. By using subtle graphical variations that are tuned to attempt to influence the user subconsciously user distraction and cognitive load are minimized. To enable a true seamless interaction, we have adapted an RF based sensor (ResMed S+ sleep sensor) to serve as a screen-mounted contact-free and respiration sensor. Traditionally, respiration sensing is achieved with either invasive or on-skin sensors (such as a chest belt); having a contact-free sensor contributes to increased ease, comfort, and user compliance, since no special actions are required from the user.

Skrin is an exploration project on digitalized body skin surface using embedded electronics and prosthetics. Human skin is a means for protection, a mediator of our senses and a presentation of ourselves. Through several projects, we expand the expression capacity of body surface and emphasize on the dynamic aesthetics of body texture by technological means.

We are developing a very intuitive and interactive platform to make complex information--especially science, technology, engineering, and mathematics (STEM) material--truly accessible to blind and visually impaired students by using a tactile device with no loss of information compared with printed materials. A key goal of this project is to develop tactile information-mapping protocols through which the tactile interface can best convey educational and other graphical materials.

Mental wellbeing is intimately tied to both social support and physical activity. The Challenge is a tool aimed at promoting social connections and decreasing sedentary activity in a workplace environment. Our system asks participants to sign up for short physical challenges and pairs them with a partner to perform the activity. Social obligation and social consensus are leveraged to promote participation. Two experiments were conducted in which participants’ overall activity levels were monitored with a fitness tracker. In the first study, we show that the system can improve users' physical activity, decrease sedentary time, and promote social connection. As part of the second study, we provide a detailed social network analysis of the participants, demonstrating that users’ physical activity and participation depends strongly on their social community.