Project

AttentivU

Copyright

MIT Media Lab

Caitlin Morris

[Update 09/12/2019: This project has two open positions for UROPs/visiting students or interns. One position focuses around electrical engineering, another one on app development and design. Do not hesitate to contact nkosmyna@media.mit.edu for more information.]

AttentivU is  a  device, in the  form factor of a pair of glasses,  which senses brain activity (electroencephalography - EEG) as well as eye movements (electrooculography - EOG) to measure different cognitive processes in real time,  including  cognitive load, fatigue, engagement, and focus. The device can be used for passive or active interventions, for example to monitor the state of the user, providing gentle audio or haptic feedback when the user is less attentive or adapting the environment when cognitive overload is detected. The system operates in a standalone, non-networked fashion to ensure privacy of the collected data. While medical  interventions for attention issues have many drawbacks, the use of the AttentivU glasses can be limited to just those moments when the user decides they want to learn to sust… View full description

[Update 09/12/2019: This project has two open positions for UROPs/visiting students or interns. One position focuses around electrical engineering, another one on app development and design. Do not hesitate to contact nkosmyna@media.mit.edu for more information.]

AttentivU is  a  device, in the  form factor of a pair of glasses,  which senses brain activity (electroencephalography - EEG) as well as eye movements (electrooculography - EOG) to measure different cognitive processes in real time,  including  cognitive load, fatigue, engagement, and focus. The device can be used for passive or active interventions, for example to monitor the state of the user, providing gentle audio or haptic feedback when the user is less attentive or adapting the environment when cognitive overload is detected. The system operates in a standalone, non-networked fashion to ensure privacy of the collected data. While medical  interventions for attention issues have many drawbacks, the use of the AttentivU glasses can be limited to just those moments when the user decides they want to learn to sustain their attention.

We have tested the first generation prototype of the device in workplace and learning settings with over 100 subjects. We performed experiments with people studying or working by themselves, viewing online lectures as well as listening to a professor at the university.  We have now completed the first test of the glasses prototype with more than 30 subjects who were performing a driving task in a simulator overnight when their attention waned, and the system reminded them with a sound or with a vibration to pay attention to the road . This research was rewarded with the honorable mention award at the 2019 AutoUI conference (paper will appear in the end of September 2019). 

Project Lead: Nataliya Kosmyna, Ph.D

Project Team: Caitlin Morris, Thanh Nguyen and Pattie Maes

Past collaborator: Utkarsh Sarawgi

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Some Q&A about AttentivU we are getting from time to time

  1. How did you have the idea for the project? What is the problem you are trying to solve?
  2. What are all these headbands, collars, and glasses about on your images? "This makes me think about a choker..."
  3. What populations do you target and work with?
  4. What about limitations and drawbacks in all this and where do you go next?
  5. Is it mind reading?
  6. What is the current state of the project?
  1. How did you have the idea for the project? What is the problem you are trying to solve?

    Initially, the idea appeared as a way to help those with  low attention span issues. Given the magnitude of information surrounding us, with around-the-clock Internet access, and the constant shifting between increasingly complex tasks, it is no surprise many of us these days suffer from low attention spans.  We thus hypothesized that real-time monitoring of attention level and issuance of feedback may be valuable  in situations where attention is paramount, producing improved learning and performance outcomes.  

    After performing a lot of tests with more than 160 people up to date, we discovered a set of new use cases, scenarios, applications, as well as cognitive processes which our system can measure, like cognitive load or fatigue.  We also updated our feedback interventions, now having passive and active options, as well as context-based interventions (if you are falling asleep in the car, a light inside of the car as well as music might be turned on to wake you up along with the suggestion to stop and get some rest). 

    Since then we  moved forward to workplace applications, driving, sports, and other environments where extreme attention, performance, or fatigue are known to occur.  

    We currently only work with neurotypical adults on this project. 

  2. What are all these headbands, collars, and glasses about on your images? "This makes me think about a choker..."

    Our primary concern was about solving a problem and not just building a new and fancy device, so for the very first prototype we took an existing hardware from the market. We used a lightweight, commercial EEG headband, similar to the ones used in other studies. Using a band facilitates the reproducibility and generalizability of the study as well as its comparability to previous work. Moreover, the bands have a lower price compared to professional headsets. As one of the core ideas (see question 1) was to also test the feasibility of having a closed-loop system, meaning providing real-time feedback, we needed to provide the users with the feedback about their brain activity. For our work we ruled out the use of visual feedback, as our system is meant to be used in visually charged environments, and we do not want users to get even more distracted by checking their phones or some other devices to view their engagement level. Audio feedback may be a viable option in the case of bone conduction earphones, as such earphones do not require covering the ear. Hence, for our very first study, we decided to use vibrotactile and auditory feedback modalities. Prior research shows that the location of vibrotactile devices on the body determines how effective the vibrotactile stimulation will be. Zeagler as well as Karuei et al. identified the upper chest area below the collar bone as well as the wrist as the optimal body locations. We chose to include the vibration elements in the form factor of a scarf, to be worn on the upper part of the chest. The rationale was to standardize haptic feedback delivery across users and avoid confusion with common, commercial wrist bands they might be used to. We considered several possible configurations of devices to be worn on the upper part of the chest and decided on a scarf as they are made of soft fabrics and thus, the vibrations are less disruptive and more private. 

    Ultimately, as we tested the device with more than 160 people (all 18+ years old), we moved to the first iteration of the system, incorporating everything within one hardware prototype, keeping two feedback options available, based on the usability tests and comments from the users. 

  3. What populations do you target and work with?

    We report and publish only with neurotypical adults (18+ years old) as participants. 

    The project closely involves users throughout the experiments so as to design and test methods that are socially and ethically acceptable to all parties involved. We are always happy to welcome more collaborators, especially in the domains  of psychology, ethics, as well as the use cases we currently target like sports and driving.

  4. What about limitations and drawbacks in all this and where do you go next?

    An important point to be made about AttentivU is that the system was not developed to be worn 24/7 nor to keep a person continuously attentive or engaged. A continuous high level of attention or engagement is not needed as it can negatively affect learning outcomes. The proposed system might be  an option for users who have problems with staying on task when attention is required, e.g., falling asleep at work. As it was suggested by several participants, other use cases are to be considered with this system, such as driving (which we implemented as a driving scenario overnight in a simulator, where users have been tired, not attentive, and even falling asleep. The paper is accepted and will appear in the end of September 2019).

    Last but not least, very much needed are discussions about ethical issues with regard to development of such devices, as well as the questions pertaining to the privacy and security of both the data but also of the users of such technology. 

  5. Is it mind reading?

    No,  and it has nothing to do with mind reading.

  6. What is the current state of the project?

    We currently test it in different environments and situations with neurotypical adults only in a number of scenarios where high attention, performance, and engagement are required (like driving or sports). We also start exploring directions beyond attention and feedback delivery for this system (cognitive load, fatigue, passive monitoring).