Project

AttentivU

Copyright

MIT Media Lab

Caitlin Morris

[Update 12/13/2019: This project has open positions for UROPs/visiting students or interns. One position focuses around electrical engineering, another one on app development and design and the third one is around ML/AI/Signal Processing. We are particularly interested in finding UROPs for an IAP/spring period! Please contact nkosmyna@media.mit.edu for more information.]

AttentivU is  a  device, in the  form factor of a pair of glasses,  which senses brain activity (electroencephalography - EEG) as well as eye movements (electrooculography - EOG) to measure different cognitive processes in real time,  including  cognitive load, fatigue, engagement, and focus (please check our FAQ on this page for the definitions!). The device can be used for passive or active interventions, for example to monitor the state of the user, providing gentle audio or haptic feedback when the user is less attentive (driving scenario) or adapting the environment when cognitive overload is detected (blocking the notifications). The system can operate in a standalone, non-networked fashion to ensure pr… View full description

[Update 12/13/2019: This project has open positions for UROPs/visiting students or interns. One position focuses around electrical engineering, another one on app development and design and the third one is around ML/AI/Signal Processing. We are particularly interested in finding UROPs for an IAP/spring period! Please contact nkosmyna@media.mit.edu for more information.]

AttentivU is  a  device, in the  form factor of a pair of glasses,  which senses brain activity (electroencephalography - EEG) as well as eye movements (electrooculography - EOG) to measure different cognitive processes in real time,  including  cognitive load, fatigue, engagement, and focus (please check our FAQ on this page for the definitions!). The device can be used for passive or active interventions, for example to monitor the state of the user, providing gentle audio or haptic feedback when the user is less attentive (driving scenario) or adapting the environment when cognitive overload is detected (blocking the notifications). The system can operate in a standalone, non-networked fashion to ensure privacy of the collected data.  The use of the AttentivU glasses can be limited to just those moments when the user decides they want to learn to sustain their attention.

We have tested the first generation prototype of the device in workplace (check the paper here ) and learning settings (check the paper here and here) with over 100 subjects. We performed experiments with people studying or working by themselves, viewing online lectures as well as listening to a professor at the university.  We have now completed the first test of the glasses prototype (check the paper here) with more than 30 subjects who were performing a driving task in a simulator overnight when their attention waned, and the system reminded them with a sound or with a vibration to pay attention to the road . This research paper was rewarded with the honorable mention award at the 2019 AutoUI conference (paper will appear in the end of September 2019). Please check the dedicated page about the car use case scenarios here.

Please check the FAQ (bottom of this page) if you have more questions about the project and/or want to collaborate with us!

Project Lead: Nataliya Kosmyna, Ph.D 

Project Team and Contributors (in alphabetical order):

Katie Bacher, Jin Dou, Alexa Gross, Thanh Nguyen - current collaborators (as of October 2019);

Caitlin Morris, Utkarsh Sarawgi - past collaborators.

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Copyright

MIT Media Lab

Awards

Fast Company’s 2019 World Changing Ideas Awards.

AttentivU  received honorable mention in Fast Company’s 2019 World Changing Ideas Awards.

Honorable mention award at AutoUI conference.

A paper on using AttentivU glasses in driving scenario to measure cognitive fatigue of drivers, has received  honorable mention award at AutoUI 2019 conference in Utrecht.

Some Q&A about AttentivU we are getting from time to time

  1. How did you have the idea for the project? What is the problem you are trying to solve?
  2. What are all these headbands, collars, and glasses about on your images? "This makes me think about a choker..."
  3. What populations do you target and work with?
  4. What about limitations and drawbacks in all this and where do you go next?
  5. Is it mind reading?
  6. What is the current state of the project?
  7. "OMG!!! Have you seen those latest videos from China showing poor kids being tracked in school! Is your system the same?"
  8. How can I help/provide my expertise/participate in the project?
  9. What is cognitive load, fatigue, engagement, and focus?
  1. How did you have the idea for the project? What is the problem you are trying to solve?

    Initially, the idea appeared as a way to help those with  low attention span issues. Given the magnitude of information surrounding us, with around-the-clock Internet access, and the constant shifting between increasingly complex tasks, it is no surprise many of us these days suffer from low attention spans.  We thus hypothesized that real-time monitoring of attention level and issuance of feedback may be valuable  in situations where attention is paramount, producing improved learning and performance outcomes.  

    After performing a lot of tests with more than 160 people up to date, we discovered a set of new use cases, scenarios, applications, as well as cognitive processes which our system can measure, like cognitive load or fatigue.  We also updated our feedback interventions, now having passive and active options, as well as context-based interventions (if you are falling asleep in the car, a light inside of the car as well as music might be turned on to wake you up along with the suggestion to stop and get some rest). 

    Since then we  moved forward to workplace applications, driving, sports, and other environments where extreme attention, performance, or fatigue are known to occur.  

    We currently only work with neurotypical adults on this project. 

  2. What are all these headbands, collars, and glasses about on your images? "This makes me think about a choker..."

    Our primary concern was about solving a problem and not just building a new and fancy device, so for the very first prototype we took an existing hardware from the market. We used a lightweight, commercial EEG headband, similar to the ones used in other studies. Using a band facilitates the reproducibility and generalizability of the study as well as its comparability to previous work. Moreover, the bands have a lower price compared to professional headsets. As one of the core ideas (see question 1) was to also test the feasibility of having a closed-loop system, meaning providing real-time feedback, we needed to provide the users with the feedback about their brain activity. For our work we ruled out the use of visual feedback, as our system is meant to be used in visually charged environments, and we do not want users to get even more distracted by checking their phones or some other devices to view their engagement level. Audio feedback may be a viable option in the case of bone conduction earphones, as such earphones do not require covering the ear. Hence, for our very first study, we decided to use vibrotactile and auditory feedback modalities. Prior research shows that the location of vibrotactile devices on the body determines how effective the vibrotactile stimulation will be. Zeagler as well as Karuei et al. identified the upper chest area below the collar bone as well as the wrist as the optimal body locations. We chose to include the vibration elements in the form factor of a scarf, to be worn on the upper part of the chest. The rationale was to standardize haptic feedback delivery across users and avoid confusion with common, commercial wrist bands they might be used to. We considered several possible configurations of devices to be worn on the upper part of the chest and decided on a scarf as they are made of soft fabrics and thus, the vibrations are less disruptive and more private. 

    Ultimately, as we tested the device with more than 160 people (all 18+ years old), we moved to the first iteration of the system, incorporating everything within one hardware prototype, keeping two feedback options available, based on the usability tests and comments from the users. 

  3. What populations do you target and work with?

    We report and publish only with neurotypical adults (18+ years old) as participants. 

    The project closely involves users throughout the experiments so as to design and test methods that are socially and ethically acceptable to all parties involved. We are always happy to welcome more collaborators, especially in the domains  of psychology, ethics, as well as the use cases we currently target like sports and driving.

  4. What about limitations and drawbacks in all this and where do you go next?

    An important point to be made about AttentivU is that the system was not developed to be worn 24/7 nor to keep a person continuously attentive or engaged. A continuous high level of attention or engagement is not needed as it can negatively affect learning outcomes. The proposed system might be  an option for users who have problems with staying on task when attention is required, e.g., falling asleep at work. As it was suggested by several participants, other use cases are to be considered with this system, such as driving (which we implemented as a driving scenario overnight in a simulator, where users have been tired, not attentive, and even falling asleep. The paper is accepted and will appear in the end of September 2019).

    Last but not least, very much needed are discussions about ethical issues with regard to development of such devices, as well as the questions pertaining to the privacy and security of both the data but also of the users of such technology. 

  5. Is it mind reading?

    No,  and it has nothing to do with mind reading.

  6. What is the current state of the project?

    We currently test it in different environments and situations with neurotypical adults only in a number of scenarios where high attention, performance, and engagement are required (like driving or sports). We also start exploring directions beyond attention and feedback delivery for this system (cognitive load, fatigue, passive monitoring).

  7. "OMG!!! Have you seen those latest videos from China showing poor kids being tracked in school! Is your system the same?"

    No it is not the same system but yes, we have seen those videos and are very well aware of the potential misuses of such systems. Good news is that 1. we are currently investigating the potential benefits/drawbacks of such systems 2. though our current prototype is a functioning and experimentally validated system,  it has already been a 4th  iteration of the design. Both its hardware and software characteristics and features have changed dramatically over time. More importantly, the use cases shift and change constantly: with feedback component or without, haptic or auditory feedback, positive reinforcement within context-based interaction or not - we test and try these different versions, as different solutions are suitable for different scenarios and populations 3. Current system is not tested with children, even more so - not in the classrooms 4. We welcome constructive feedback and we are happy to share more information: we have recently had a very big discussion over different social media with general public as well as specialists in ethics and teachers,  we have collected a lot of feedback for the use cases regarding tests with minors  5. We welcome external collaborators, especially in the domain of ethics and teaching 6. All our protocols are approved by IRB of the institution. Finally, we fully adhere to the mission statement of our group, and this project is not an exception, please check the statement here.  

  8. How can I help/provide my expertise/participate in the project?

    We welcome external collaborators, especially those who work in ethics domain and teachers! Please contact us at nkosmyna AT media.mit.edu and tell us more!

  9. What is cognitive load, fatigue, engagement, and focus?

    There are so many definitions of cognitive load, fatigue, engagement, and focus in the literature! These definitions vary a lot of depending on which literature you are reading: human-computer interaction, neuroscience, etc. In our papers we always provide the definitions of what and how we are measuring.