Dormio: Interfacing with Dreams

Frequently Asked Questions

1. In layman's terms, how does Dormio work?
2. Why are you doing this?
3. What else should I read about this, who else is working on this, and what inspired you?
4. How did you come up with this idea and who is on the team making it happen?
5. Is this lucid dreaming?
6. Is this dreaming? What is stage 1 sleep?
7. What's new here, compared to the Steel Ball Technique and past sleep neuroscience work?
8. Can I have one?
9. I don't have a robot!
10. Can this be used for evil/authoritarian mind control?
11. Where has this work been shown?
12. What are the next projects you're planning?
13. What else are you all building over there?

1. In layman's terms, how does Dormio work?

   The system is conceptually quite simple. The aim is to influence and extend a transitional state of sleep. To do so, we must track this transitional state (hypnagogia) and interrupt when it is ending. So,  a user wears a device which collects biosignals that track transitions in sleep stages. In our new device, those signals come from the hand, where we can gather data on loss of muscle tone, heart rate changes, and changes in skin conductance. When those biosignals appear to signal the end of a transitional state, audio from the social robot is triggered, and that person is knocked just a little bit back into wakefulness, but not into full wakefulness. We use this audio cue as an inception protocol, doing this slight wake up with words (like "fork" or "rabbit"), and have found that in the subjects we tested, those words reliably entered the hypnagogic dreams as dream content. Pretty happy about that! After this slight wake up, we initiate a conversation about dream content with users via the Jibo social robot and record anything that is said, as hypnagogic amnesia is reported and we don't want people forgetting their useful ideas. After this conversation, the system lets users drift back towards sleep, only interrupting again when their biosignals appear to signal another transition into deeper sleep. This is done repetitively to incept dreams and extract dream reports. 

2. Why are you doing this?

   I find the idea that there is a state of mind which composes and constructs my conscious self, but remains inaccessible to me, both frustrating and alluring. Hypnagogia is a "me" that I am unfamiliar with, a "me" that slips past memory as we drift into unconsciousness. Good neuroscience, to me, is effective self-examination. Good technology in service of making neuroscience relevant outside the laboratory, then, should facilitate self-examination. The ends of this project are both practical and philosophical. I have no doubt that hypnagogia holds applications for augmenting memory, learning, and creativity. Yet also, after having explored the state myself, I find it to be a deeply valuable and inspiring sort of self-seeing which was inaccessible to me previously. As Nobel Prize winner Eric Kandel said, "human creativity...stems from access to underlying, unconscious forces." To know myself, and to be my most creative self, I'm interested in building tools for self-exploration in this sleep state. I would like to create a tool that I can hand people, that they can take home, and on their own explore and augment themselves. (Adam)

3. What else should I read about this, who else is working on this, and what inspired you?

   This project is inspired by a very old technique—the Steel Ball technique—described in the paper we wrote for alt.CHI. Our work, which extends this technique in scope and capability, is impossible without past work investigating possibilities of influencing dreams in the neuroscience lab. Read about each of them if Dormio has piqued your curiosity. Scientists like Stephen LaBerge and Benjamin Baird do wonderful work on later-stage lucid dreaming, focusing on the REM state. Scientists like Jonathan Smallwood and Jonathan Schooler have done work on mind-wandering and creativity, inspiring our idea that fluid thinking outside of executive control in hypnagogia (like mind-wandering) could augment creativity. Work by Deirdre Barrett compiling moments of inspiration found in sleep, and work by Robert Stickgold and Tore Nielsen on microdream phenomenology, all encouraged and informed us. Andreas Mavromatis wrote a whole thesis on hypnagogia, and his writing gave us a sense of the poetry and practical applications of this state (as did Nabokov, Oliver Sacks, Yoga Nidra practitioners, and Edgar Allen Poe writing on hypnagogia). Our sense of this vast work was given to us by the three advisors who have helped us most throughout this work, in and out of the classroom—Professors Pattie Maes, Ed Pace-Schott and Robert Stickgold.

4. How did you come up with this idea and who is on the team making it happen?

   None of this would have happened without a deeply interdisciplinary team enabled by the Media Lab community, including Ishaan Grover, Pedro Reynolds-Cuéllar, Adam Haar Horowitz, Aby Jain, Tomás Vega, Oscar Rosello, and more. There is a bit written about that process on this website, and you can see earlier versions of the project there too. It has been a long process! Ishaan Grover and Adam Haar Horowitz took the Human Machine Symbiosis class with Prof. Pattie Maes, while listening to Prof. Robert Stickgold lecture on dream states. We got together and tried to make an EEG detect sleep stages, and did a decent job detecting sleep spindles with a cheap Muse EEG, but thought that detection wouldn't generalize across individuals. We had friends like Rebecca Kleinberger and  Sophia Yang  who helped with our first glove prototype, and helped think through possible interaction designs. We had awesome subjects like Marie Therese Png who went in and out of hypnagogia and explained how our tech tweaks changed their experiences. And now we have a team of cognitive scientists, engineers, and makers who create experiments and possibilities. Plus we've had incredible past work to build with and build on from the neuroscience and HCI worlds, and advice and inspiration from mentors including Professors Pattie Maes, Ed Pace-Schott, and Robert Stickgold. 

5. Is this lucid dreaming?

   A lucid dream is any dream in which the dreamer is aware of dreaming. In that sense, yes, this is lucid dreaming. But the popular culture reference to "lucid dreaming" usually refers to regaining lucidity in much later stage REM sleep many hours into sleep, not the hypnagogia that Dormio users experience within minutes after sleep onset. When we tried later-stage lucid dreaming, we found a few challenges that made it impractical for interface building. Later-stage REM lucid dreaming requires a full night of sleep, not just a nap like hypnagogia: it carries the risk of, and often requires, waking you up in the middle of the night and disturbing your sleep schedule; it involves full atonia unlike hypnagogia, so people cannot describe their dreams without exiting them fully; and lastly,  it's just really hard to do. More subtly, we were interested in a state that was semi-lucid, not fully lucid like late stage lucid dreaming. In our experience of this state, you are fully in control of your cognition in a way that felt less spontaneous than hypnagogic microdreams. In hypnagogia, you often feel like a third-person observer of the machinations of your mind. We personally found the latter more interesting for creative idea generation. Late-stage lucid dreaming is still super cool, important to the sciences, useful for self-exploration to so many people, and really fun. You should try it! 
   

6. Is this dreaming? What is stage 1 sleep?

   There is a healthy debate around that! Hypnagogia, the transition from sleep to wake,  is variably called dreaming, lucid dreaming, hallucinating, or micro dreaming. Hypnagogia begins in stage 1 sleep, and as it is transitional, many think it carries over a bit into early stage 2 sleep. This is also subject to debate! If you're interested in a classification of dreaming's core phenomenology and the placement of hypnagogia along this spectrum, please check out Tore Nielsen's 2017 Paper on Microdream Neurophenomenology.

7. What's new here, compared to the Steel Ball Technique and past sleep neuroscience work?

   The Steel Ball technique is inspiring but limited. It only allows for one round of hypnagogia, can only intervene at one threshold (loss of muscle control), requires that people wake themselves up fully to record any dream content, relies only on coarse atonia signal, offers no quantified self biosignal recording, and doesn't facilitate dream theme control. Dormio improves granularity of tracking, adding capabilities for detection based on electrodermal activity, heart-rate variability, and muscle tension. We tie each of these to adjustable thresholds in software, enabling multimodal forms of feedback at pre-determined biosignal levels which each tie to unique experiences in hypnagogia. We tailor our interruption such that users cannot pass into stage 2 sleep, but are not startled fully awake, allowing multiple rounds of hypnagogia. We initiate automatic audio recording alongside interruption, allowing users to speak their dreams in semi-lucid states, not waking themselves up entirely to issue reports. We use this audio interruption for inception, as our experience thus far has indicated audio triggered in this threshold state reliably enters dream content. This handworn device is informed by past sleep neuroscience work, but aims to make it relevant outside the laboratory setting. Right now sleep tracking is done via polysomnography, both unpleasant to wear and hugely expensive. We are pushing past both of these constraints, by making hardware and software tailored to specific stages and unique interactions. This has allowed us to make cheap, wearable, simple tracking and intervention systems. We are building more of these, for more parts of the body! 

8. Can I have one?

   The technology that drives Dormio is open source. That means the software for our biosignal tracking is on Github. And our Eagle file for circuit board design is online too. And Tomás, who led that build, wrote about it step by step here. You can, for now, build yourself one. We'd like to get it to the point where we make enough for you. That hasn't happened yet. 

9. I don't have a robot!

   That's fine. This all works on a cell phone too. We've built both an Android and iOS app.

10. Can this be used for evil/authoritarian mind control?

    This question is appreciated, as are any other ethical issues we might be overlooking. But right now, no. This technology is at the beginning stages, with much perfecting still necessary. But more importantly, in hypnagogia, the subjects we've worked with are not entirely asleep (and not entirely awake), making them much less vulnerable than most people typically assume when they first learn about this project. They're aware that they are in an experiment room, though that awareness drifts in and out. Most, but not all, remember what they said throughout the experiment. But critically, we have had people forcibly wake themselves up when they had a weird enough dream that they did not want us experimenters to hear about it. People are monitoring their environment and aware of their descent into sleep,  limiting the capacity for inserting any ideas people don't want inserted, or extracting ideas they don't want extracted. All that being said, we don't dismiss these concerns at all. Encouraging people to dream about certain subjects may change how they consider those subjects after waking up. And we would not discount some diminished capacity for resistance to new ideas presented in hypnagogia. These are things to keep in mind, though it is also worth acknowledging that hypnagogia has been known about and used for hundreds of years (Edison, etc.) and not used to nefarious ends, as far as we are aware. Please get in touch if you think we're missing anything.

11. Where has this work been shown?

    A mix of spaces! At the CHI 2018 Conference, at the Science of Consciousness 2018 Conference, and at the National Academy of Sciences as part of the Sackler Fellows symposium on Cybernetic Serendipity. In less traditional spaces, this project has been shown in MIT's Museum Studio, covered in Studio International, and shown on ABC's 60 Minutes.
    

12. What are the next projects you're planning?

    We have lots of work to do! While we're inspired by the results with the ~15 people who have used Dormio so far, we need to increase testing numbers to be sure of the system's capabilities. Secondarily, we want to test whether content we incept into hypnagogia microdreams transfers into later stage REM dream content. Thirdly, we want to test the memory, learning, and emotion regulation effect of incepted hypnagogic content as a method of targeted reactivation. 

13. What else are you all building over there?

    A lot!