3D interaction is a magical way for users to interact with a system without touching it. This is usually accomplished using a camera, but that can intrude heavily on privacy. Using RF, we can create interaction mechanisms that do not take pictures of us and help to preserve our privacy.
Here I discuss 2 projects, one is a through-table interaction mechanism, and the other is a through-wall interaction mechanism.
Through Table Interaction:
An invisible user interaction surface using an array of radio-frequency doppler sensors to capture the motion of a user’s arm and hand above a table. The 10GHz RF signal passes through obstructions like drywall and wood, enabling a camera-less, hidden-device surface. Demo at the MIT Media Lab Pavillion at Siggraph 2015. See here
The key piece behind this is a set of 4 HB100 RF motion detectors mounted underneath the table. Each one measures the velocity of objects which are in front of it by recording the doppler shift caused by moving reflectors. The transmitted frequency is 10.2 GHz, so the signal easily passes through the wooden table. The reflected signal is then mixed (multiplied) with the originally transmitted signal. Any velocity toward or away from the sensor will be captured as a <100Hz sinusoid, with faster movements resulting in higher frequency sine waves. Furthermore, the amplitude of the wave correlates with the distance of objects from the transmitter. This can be sampled by an ADC.
The 4 HB100’s are connected to an ADS1120 ADC/amplifier which is sampled via Matlab and connected to a game running in Unity. The files can be found below, and more details are to follow.
Through Wall Interaction:
Radio-Frequency based hand-tracking with 5cm accuracy. Demo at CVPR 2015 and UIST 2015.
Hisham Bedri, Otkrist Gupta, Andrew Temme, Micha Feigin, Gregory L. Charvat, Ramesh Raskar: RFlow: User Interaction Beyond Walls. UIST (Adjunct Volume) 2015: 45-46
PDF: https://dl.dropboxusercontent.com/u/110033260/projects/blog_files/p45-bedri.pdf