Sometimes you want to be able to see things that aren't visible. You may wonder what path a burrowing animal is taking underground, or how long it is going to take that cheeseburger you just ate to work its way through your body. An athlete may be interested in the state of their muscles, tendons, or bones during training, and a roboticist may want to use this same information to control an exoskeleton or a prosthetic limb. Perhaps, an augmented reality (AR) user may want their AR headset to track complicated finger or figurine movements without a line of sight, such as hand signs, pen movements, or an arrangement of action figures or building blocks. So where are we at with technology that tracks objects you can't see?

There are various methods for tracking otherwise invisible objects, such as x-ray, ultrasound, and radar. Most of these techniques can only track objects with a long time delay (from a large fraction of a second to more than a minute) or using a bulky device (making it difficult to carry around). In the 1950's, however, a new tracking technology was invented. People began using magnets to watch ingestible pills move through the body. They realized that all magnets are surrounded by a similar pattern of magnetic fields. Using the same sensors as in your smartphone's compass, the magnetic fields can be measured at several locations, and the magnetic field pattern can be used to determine the locations of one or more magnets. Conveniently, magnetic fields are not affected by materials such as wood, plastic, rubber, or the human body, so objects with magnets attached can be tracked through any of these materials.

Computers are much faster today, but it still takes a noticeable time (40 to 200 thousandths of a second) to track magnets using traditional methods. This means that if a technology needs to know where magnets are within a hundredth of a second, these methods aren't fast enough. For some applications this isn't an issue. For instance, when tracking how quickly food moves through the body, magnetic field data can be saved on the computer hundreds of times per second and the location of the magnets can be calculated later, after all of the data has been collected. But for other technologies, the tracking delay is critical. Imagine driving a car where the brake pedal and steering wheel both have a half-second delay. Driving would be exhausting, not to mention dangerous. Many technologies require a similarly fast response time.

In light of this need for a fast response time, we recently developed an improved method of tracking magnets. This method extends magnet tracking technology to new high-speed applications. But to understand how we did it, you need to understand how magnet tracking works. Traditionally, magnet tracking is done using a strategy called mathematical optimization. Mathematical optimization is essentially the Warmer/Colder game in computer code. In the Warmer/Colder game, a friend guides you to find a hidden object by saying “warmer” when you are closer to the object and “colder” when you are farther away. Just like your sister guiding you to the salt shaker in the cupboard ("warmer..., warmer..."), the computer is making guesses about the magnet locations and checking to see how closely it predicts the measured magnetic fields. When the error in the magnetic field prediction drops ("hot, hot, really hot!"), the computer moves its guesses in the direction of smaller errors.

But finding the salt shaker becomes easier (though arguably less amusing) if your sister is willing to provide additional information (“left, up, farther back…”). So what if there was a way for the computer to know what direction was "warmest" before it moved its guess? For some mathematical equations you can directly calculate this direction, saving some unnecessary wandering. This is a standard mathematical optimization trick, and the direction the error drops the fastest is called the gradient. In working to decrease the time delay of magnet tracking, we realized that not only could we derive these equations for the gradient, but that this gradient could be calculated very quickly using a few simple computer coding tricks.

This strategy allowed us to track magnets faster than has ever been reported, but there was still one more issue. Many high-speed tracking applications need to be portable. But if you hold a compass and turn around, you'll notice that the effect of the earth's magnetic field on the compass changes as you move. This magnetic field combines with the magnetic fields from the magnets you are tracking and makes it harder for the computer to find the magnets. So how can you compensate for the magnetic field from the earth? You could attach your sensors to a table so that the field doesn't change, or you could put the sensors in a room fully shielded by steel, but neither of these options allow portability. Alternatively, you could place an extra sensor far away from the other sensors, but this makes your system much larger and more complicated.

In search for a more compact solution, we found that the problem could instead be solved with a change to the computer software. If all of the sensors are close enough to each other, they will see the same magnetic field. As the computer searches for its best guess of the magnet locations, it can also be programmed to search for this common magnetic field. The disturbance from the earth's magnetic field, then, is no longer noise, but another signal to be tracked. And, fortunately for those who want to track magnets portably, tracking the earth’s magnetic field along with the magnets is still remarkably fast. To compensate for magnetic field disturbances in high-speed magnet tracking, we simply expanded the gradient to include the relationship between the tracking guesses and errors. In this case, these elements are so simple that they can be hard-coded into the gradient, allowing disturbance compensation with very little added delay.

So now the world has multi-magnet tracking that is fast and portable. This technology is currently limited by the accuracy of magnetic field sensors, but we're confident that magnet tracking is now in a position to change the way we interact with machines. Picture a future where sensors can track the movements of every muscle and tendon in your body. These movements can be used to control your phone, your environment, and an exoskeleton, giving you superhuman strength, endurance, and power that feels like it's a part of you. And all you have to do is flex.