Christopher R. Wren, Alex Pentland

Abstract

This paper describes experiments in human motion understanding, defined here as estimation of the physical state of the body (the Plant) combined with interpretation of that part of the motion that cannot be predicted by the plant alone (the Behavior). The described behavior system operates in conjunction with a real-time, fully-dynamic, 3-D person tracking system that provides a mathematically concise formulation for incorporating a wide variety of physical constraints and probabilistic in- fluences. The framework takes the form of a non-linear recursive filter that enables pixel-level, probabilistic processes to take advantage of the contextual knowledge encoded in the higher-level models. Results are shown that demonstrate both qualitative and quantitative gains in tracking performance.