Philip DeCamp, Deb Roy

Abstract

Although the availability of large video corpora are on the rise, the value of these datasets remain largely untapped due to the difficulty of analyzing their contents. Automatic video analyses produce low to medium accuracy for all but the simplest analysis tasks, while manual approaches are prohibitively expensive. In the tradeoff between accuracy and cost, human-machine collaborative systems that synergistically combine approaches may achieve far greater accuracy than automatic approaches at far less cost than manual. This paper presents TrackMarks, a system for annotating the location and identity of people and objects in large corpora of multi-camera video. TrackMarks incorporates a user interface that enables a human annotator to create, review, and edit video annotations, but also incorporates tracking agents that respond fluidly to the users actions, processing video automatically where possible, and making efficient use of available computing resources. In evaluation, TrackMarks is shown to improve the speed of a multi-object tracking task by an order of magnitude over manual annotation while retaining similarly high accuracy