A Suite of Techniques for Describing Activity in Terms of Events

Abstract

This report presents a set of software techniques that support the tasks of event recognition, summarization of event sequences, explanation of recognized events, explanation of non-recognized events, prediction of event completions, and question answering by leveraging language-encoded human knowledge of what typically happens during various types of events. The techniques operate on sequences of timestamped, three-dimensional positions and contacts for humans, body parts, and objects, provided by a Microsoft Kinect sensor plus associated software. Appendices describe 64 activity sequences used for development and testing of the techniques and 102 event models created as part of the effort.