Representing the Content of Video: Artifact or Process?

Abstract

An approach to semantics based on traditional paradigms of knowledge representation (e.g., developing reductionist models of video document genres), while not entirely of the mark, may be significantly misdirected. Understanding the semantics of video and multimedia must begin with understanding how video (and film) are “read” and “written.” The purpose of this paper is to set an agenda for coming to an understanding of reading and writing multimedia and to address the representationalist implications of achieving that end. Further, we illustrate this research agenda, showing how we have applied concepts and methods from film theory (to the reading process) and interaction analysis (to the work of multimedia production), and what our preliminary findings might mean for computational support and knowledge representation in multimedia.