Multimedia Capture and Access

Tools for capturing and reusing siloed knowledge

Work tools are increasingly fragmented. You might teleconference with a consulting group using Skype in the morning, brainstorm ideas on a whiteboard with an internal team in the afternoon, and teleconference again with another team in a country halfway around the world with Google Hangouts in the evening, all while chatting with local and remote team members throughout the day on Slack. While this diversity of tools can help you expand the scope of your collaboration, it can also silo your work data, making it difficult to extract and integrate information across projects.

The goal of the Multimedia Capture and Access project is to provide tools that can capture information from a variety of platforms and activities to make it easier to reuse and synthesize knowledge.

This project spans activities focused on digital as well as physical document capture.

Effectively accessing captured multimedia information is an additional area of active exploration. Our current work focuses on educational video generally and massive open online courses (MOOCs) in particular. These courses are often constructed around a ‘one-size-fits-all’ prescribed curriculum which is not always well suited to the massive diversity of learner profiles involving different skill levels, interests, and expectations. Aggregating information across siloed platforms (e.g., Coursera, edX, …) can enable greater flexibility.

We are developing a unified topic-based representation to generate recommended sequences of video segments. Recommendations exploit both instructors’ course structures and prevalent topic transitions. Also, topic representations can be used to provide explanatory context for recommendations. Our aim is to integrate multimodal segment sequences with strong facets for diversity in recommendation.

Technical Contact

Related Publications

2017
Publication Details
  • TRECVID Workshop
  • Mar 1, 2017

Abstract

Close
This is a summary of our participation in the TRECVID 2016 video hyperlinking task (LNK). We submitted four runs in total. A baseline system combined on established vectorspace text indexing and cosine similarity. Our other runs explored the use of distributed word representations in combination with fine-grained inter-segment text similarity measures.
2016
Publication Details
  • ACM MM
  • Oct 15, 2016

Abstract

Close
The proliferation of workplace multimedia collaboration applications has meant on one hand more opportunities for group work but on the other more data locked away in proprietary interfaces. We are developing new tools to capture and access multimedia content from any source. In this demo, we focus primarily on new methods that allow users to rapidly reconstitute, enhance, and share document-based information.
Publication Details
  • Document Engineering DocEng 2016
  • Sep 13, 2016

Abstract

Close
In this paper we describe DocuGram, a novel tool to capture and share documents from any application. As users scroll through pages of their document inside the native application (Word, Google Docs, web browser), the system captures and analyses in real-time the video frames and reconstitutes the original document pages into an easy to view HTML-based representation. In addition to regenerating the document pages, a DocuGram also includes the interactions users had over them, e.g. mouse motions and voice comments. A DocuGram acts as a modern copy machine, allowing users to copy and share any document from any application.
Publication Details
  • Mobile HCI 2016
  • Sep 6, 2016

Abstract

Close
Most teleconferencing tools treat users in distributed meetings monolithically: all participants are meant to be connected to one another in the more-or-less the same manner. In reality, though, people connect to meetings in all manner of different contexts, sometimes sitting in front of a laptop or tablet giving their full attention, but at other times mobile and involved in other tasks or as a liminal participant in a larger group meeting. In this paper we present the design and evaluation of two applications, Penny and MeetingMate, designed to help users in non-standard contexts participate in meetings.