Video Hyperlinking and Recommendation

Modeling multimedia content to improve recommendation systems

We are building video recommendation systems that explore two directions of inquiry. First, for domains such as educational video (i.e. videos from massive open online courses) we explore how to build recommendation systems that model both the prominent topical structure of the content, and also account for the sequential nature of inter-topic relationships. Second, we explore how to use automatic understanding of the content’s topics to better contextualize recommendations provided to users, and also to improve their navigation experience within large content collections.

This project also includes participation in the TRECVID video hyperlinking benchmark in which we have successfully validated our content processing pipelines.

Technical Contact

Related Publications

Publication Details
  • Multimedia Modeling 2018
  • Feb 5, 2018


This paper examines content-based recommendation in domains exhibiting sequential topical structure. An example is educational video, including Massive Open Online Courses (MOOCs) in which knowledge builds within and across courses. Conventional content-based or collaborative filtering recommendation methods do not exploit courses' sequential nature. We describe a system for video recommendation that combines topic-based video representation with sequential pattern mining of inter-topic relationships. Unsupervised topic modeling provides a scalable and domain-independent representation. We mine inter-topic relationships from manually constructed syllabi that instructors provide to guide students through their courses. This approach also allows the inclusion of multi-video sequences among the recommendation results. Integrating the resulting sequential information with content-level similarity provides relevant as well as diversified recommendations. Quantitative evaluation indicates that the proposed system, \textit{SeqSense}, recommends fewer redundant videos than baseline methods, and instead emphasizes results consistent with mined topic transitions.
Publication Details
  • TRECVID Workshop
  • Mar 1, 2017


This is a summary of our participation in the TRECVID 2016 video hyperlinking task (LNK). We submitted four runs in total. A baseline system combined on established vectorspace text indexing and cosine similarity. Our other runs explored the use of distributed word representations in combination with fine-grained inter-segment text similarity measures.