What is Manga?
Manga is a pictorial summary of video named after a Japanese word for
"comic book". A video is automatically analyzed and represented with
different-sized keyframes packed in a visually pleasing form reminiscent of a comic book.
Video Manga allows users to get a quick overview of a video's contents at a glance without
watching the video from beginning to end. The visual summaries are suitable for printing,
and they can also be used to help users browse through videos.
To generate a Manga, a video is segmented based on the color features of each frame.
The segments are also clustered according to their similarities. We have introduced an
importance score to rank the segments. A segment is considered to be important if
it is long and rare. Keyframes are extracted from highly ranked segments and sized
according to their scores so that more important keyframes are presented as bigger frames.
Our frame-packing algorithm puts the different-sized keyframes in a compact "comic
The visual summary can be enhanced by adding captions to its frames. Text data for
captions may be retrieved from transcripts of the video or embedded closed-caption data.
Manga can also handle caption data manually typed in by users.
Manga on the Web
We have implemented an interactive version of the pictorial summary as an
abstract-level browsing tool for videos in our MBase system.
Moving the mouse over the displayed frames highlights the frame and the corresponding
segment in the timeline. This allows users to explore the temporal properties of a video.
Clicking on a keyframe starts video playback from the beginning of that segment. We have
also implemented a way to explore segments that are not presented at the top level of the
Generated from ISO MPEG 7 Content Set, Item V20, "Korea's
Pop Singers' Live Music Show", Korean Broadcasting System, 1998
Technical Contact: Andreas Girgensohn