Publications

FXPAL publishes in top scientific conferences and journals.

2000

An Introduction to Quantum Computing for Non-Physicists.

Publication Details
  • ACM Computing Surveys, Vol. 32(3), pp. 300 - 335
  • Sep 1, 2000

Abstract

Close
Richard Feynman's observation that quantum mechanical effects could not be simulated efficiently on a computer led to speculation that computation in general could be done more efficiently if it used quantum effects. This speculation appeared justified when Peter Shor described a polynomial time quantum algorithm for factoring integers. In quantum systems, the computational space increases exponentially with the size of the system which enables exponential parallelism. This parallelism could lead to exponentially faster quantum algorithms than possible classically. The catch is that accessing the results, which requires measurement, proves tricky and requires new non-traditional programming techniques. The aim of this paper is to guide computer scientists and other non-physicists through the conceptual and notational barriers that separate quantum computing from conventional computing. We introduce basic principles of quantum mechanics to explain where the power of quantum computers comes from and why it is difficult to harness. We describe quantum cryptography, teleportation, and dense coding. Various approaches to harnessing the power of quantum parallelism are explained, including Shor's algorithm, Grover's algorithm, and Hogg's algorithms. We conclude with a discussion of quantum error correction.
Publication Details
  • In Multimedia Tools and Applications, 11(3), pp. 347-358, 2000.
  • Aug 1, 2000

Abstract

Close
In accessing large collections of digitized videos, it is often difficult to find both the appropriate video file and the portion of the video that is of interest. This paper describes a novel technique for determining keyframes that are different from each other and provide a good representation of the whole video. We use keyframes to distinguish videos from each other, to summarize videos, and to provide access points into them. The technique can determine any number of keyframes by clustering the frames in a video and by selecting a representative frame from each cluster. Temporal constraints are used to filter out some clusters and to determine the representative frame for a cluster. Desirable visual features can be emphasized in the set of keyframes. An application for browsing a collection of videos makes use of the keyframes to support skimming and to provide visual summaries.

Expanding a Tangible User Interface

Publication Details
  • In proceedings of DIS'2000, ACM Press, August 2000.
  • Aug 1, 2000
Publication Details
  • In Proceedings of IEEE International Conference on Multimedia and Expo, vol. III, pp. 1329-1332, 2000.
  • Jul 30, 2000

Abstract

Close
We describe a genetic segmentation algorithm for video. This algorithm operates on segments of a string representation. It is similar to both classical genetic algorithms that operate on bits of a string and genetic grouping algorithms that operate on subsets of a set. For evaluating segmentations, we define similarity adjacency functions, which are extremely expensive to optimize with traditional methods. The evolutionary nature of genetic algorithms offers a further advantage by enabling incremental segmentation. Applications include video summarization and indexing for browsing, plus adapting to user access patterns.
Publication Details
  • In Proceedings of the Genetic and Evolutionary Computation Conference, Morgan Kaufmann Publishers, pp. 666-673, 2000.
  • Jul 8, 2000

Abstract

Close
We describe a genetic segmentation algorithm for image data streams and video. This algorithm operates on segments of a string representation. It is similar to both classical genetic algorithms that operate on bits of a string and genetic grouping algorithms that operate on subsets of a set. It employs a segment fair crossover operation. For evaluating segmentations, we define similarity adjacency functions, which are extremely expensive to optimize with traditional methods. The evolutionary nature of genetic algorithms offers a further advantage by enabling incremental segmentation. Applications include browsing and summarizing video and collections of visually rich documents, plus a way of adapting to user access patterns.
Publication Details
  • In Japan Hardcopy 2000, The Annual Conference of the Imaging Society of Japan. 6/12 6/14 2000.
  • Jun 12, 2000
Publication Details
  • In Proceedings of Hypertext '00, ACM Press, pp. 244-245, 2000.
  • May 30, 2000

Abstract

Close
We describe a way to make a hypermedia meeting record from multimedia meeting documents by automatically generating links through image matching. In particular, we look at video recordings and scanned paper handouts of presentation slides with ink annotations. The algorithm that we employ is the Discrete Cosine Transform (DCT). Interactions with multipath links and paper interfaces are discussed.

Hypertext Interaction Revisited

Publication Details
  • In Proceedings of Hypertext '00, ACM Press, pp. 171-179, 2000
  • May 30, 2000

Abstract

Close
Much of hypertext narrative relies on links to shape a reader's interaction with the text. But links may be too limited to express ambiguity, imprecision, and entropy, or to admit new modes of participation short of full collaboration. We use an e-book form to explore the implications of freeform annotation-based interaction with hypertext narrative. Readers' marks on the text can be used to guide navigation, create a persistent record of a reading, or to recombine textual elements as a means of creating a new narrative. In this paper, we describe how such an experimental capability was created on top of XLibris, a next generation e-book, using Forward Anywhere as the hypernarrative. We work through a scenario of interaction, and discuss the issues the work raises
Publication Details
  • In RIAO'2000 Conference Proceedings, Content-Based Multimedia Information Access, C.I.D., pp. 637-648, 2000.
  • Apr 12, 2000

Abstract

Close
We present and interactive system that allows a user to locate regions of video that are similar to a video query. Thus segments of video can be found by simply providing an example of the video of interest. The user selects a video segment for the query from either a static frame-based interface or a video player. A statistical model of the query is calculated on-the-fly, and is used to find similar regions of video. The similarity measure is based on a Gaussian model of reduced frame image transform coefficients. Similarity in a single video is displayed in the Metadata Media Player. The player can be used to navigate through the video by jumping between regions of similarity. Similarity can be rapidly calculated for multiple video files as well. These results are displayed in MBase, a Web-based video browser that allows similarity in multiple video files to be visualized simultaneously.

Anchored Conversations. Chatting in the Context of a Document.

Publication Details
  • In CHI 2000 Conference Proceedings, ACM Press, pp. 454-461, 2000.
  • Mar 31, 2000

Abstract

Close
This paper describes an application-independent tool called Anchored Conversations that brings together text-based conversations and documents. The design of Anchored Conversations is based on our observations of the use of documents and text chats in collaborative settings. We observed that chat spaces support work conversations, but they do not allow the close integration of conversations with work documents that can be seen when people are working together face-to-face. Anchored Conversations directly addresses this problem by allowing text chats to be anchored into documents. Anchored Conversations also facilitates document sharing; accepting an invitation to an anchored conversation results in the document being automatically uploaded. In addition, Anchored Conversations provides support for review, catch-up and asynchronous communications through a database. In this paper we describe motivating fieldwork, the design of Anchored Conversations, a scenario of use, and some preliminary results from a user study.
Publication Details
  • In CHI 2000 Conference Proceedings, ACM Press, pp. 185-192, 2000.
  • Mar 31, 2000

Abstract

Close
This paper presents a method for generating compact pictorial summarizations of video. We developed a novel approach for selecting still images from a video suitable for summarizing the video and for providing entry points into it. Images are laid out in a compact, visually pleasing display reminiscent of a comic book or Japanese manga. Users can explore the video by interacting with the presented summary. Links from each keyframe start video playback and/or present additional detail. Captions can be added to presentation frames to include commentary or descriptions such as the minutes of a recorded meeting. We conducted a study to compare variants of our summarization technique. The study participants judged the manga summary to be significantly better than the other two conditions with respect to their suitability for summaries and navigation, and their visual appeal.

Beyond Bits: The Future of Quantum Information Processing.

Publication Details
  • IEEE Computer, pp. 38-45, January 2000.
  • Feb 1, 2000

Abstract

Close
Recently, physicists and computer scientists have realized that not only do our ideas about computing rest on only partly accurate principles, but they miss out on a whole class of computation. Quantum physics offers powerful methods of encoding and manipulating information that are not possible within a classical framework. The potential applications of these quantum information processing methods include provably secure key distribution for cryptography, rapid integer factoring, and quantum simulation.
1999
Publication Details
  • In Proceedings of GROUP '99 (Phoenix, AZ), ACM Press, 1999.
  • Nov 14, 1999

Abstract

Close
The development of tools to support synchronous communications between non-collocated colleagues has received considerable attention in recent years. Much of the work has focused on increasing a sense of co-presence between interlocutors by supporting aspects of face-to-face conversations that go beyond mere words (e.g. gaze, postural shifts). In this regard, a design goal for many environments is the provision of as much media-richness as possible to support non-collocated communication. In this paper we present results from our most recent interviews studying the use of a text-based virtual environment to support work collaborations. We describe how such an environment, though lacking almost all the visual and auditory cues known to be important in face-to-face conversation, has played an important role in day-to-day communication. We offer a set of characteristics we feel are important to the success of this text-only tool and discuss issues emerging from its long-term use.
Publication Details
  • In Proceedings of ACM Multimedia '99, Orlando, Florida, November 1999.
  • Oct 30, 1999

Abstract

Close
NoteLook is a client-server system designed and built to support multimedia note taking in meetings with digital video and ink. It is integrated into a conference room equipped with computer controllable video cameras, video conference camera, and a large display rear video projector. The NoteLook client application runs on wireless pen-based notebook computers. Video channels containing images of the room activity and presentation material are transmitted by the NoteLook servers to the clients, and the images can be interactively and automatically incorporated into the note pages. Users can select channels, snap in large background images and sequences of thumbnails, and write freeform ink notes. A smart video source management component enables the capture of high quality images of the presentation material from a variety of sources. For accessing and browsing the notes and recorded video, NoteLook generates Web pages with links from the images and ink strokes correlated to the video.
Publication Details
  • In Proceedings ACM Multimedia, (Orlando, FL) ACM Press, pp. 383-392, 1999.
  • Oct 30, 1999

Abstract

Close
This paper presents methods for automatically creating pictorial video summaries that resemble comic books. The relative importance of video segments is computed from their length and novelty. Image and audio analysis is used to automatically detect and emphasize meaningful events. Based on this importance measure, we choose relevant keyframes. Selected keyframes are sized by importance, and then efficiently packed into a pictorial summary. We present a quantitative measure of how well a summary captures the salient events in a video, and show how it can be used to improve our summaries. The result is a compact and visually pleasing summary that captures semantically important events, and is suitable for printing or Web access. Such a summary can be further enhanced by including text captions derived from OCR or other methods. We describe how the automatically generated summaries are used to simplify access to a large collection of videos.
Publication Details
  • In Proceedings of ACM Multimedia '99, pp. 77-80, Orlando, Florida, November 1999
  • Oct 30, 1999

Abstract

Close
This paper presents a novel approach to visualizing the time structure of music and audio. The acoustic similarity between any two instants of an audio recording is calculated and displayed as a two-dimensional representation. Similar or repeating elements are visually distinct, allowing identification of structural and rhythmic characteristics. Visualization examples are presented for orchestral, jazz, and popular music. Applications include content-based analysis and segmentation, as well as tempo and structure extraction.

Tools for Quantum Algorithms

Publication Details
  • Int.J.Mod.Phys. C10 (1999) 1347-1362
  • Oct 29, 1999

Abstract

Close
We present efficient implementations of a number of operations for quantum computers. These include controlled phase adjustments of the amplitudes in a superposition, permutations, approximations of transformations and generalizations of the phase adjustments to block matrix transformations. These operations generalize those used in proposed quantum search algorithms.
Publication Details
  • In Proceedings of the Second International Workshop on Cooperative Buildings (CoBuild'99). Lecture Notes in Computer Science, Vol. 1670 Springer-Verlag, pp. 79-88, 1999.
  • Oct 1, 1999

Abstract

Close
We describe a media enriched conference room designed for capturing meetings. Our goal is to do this in a flexible, seamless, and unobtrusive manner in a public conference room that is used for everyday work. Room activity is captured by computer controllable video cameras, video conference cameras, and ceiling microphones. Presentation material displayed on a large screen rear video projector is captured by a smart video source management component that automatically locates the highest fidelity image source. Wireless pen-based notebook computers are used to take notes, which provide indexes to the captured meeting. Images can be interactively and automatically incorporated into the notes. Captured meetings may be browsed on the Web with links to recorded video.
Publication Details
  • In Human-Computer Interaction INTERACT '99, IOS Press, pp. 458-465, 1999.
  • Aug 30, 1999

Abstract

Close
In our Portholes research, we found that users needed to have a sense of being in public and to know who can see them (audience) and who is looking currently at them (lookback). Two redesigns of the Portholes display present a 3D theater view of the audience. Different sections display core team members, non-core team members and lookback. An experiment determined that people have strong preferences about audience information and how it should be displayed. Layout preferences are varied, but unfolding techniques and cluster analysis reveal that these preference perspectives fall into four groups of similar preferences.
Publication Details
  • In Human-Computer Interaction INTERACT '99, IOS Press, pp. 205-212, 1999.
  • Aug 30, 1999

Abstract

Close
When reviewing collections of video such as recorded meetings or presentations, users are often interested only in an overview or short segments of these documents. We present techniques that use automatic feature analysis, such as slide detection and applause detection, to help locate the desired video and to navigate to regions of interest within it. We built a web-based interface that graphically presents information about the contents of each video in a collection such as its keyframes and the distribution of a particular feature over time. A media player is tightly integrated with the web interface. It supports navigation within a selected file by visualiz-ing confidence scores for the presence of features and by using them as index points. We conducted a user study to refine the usability of these tools.

From Reading to Retrieval: Freeform Ink Annotations as Queries

Publication Details
  • In Proceedings of ACM SIGIR 99, ACM Press, pp. 19-25, 1999.
  • Aug 15, 1999

Abstract

Close
User interfaces for digital libraries tend to focus on retrieval: users retrieve documents online, but then print them out and work with them on paper. One reason for printing documents is to annotate them with freeform ink while reading. Annotation can help readers to understand documents and to make them their own. In addition, annotation can reveal readers' interests with respect to a particular document. In particular, it is possible to construct full-text queries based on annotated passages of documents. We describe an experiment that tested the effectiveness of such queries, as compared to relevance feedback query techniques. For a set of TREC topics and documents, queries derived from annotated passages produced significantly better results than queries derived from subjects' judgments of relevance.

Introducing a Digital Library Reading Appliance into a Reading Group.

Publication Details
  • In Proceedings of ACM Digital Libraries 99, ACM Press, pp. 77-84, 1999.
  • Aug 11, 1999

Abstract

Close
How will we read digital library materials? This paper describes the reading practices of an on-going reading group, and how these practices changed when we introduced XLibris, a digital library reading appliance that uses a pen tablet computer to provide a paper-like interface. We interviewed group members about their reading practices, observed their meetings, and analyzed their annotations, both when they read a paper document and when they read using XLibris. We use these data to characterize their analytic reading, reference use, and annotation practices. We also describe the use of the Reader's Notebook, a list of clippings that XLibris computes from a reader's annotations. Implications for digital libraries stem from our findings on reading and mobility, the complexity of analytic reading, the social nature of reference following, and the unselfconscious nature of readers' annotations.

Palette: A Paper Interface for Giving Presentations.

Publication Details
  • In Proceeding of the CHI 99 Conference on Human Factors in Computing Systems, ACM Press, pp. 354-361, 1999.
  • May 18, 1999

Abstract

Close
The Palette is a digital appliance designed for intuitive control of electronic slide shows. Current interfaces demand too much of our attention to permit effective computer use in situations where we can not give the technology our fullest concentration. The Palette uses index cards that are printed with slide content that is easily identified by both humans and computers. The presenter controls the presentation by directly manipulating the cards. The Palette design is based on our observation of presentations given in a real work setting. Our experiences using the system are described, including new practices (e.g., collaborative presentation, enhanced notetaking) that arise from the affordances of this new approach. This system is an example of a new interaction paradigm called tacit interaction that supports users who can spare very little attention to a computer interface.

NotePals: Lightweight Note Sharing by the Group, for the Group.

Publication Details
  • In Proceeding of the CHI 99 Conference on Human Factors in Computing Systems, ACM Press, pp. 338-345, 1999.
  • May 18, 1999

Abstract

Close
NotePals is a lightweight note sharing system that gives group members easy access to each other's experiences through their personal notes. The system allows notes taken by group members in any context to be uploaded to a shared repository. Group members view these notes with browsers that allow them to retrieve all notes taken in a given context or to access notes from other related notes or documents. This is possible because NotePals records the context in which each note is created (e.g., its author, subject, and creation time). The system is "lightweight" because it fits easily into group members' regular note-taking practices, and uses informal, ink-based user interfaces that run on portable, inexpensive hardware. In this paper we describe NotePals, show how we have used it to share our notes, and present our evaluations of the system.

Face-to-Face Interfaces.

Publication Details
  • In CHI 99 Extended Abstracts, ACM Press, pp. 244-245, 1999.
  • May 18, 1999

Abstract

Close

Recent work on the social nature of human-computer interactions [3] has prompted research on animated, anthropomorphic characters in user interfaces. Such interfaces may simplify user interactions by allowing them to use and interpret natural face-to-face communication techniques such as speech, gestures and facial expressions. We describe our initial implementation, a character that controls the A/V facilities in a state-of-the-art conference room, and outline the goals of our ongoing project.

Printertainment: Printing With Interactive Cover Sheets.

Publication Details
  • In CHI 99 Extended Abstracts, ACM Press, pp. 240-241, 1999.
  • May 18, 1999

Abstract

Close
We explored a new type of user interface, interactive cover sheets: computer forms laid out on the banner pages of print jobs that people can mark on, scan back into a multifunction printer/scanner, and use as input to applications. Cover sheets are commonly strewn around printer rooms; with interactivity, they can let people see what others have to say, add their own comments, or play games, all while waiting for their print jobs. We designed three prototype applications and deployed them briefly in our research lab. We found that interactive cover sheets can be very appealing, that the sheets must be designed so that people can still identify these pages as cover sheets, and that the slow interaction cycle favors asynchronous applications.
Publication Details
  • In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (Phoenix, AZ), vol. 6, pp. 3041-3044, 1999.
  • Mar 14, 1999

Abstract

Close
This paper presents methods of generating compact pictorial summarizations of video. By calculating a measure of shot importance video can be summarized by de-emphasizing or discarding less important information, such as repeated or common scenes. In contrast to other approaches that present keyframes for each shot, this measure allows summarization by presenting only the most important shots. Selected keyframes can also be resized depending on their relative importance. We present an efficient packing algorithm that constructs a pictorial representation from differently-sized keyframes. This results in a compact and visually pleasing summary reminiscent of a comic book.
Publication Details
  • In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (Phoenix, AZ), vol. 6, pp. 3453-3456, 1999.
  • Mar 14, 1999

Abstract

Close
This paper describes a technique for automatically creating an index for handwritten notes captured as digital ink. No text recog-nition is performed. Rather, a dictionary of possible index terms is built by clustering groups of ink strokes corresponding roughly to words. Terms whose distribution varies significantly across note pages are selected for the index. An index page containing the index terms is created, and terms are hyper-linked back to their original location in the notes. Further, index terms occurring in a note page are highlighted to aid browsing.
Publication Details
  • In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (Phoenix, AZ), vol. 6, pp. 3045-3048, 1999.
  • Mar 14, 1999

Abstract

Close
This paper describes techniques for classifying video frames using statistical models of reduced DCT or Hadamard transform coefficients. When decimated in time and reduced using truncation or principal component analysis, transform coefficients taken across an entire frame image allow rapid modeling, segmentation, and similarity calculation. Unlike color-histogram metrics, this approach models image composition and works on grayscale images. Modeling the statistics of the transformed video frame images gives a likelihood measure that allows video to be segmented, classified, and ranked by similarity for retrieval. Experiments are presented that show an 87% correct classification rate for different classes. Applications are presented including a content-aware video browser.
Publication Details
  • In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (Phoenix, AZ), vol. 6, pp. 3029-3032, 1999.
  • Mar 14, 1999

Abstract

Close
This paper describes a method for finding segments in video-recorded meetings that correspond to presentations. These segments serve as indexes into the recorded meeting. The system automatically detects intervals of video that correspond to presentation slides. We assume that only one person speaks during an interval when slides are detected. Thus these intervals can be used as training data for a speaker spotting system. An HMM is automatically constructed and trained on the audio data from each slide interval. A Viterbi alignment then resegments the audio according to speaker. Since the same speaker may talk across multiple slide intervals, the acoustic data from these intervals is clustered to yield an estimate of the number of distinct speakers and their order. This allows the individual presentations in the video to be identified from the location of each presenter's speech. Results are presented for a corpus of six meeting videos.
Publication Details
  • In Proceedings of the International Joint Conference on Work Activities Coordination and Collaboration, pp. 147-156, 1999.
  • Feb 22, 1999

Abstract

Close
In many hierarchical companies, reports from several independent groups must be merged to form a single, company-wide report. This paper describes a process and system for creating and structuring such reports and for propagating contributions up the organization. The system has been in regular use, in-house, by about 30 users for over a year to create monthly status reports. Our experiences indicate that it is possible to change a monthly reporting practice so that the system is easy to use, improves the quality of the written report, fosters collaboration across projects and creates a corporate memory for the company. These results were achieved as a consequence of our design effort to directly support the hierarchical and collaborative process of creating and assembling the report within the organization. User feedback has led to many improvements in the usability and functionality of the system. Further enhancements using information retrieval and text summarization techniques are in progress.
Publication Details
  • In Proceedings of the Thirty-Second Annual Hawaii International Conference on System Sciences (HICSS-32), R. Sprague, Jr., editor, 1999.
  • Feb 5, 1999

Abstract

Close
This is a critical view of the hypothesis that better access to a broader repertoire of media resources will significantly enhance our ability to communicate more effectively. It begins by laying down a foundation of some basic principles concerning the nature of knowledge creation. This foundation is framed in a manner that involves the potential relevance of two particularly creative activities, storytelling and making jazz. This foundation provides the basis for a critical examination of several media-rich presentations that were delivered at the Institute for the Future Outlook Exchange in November of 1997, since these presentations actually pertained to the practices of digital storytelling and jamming. This critique is followed by a more detailed examination of what we may learn from jazz if we wish to invoke it as a metaphor for knowledge creation. The report then concludes by discussing the implications of these observations for a new world of work experiences in which knowledge creation is a critical element.
Publication Details
  • In Proceedings of the Thirty-Second Annual Hawaii International Conference on System Sciences (HICSS-32), R. Sprague, Jr., editor, 1999.
  • Feb 5, 1999

Abstract

Close
The genre of mathematics writing has several distinctive features that point to some of the weaknesses of current digital documents. Some of these weaknesses are surprising. While it might be expected that the importance of formatting and special symbols in mathematics writing would pose challenges for digital documents, the linked, chunked style of mathematics writing, with its Theorems, Lemmas, Corollaries and Remarks explicitly referring to each other, resembles standard hypertext so closely that one would expect that mathematics writing would take well to online hypertext form. It does not. This failure points to deficiencies in our understanding of the true strengths and weaknesses of digital documents. This paper describes mathematics writing, with particular emphasis on features of interest with respect to digital documents. The difficulties in producing effective digital mathematics documents are then examined and used as a basis for talking about general challenges for digital documents. The paper then discusses strengths of digital documents and some of the problems that need to be overcome before digital documents can live up to claims made for them. It also examines some of the misguided claims, such as superior support for non-linearity, that are commonly made for digital documents, explains why these claims are unwarranted, and speculates on why the claims have been made anyway. Suggestions are then given as to what the true benefits of digitization are, including performing computations on text, flexible control of time, and better support for hiding information. The paper concludes with a list of questions whose answers are critical to understanding the capabilities, and therefore the future, of digital documents.

Distributed Research Teams: Meeting Asynchronously in Virtual Space.

Publication Details
  • In Proceedings of the Thirty-second Annual Hawaii International Conference on System Sciences (Wailea, Hawaii, January 1999).
  • Feb 1, 1999

Abstract

Close
As computer networks improve, more social and work interactions are carried out "virtually" by geographically separated group members. In this paper we discuss the design of a tool, PAVE, to support remote work interactions among colleagues in different time zones. PAVE extends a 2D graphical MOO and supports synchronous and asynchronous interactions. PAVE logs and indexes activities in the space. This capture facility enables playback and augmentation of meeting interactions by non-collocated group members. Thus, members can participate asynchronously in meetings they could not attend in real time, not just review them.

Autonomous Synthetic Computer Characters as Personal Representatives.

Publication Details
  • In Human Cognition and Social Agent Technology, Kerstin Dautenhahn (Guest-editor), Advances in Consciousness Research Series. John Benjamins Publishing Company.
  • Feb 1, 1999
Publication Details
  • In The Computer Journal, 42 (6), pp. 534-546, 1999.
  • Feb 1, 1999

Abstract

Close
The Digestor system automatically converts web-based documents designed for desktop viewing into formats appropriate for handheld devices with small display screens, such as Palm-PCs, PDAs, and cellular phones. Digestor employs a heuristic planning algorithm and a set of structural page transformations to produce the "best" looking document for a given display size. Digestor can also be instructed, via a scripting language, to render portions of documents, thereby avoiding navigation through many screens of information. Two versions of Digestor have been deployed, one that re-authors HTML into HTML for conventional browsers, and one that converts HTML into HDML for Unwired Planet's micro-browsers. Digestor provides a crucial technology for rapidly accessing, scanning and processing information from arbitrary web-based documents from any location reachable by wired or unwired communication.
Publication Details
  • In IEEE Multimedia Systems '99, IEEE Computer Society, vol. 1, pp. 756-761, 1999.
  • Feb 1, 1999

Abstract

Close
In accessing large collections of digitized videos, it is often difficult to find both the appropriate video file and the portion of the video that is of interest. This paper describes a novel technique for determining keyframes that are different from each other and provide a good representation of the whole video. We use keyframes to distinguish videos from each other, to summarize videos, and to provide access points into them. The technique can determine any number of keyframes by clustering the frames in a video and by selecting a representative frame from each cluster. Temporal constraints are used to filter out some clusters and to determine the representative frame for a cluster. Desirable visual features can be emphasized in the set of keyframes. An application for browsing a collection of videos makes use of the keyframes to support skimming and to provide visual summaries.

As We May Read: The Reading Appliance Revolution.

Publication Details
  • Computer, Vol. 32, No. 1, January 1999, pp. 65-73.
  • Feb 1, 1999

Abstract

Close
Reading appliances allow people to work on electronic documents much as they would on paper. They therefore provide an alternative to the standard "browse or search and then print" model of reading online. By integrating a wide variety of document activities, such as searching, organizing, and skimming, and by allowing fluid movement among them, reading appliances eliminate disruptive transitions between paper and digital media.

Collaborating over Portable Reading Appliances.

Publication Details
  • In Personal Technologies, Vol. 3, No. 1, 1999.
  • Feb 1, 1999

Abstract

Close
Reading appliances or e-books hold substantial promise to help us collaborate. In this paper, we use a study of a group activity - a reading group that meets to discuss articles of mutual interest - to explore four scenarios for collaborating with e-books: (1) meetings and face-to-face discussions; (2) serendipitous sharing of annotations, as when we borrow a document from a colleague or buy a used book; (3) community-wide use of anonymous annotations to guide future readers; and (4) e-books as a basis for initiating interaction between people. In so doing, we describe some methods for implementing these facilities, and introduce design guidelines.
1998
Publication Details
  • UIST '98, ACM Press, 1998, pp. 195-202.
  • Oct 31, 1998

Abstract

Close
In this paper, we describe a technique for dynamically grouping digital ink and audio to support user interaction in freeform note-taking systems. For ink, groups of strokes might correspond to words, lines, or paragraphs of handwritten text. For audio, groups might be a complete spoken phrase or a speaker turn in a conversation. Ink and audio grouping is important for editing operations such as deleting or moving chunks of ink and audio notes. The grouping technique is based on hierarchical agglomerative clustering. This clustering algorithm yields groups of ink or audio in a range of sizes, depending on the level in the hierarchy, and thus provides structure for simple interactive selection and rapid non-linear expansion of a selection. Ink and audio grouping is also important for marking portions of notes for subsequent browsing and retrieval. Integration of the ink and audio clusters provides a flexible way to browse the notes by selecting the ink cluster and playing the corresponding audio cluster.

A Framework for Sharing Handwritten Notes.

Publication Details
  • UIST '98, ACM Press, 1998, pp. 119-120.
  • Oct 31, 1998

Abstract

Close
NotePals is an ink-based, collaborative note taking application that runs on personal digital assistants (PDAs). Meeting participants write notes in their own handwriting on a PDA. These notes are shared with other participants by synchronizing later with a shared note repository that can be viewed using a desktop-based web browser. NotePals is distinguished by its lightweight process, interface, and hardware. This demonstration illustrates the design of two different NotePals clients and our web-based note browser.
Publication Details
  • MULTIMEDIA '98, ACM Press, 1998, pp. 375-380.
  • Sep 14, 1998

Abstract

Close
Many techniques can extract information from an multimedia stream, such as speaker identity or shot boundaries. We present a browser that uses this information to navigate through stored media. Because automatically-derived information is not wholly reliable, it is transformed into a time-dependent "confidence score." When presented graphically, confidence scores enable users to make informed decisions about regions of interest in the media, so that non-interesting areas may be skipped. Additionally, index points may be determined automatically for easy navigation, selection, editing, and annotation and will support analysis types other than the speaker identification and shot detection used here.

Digital Library Information Appliances.

Publication Details
  • In Proceedings of Digital Libraries 98 (Pittsburgh, PA June 23-26), ACM Press, 1998, pp. 217-226.
  • Jun 23, 1998

Abstract

Close
Although digital libraries are intended to support education and knowledge work, current digital library interfaces are narrowly focused on retrieval. Furthermore, they are designed for desktop computers with keyboards, mice, and high-speed network connections. Desktop computers fail to support many key aspects of knowledge work, including active reading, free form ink annotation, fluid movement among document activities, and physical mobility. This paper proposes portable computers specialized for knowledge work, or digital library information appliances, as a new platform for accessing digital libraries. We present a number of ways that knowledge work can be augmented and transformed by the use of such appliances. These insights are based on our implementation of two research prototype systems: XLibris,™ an "active reading machine," and TeleWeb, a mobile World Wide Web browser.

Linking By Inking: Trailblazing in a Paper-like Hypertext

Publication Details
  • In Proceedings of Hypertext '98 (Pittsburgh, PA), ACM Press, 1998, pp. 30-39.
  • Jun 20, 1998

Abstract

Close
"Linking by inking" is a new interface for reader-directed link construction that bridges reading and browsing activities. We are developing linking by inking in XLibris,™ a hypertext system based on the paper document metaphor. Readers use a pen computer to annotate page images with free-form ink, much as they would on paper, and the computer constructs hypertext links based on the ink marks. This paper proposes two kinds of reader-directed links: automatic and manual. Automatic links are created in response to readers' annotations. The system extracts the text near free-form ink marks, uses these terms to construct queries, executes queries against a collection of documents, and unobtrusively displays links to related documents in the margin or as "further reading lists." We also present a design for manual (ad hoc) linking: circling an ink symbol generates a multi-way link to other instances of the same symbol.
Publication Details
  • Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (Seattle, WA), Vol. 6, 1998, pp. 3741-3744.
  • May 12, 1998

Abstract

Close
This paper describes a technique for segmenting video using hidden Markov models (HMM). Video is segmented into regions defined by shots, shot boundaries, and camera movement within shots. Features for segmentation include an image-based distance between adjacent video frames, an audio distance based on the acoustic difference in intervals just before and after the frames, and an estimate of motion between the two frames. Typical video segmentation algorithms classify shot boundaries by computing an image-based distance between adjacent frames and comparing this distance to fixed, manually determined thresholds. Motion and audio information is used separately. In contrast, our segmentation technique allows features to be combined within the HMM framework. Further, thresholds are not required since automatically trained HMMs take their place. This algorithm has been tested on a video data base, and has been shown to improve the accuracy of video segmentation over standard threshold-based systems.

Animated Autonomous Personal Representatives.

Publication Details
  • In Proceedings of the Second International Conference on Autonomous Agents (Minneapolis, MN), 1998, pp 8-15.
  • May 9, 1998

Abstract

Close
We describe the research goals and issues in constructing autonomous personal representatives, and the desirability of using synthetic characters as the user interface for such artifacts. An application of these autonomous representatives is then described in which characters can be attached to a document to express a user's point of view or give guided tours or presentations of the document's contents.

XLibris: The Active Reading Machine.

Publication Details
  • CHI 98 Summary, ACM Press, 1998, pp. 22-23.
  • Apr 18, 1998

Abstract

Close

Active reading is the combination of reading with critical thinking and learning, and involves not just reading per se, but also underlining, highlighting and commenting. We have built the XLibris™ "Active Reading Machine" to explore the premise that computation can enhance the active reading process. XLibris™ uses a high-resolution pen tablet display along with a paper-like user interface to emulate the physical experience of reading a document on paper: the reader can hold a scanned image of a page in his lap and mark on it with digital ink. XLibris™ monitors free-form ink annotations made while reading, and uses these to organize and to search for information. Readers can review, sort and filter clippings of their annotated text in a "Reader's Notebook." Finally, XLibris™ searches for material related to the annotated text, and displays links unobtrusively in the margin. XLibris™ demonstrates that computers can help active readers organize and find information while retaining many of the advantages of reading on paper.

The Rise of Personal Web Pages at Work.

Publication Details
  • CHI 98 Summary, ACM Press, 1998, pp. 313-314.
  • Apr 18, 1998

Abstract

Close
A series of 20 interviews in four organizations explores the ways in which employees take advantage of personal web pages to support their work and to reflect who they are. Both interviewee comments and web page examples suggest the importance of individual personalizations of information management and dissemination, presentation and perception of personality, and usage from the reader's perspective. These results can inform the development of future web technologies for use in organizations. Furthermore, this self representation on web pages is a way of making individual knowledge more available in the workplace.
Publication Details
  • CHI 98 Summary, ACM Press, 1998, pp. 141-142.
  • Apr 18, 1998

Abstract

Close
The World Wide Web is often viewed as the latest and most user friendly way of providing information over the Internet (i.e., server of documents). It is not customarily viewed as a platform for developing and deploying applications. In this tutorial, we introduce, demonstrate, and discuss how Web technologies like CGI scripts, Javascript, and Java can be used in combination with Web browsers to design, create, distribute and execute collaborative applications. We discuss constraints with the Web approach as well as recent extensions that support application development.

Beyond Paper: Supporting Active Reading with Free Form Digital Ink Annotations.

Publication Details
  • In CHI 97 Extended Abstracts, ACM Press, 1997, pp. 22-23.
  • Apr 18, 1998

Abstract

Close
Reading frequently involves not just looking at words on a page, but also underlining, highlighting and commenting, either on the text or in a separate notebook. This combination of reading with critical thinking and learning is called active reading [2]. To explore the premise that computation can enhance active reading we have built the XLibris™ "active reading machine." XLibris™ uses a commercial high-resolution pen tablet display along with a paper-like user interface to support the key affordances of paper for active reading: the reader can hold a scanned image of a page in his lap and mark on it with digital ink. To go beyond paper, XXLibris™ monitors the free-form ink annotations made while reading, and uses these to organize and to search for information. Readers can review, sort and filter clippings of their annotated text in a "Reader's Notebook." XLibris™ also searches for material related to the annotated text, and displays links to similar documents unobtrusively in the margin. XLibris™ demonstrates that computers can help active readers organize and find information while retaining many of the advantages of reading on paper.
Publication Details
  • CHI 98 Summary, ACM Press, 1998, pp. 283-284.
  • Apr 18, 1998

Abstract

Close
Peripheral awareness is a powerful human resource that has only recently been addressed in media space design. The challenge is to figure out what would be important to convey remotely and to strike a balance between too much and too little. Symbolic representation of remote activity is a powerful way to go, but as it turns out also easy to do wrong. This paper presents some early findings on problems and promises of using symbolic representation: it reports from informal studies of people using the AROMA prototype in regular office and home settings, and it conveys some lessons and designing appropriate and effective symbolic representations.

Meetings in a Virtual Space: Creating a Digital Document.

Publication Details
  • In Proceedings of the Thirty-first Annual Hawaii International Conference on System Sciences (Wailea, Hawaii, January 1998).
  • Feb 6, 1998

Abstract

Close
Improvements in computer network infrastructures and information utilities have led to an increase in the number of social and work interactions carried out 'virtually' by geographically separated group members [1, 5, 6, 7]. In this paper we describe the design and evaluation of a prototype system that supports synchronous and asynchronous collaboration between researchers separated by space and time. The system provides non-collocated team members with a digital, virtual space for information sharing and discussion. For synchronous interactions, our design prioritizes provision of shared context, real-time discourse, and real-time problem solving and negotiation between the team members. In the case of asynchronous interactions, we have prioritized the capture of team decision making and negotiation processes and the representation of these processes in a context-rich, hypertextual document of team problem solving and negotiation.
Publication Details
  • In Proceedings of the Thirty-first Annual Hawaii International Conference on System Sciences (Wailea, Hawaii, January 1998), Volume II, pp. 259-267.
  • Feb 6, 1998

Abstract

Close
In this paper we describe a method for indexing and retrieval of multimedia data based on annotation and segmentation. Our goal is the retrieval of segments of audio and video suitable for inclusion in multimedia documents. Annotation refers to the association of text data with particular time locations of the media. Segmentation is the partitioning of continuous media into homogenous regions. Retrieval is performed over segments of the media using the annotations associated with the segments. We present two scenarios that describe how these techniques might be applied. In the first, we describe how excerpts from a video-taped usage study of a new device are located for inclusion in a report on the utility of the device. In the second, we show how sound bites from a recorded meeting are obtained for use in authoring a summary of the meeting.

AESOP: An Outline-Oriented Authoring System.

Publication Details
  • In Proceedings of the Thirty-first Annual Hawaii International Conference on System Sciences (Wailea, Hawaii, January 1998), Volume II, pp. 207-215.
  • Feb 6, 1998

Abstract

Close
Because a hypermedia document is more complex than conventional text, it requires preparation with respect to two key aspects. First, the author begins to develop a "vision" of the document-usually based on some outline-level description of his objectives. At the same time, as this outline is being developed, the author begins to extract useful segments from his resource materials and prepares his first version of the logic of a system of hyperlinks among those segments. In this paper we present a system named "Authoring Environment for the deSktOP" (AESOP) with two different types of "outlining" tools to handle these aspects. Planning the "vision" consists in defining a "logical" tree structure of the document. The plan for the link structure is based on a primitive unit called the view area, and AESOP provides a construct named Bento-Box for creating and manipulating view areas. Authors specify spatial and temporal layout within a single Bento-Box and define hyperlinks among the Bento-Boxes.
1997
Publication Details
  • In Proceedings: VISual'97; Second International Conference on Visual Information Systems (San Diego, CA), 1997, pp. 53-60.
  • Dec 15, 1997

Abstract

Close
As the concept of what constitutes a "content-based" search grows more mature, it becomes valuable for us to establish a clear sense of just what is meant by "content." Recent multimedia document retrieval systems have dealt with this problem by indexing across multiple indexes; but it is important to identify how such multiple indexes are dealing with multiple dimensions of a description space, rather than simply providing the user with more descriptors. In this paper we consider a description space for multimedia documents based on three "dimensions" of a document, namely context, form, and content. We analyze the nature of this space with respect to three challenging examples of multimedia search tasks, and we address the nature of the index structures that would facilitate how these tasks may be achieved. These examples then lead us to some general conclusions on the nature of multimedia indexing and the intuitions we have inherited from the tradition of books and libraries.
Publication Details
  • In GROUP'97, Proceedings of the International ACM SIGGROUP Conference on Supporting Group Work, ACM Press, 1997, pp. 385-394.
  • Nov 16, 1997

Abstract

Close
The prevalence of audio and video options on computers, coupled with the promise of bandwidth, have many prognosticators predicting a revolution in human communications. But what if the revolution materializes and no users show up? We were confronted with this question when we began deploying and studying the use of a video-based, background awareness application within our organization. Repeatedly, new users raised strong concerns about self-presentation, surveillance, privacy, video snapshots, and lack of audience cues. We describe how we addressed these concerns by evolving the application. As a consequence, we are also redesigning the user interface to the application.
Publication Details
  • Computer Networks and ISDN Systems, 29(8-13), pp. 1531-1542
  • Sep 30, 1997

Abstract

Close
The phenomenal interest and growth of the World Wide Web as an application server has pushed the Web model to its limits. Specifically, the Web offers limited interactivity and versatility as a platform for networked applications. One major challenge for the HCI community is to determine how to improve the human-computer interface for Web-based applications. This paper focuses on a significant Web deficiency - supporting truly interactive and dynamic form-based input. We propose a well-worked form interaction abstraction that alleviates this Web deficiency. We describe how the abstraction is seamlessly integrated into the Web framework by leveraging on the virtues of the Web and fitting within the interaction and usage model of the Web.

Formal experiments in causal attire: Case studies in information exploration

Publication Details
  • New Review of Hypermedia and Multimedia . Vol 3. (1997), Taylor Graham. pp. 123-158.
  • Jun 1, 1997

Abstract

Close
This paper addresses the issue of how research methodology can be developed for the specific needs of research into information exploration behavior, based on a four year program of research on individual strategies in information exploration. We propose a meta-experimental framework where research is carried out through a dynamic interaction between what and why questions, and between confirmatory and exploratory analyses. This approach preserves many of the advantages of formal experimentation, while permitting a more holistic examination of phenomena that is characteristic of ethnography. The application of the meta-theoretical framework is illustrated in three case studies that examined new information exploration functionalities and interfaces and their relationship to expertise and exploration strategy.

The Newspaper as an Information Exploration Metaphor

Publication Details
  • Journal of Information Processing & Management.33(5) pp. 663-683.
  • Jun 1, 1997

Abstract

Close
The newspaper represents a mature information presentation medium that is well-suited to the presentation of relatively short, loosely related pieces of text. This work examines the implementation of the newspaper metaphor in an information exploration interface. Based on an analysis of differences between electronic books and electronic newspapers, we submit that the newspaper metaphor is an appropriate interface paradigm for large-scale full-text databases. Similarities between newspapers and hypertext databases lead us to suggest that this metaphor is appropriate for large automatically-generated hypertexts, independent of the nature of their content. We describe VOIR, a software prototype that we have used as an electronic newspaper workbench. The program constructs newspaper pages interactively, and allows users to specify their information-seeking intent in a variety of ways, including graphical Boolean queries, hypertext links, and typed-in queries. Finally, we discuss some implications that this work has for hypertext and information retrieval in general.

Signs, Links, and the Semiotics of Hypertext,

Publication Details
  • Proceedings: 1st International Workshop on Computational Semiotics (SEMIOTICS'97), 1997.
  • May 26, 1997

Abstract

Close
This paper examines the semiotic nature of the hypertext document from two points of view, both of which are based on Roland Barthes' ELEMENTS OF SEMIOLOGY. From the more conventional point of view, the hypertext document is discussed with respect to the four areas analyzed by Barthes: the distinction between language and speech, the relationship between the signifier and the signified, the syntagmatic and associative relationships among signs, and the hierarchical embedding of signs through connotation and metalanguage. The second examination is a hypertext document designed in such a way that the reader may EXPERIENCE Barthes' elements of semiology, rather than serve as a passive receiver of his exposition. The result is an interactive environment in its own right, entitled "Signs and Links," that, through the experience of interaction, engages the reader in the definition of the conceptual space of Barthes' text.

Digestor: Device-Independent Access to the World Wide Web.

Publication Details
  • In Proceedings for the Sixth International World Wide Web Conference, 1997, pp. 655-663.
  • Apr 7, 1997

Abstract

Close
Digestor is a software system which automatically re-authors arbitrary documents from the world-wide web to display appropriately on small screen devices such as PDAs and cellular phones, providing device-independent access to the web. Digestor is implemented as an HTTP proxy which dynamically re-authors requested web pages using a heuristic planning algorithm and a set of structural page transformations to achieve the best looking document for a given display size.
Publication Details
  • In Proceedings of Hypertext 97, ACM Press, 1997, pp. 67-74.
  • Apr 5, 1997

Abstract

Close
Traditionally hypertexts have been limited in size by the manual effort required to create hypertext links. In addition, large hyper-linked collections may overwhelm users with the range of possible links from any node, only a fraction of which may be appropriate for a given user at any time. This work explores automatic methods of link construction based on feedback from users collected during browsing. A full-text search engine mediates the linking process. Query terms that distinguish well among documents in the database become candidate anchors; links are mediated by passage-based relevance feedback queries. The newspaper metaphor is used to organize the retrieval results. VOIR, a software prototype that implements these algorithms has been used to browse a 74,500 node (250MB) database of newspaper articles. An experiment has been conducted to test the relative effectiveness of dynamic links and user-specified queries. Experimental results suggest that link-mediated queries are more effective than user-specified queries in retrieving relevant information. The paper concludes with a discussion of possible extensions to the linking algorithms.
Publication Details
  • In CHI 97 Conference Proceedings, ACM Press, 1997, pp. 186-193.
  • Mar 21, 1997

Abstract

Close
Dynomite is a portable electronic notebook for the capture and retrieval of handwritten and audio notes. The goal of Dynomite is to merge the organization, search, and data acquisition capabilities of a computer with the benefits of a paper-based notebook. Dynomite provides novel solutions in four key problem areas. First, Dynomite uses a casual, low cognitive overhead interface. Second, for content indexing of notes, Dynomite uses ink properties and keywords. Third, to assist organization, Dynomite's properties and keywords define views, presenting a subset of the notebook content that dynamically changes as users add new information. Finally, to augment handwritten notes with audio on devices with limited storage, Dynomite continuously records audio, but only permanently stores those parts highlighted by the user.

Sensing Activity in Video Images.

Publication Details
  • In CHI 97 Extended Abstracts, ACM Press, 1997, pp. 319-320.
  • Mar 21, 1997

Abstract

Close
Video-based awareness tools increase familiarity among remote group members and provide pre-communication information. Low-cost iconic indicators provide less but more succinct information than video images while preserving privacy. Observations of and feedback from users of our video awareness tool suggest that an activity sensing feature along with a variety of privacy options combines advantages of both the video images and iconic indicator approaches. We introduced the activity sensing feature in response to user requests. It derives activity information from video images and provides options to control privacy and improves the usability of video-based awareness tools.
Publication Details
  • In CHI 97 Extended Abstracts, ACM Press, 1997, pp. 22-23.
  • Mar 21, 1997

Abstract

Close
Dynomite is a portable electronic notebook that merges the benefits of paper note-taking with the organizational capabilities of computers. Dynomite incorporates four complementary features which combine to form an easy to use system for the capture and retrieval of handwritten and audio notes. First, Dynomite has a paper-like user interface. Second, Dynomite uses ink properties and keywords for content indexing of notes. Third, Dynomite's properties and keywords allows retrieval of specific ink and notes. The user is shown a view, or a subset of the notebook content, that dynamically changes as new information is added. Finally, Dynomite continuously records audio, but only permanently stores highlighted portions so that it is possible to augment handwritten notes with audio on devices with limited storage.
Publication Details
  • In CHI 97 Conference Proceedings, ACM Press, 1997, pp. 407-414.
  • Mar 21, 1997

Abstract

Close
Hypertext interfaces are considered appropriate for information exploration tasks. The prohibitively expensive link creation effort, however, prevents traditional hypertext interfaces from being used with large coherent collections of text. Such collections typically require query-based interfaces. This paper examines a hybrid approach: the system described here creates anchors dynamically based on users' queries, and uses anchor selection as a query expansion mechanism. An experiment was conducted to compare browsing behavior in query- and link-based interfaces. Results suggest that query-mediated links are as effective as explicit queries, and that strategies adopted by users affect performance. This work has implications for the design of information exploration interfaces; the dynamic link algorithms described here are being incorporated into a Web server.
Publication Details
  • In CHI 97 Conference Proceedings, ACM Press, 1997, pp. 550-551.
  • Mar 21, 1997

Abstract

Close
Palplates are a collection of touch-screen terminals placed around the office enabling human-computer interactions at the point of need. Supporting a community of mobile authenticated workers with a small number of stationary devices is an alternative to providing each person with a portable wireless computer. In contrast to the PC's desktop metaphor, Palplates use a place metaphor that reflect the actual rooms, corridors, and buildings that are part of the office place. Users interact graphically with applications supported by a geographic database. The user interface is generated dynamically based on the user's identity, the point-of-access, and the changing collection of physical office equipment, electronic documents and applications present at any given location.

Metadata for Mixed Media Access.

Publication Details
  • In Managing Multimedia Data: Using Metadata to Integrate and Apply Digital Data. A. Sheth and W. Klas (eds.), McGraw Hill, 1997.
  • Feb 1, 1997

Abstract

Close
In this chapter, we discuss mixed-media access, an information access paradigm for multimedia data in which the media type of a query may differ from that of the data. This allows a single query to be used to retrieve information from data consisting of multiple types of media. In addition, multiple queries formulated in different media types can be used to more accurately specify the data to be retrieved. The types of media considered in this paper are speech, images of text, and full-length text. Some examples of metadata for mixed-media access are locations of keywords in speech and images, identification of speakers, locations of emphasized regions in speech, and locations of topic boundaries in text. Algorithms for automatically generating this metadata are described, including word spotting, speaker segmentation, emphatic speech detection, and subtopic boundary location. We illustrate the use of mixed-media access with an example of information access from multimedia data surrounding a formal presentation.

Text Types in Hypermedia

Publication Details
  • In Proceedings of the Thirtieth Annual Hawaii International Conference on System Sciences (Wailea, Hawaii, January 1997), Volume VI, pp. 68-77.
  • Jan 7, 1997

Abstract

Close
The discipline of narratology has long recognized the need to classify documents as instances of different text types. We have discovered that classification is as applicable to hypermedia as it is to any other document presentation. Following the work of S. Chatman (1978; 1990), we consider three such text types: description, argument and narrative. The goal of a description document is to describe some object or concept; this is usually achieved by describing component parts and then describing how those parts combine to constitute the entirety. An argument document, on the other hand, is concerned with establishing some assertion or point of view, and it is based on supporting evidence, as well as possible refutations and justifications for defeating those regulations. Finally, a narrative document recounts some sequence of events in time, addressing relationships such as causality and contingency among those events. We analyze these types through case studies that give an example of each as a hypermedia document. We then argue that this classification provides an organizational framework that facilitates the construction of outlines that serve the writer in preparing the actual content of a document. Such outlines can also benefit the reader's understanding of the content that the writer intended to convey; if the writer does not make those outlines available explicitly to the reader, the reader can use knowledge of the document type to construct his own version of those outlines. Finally, we review some early work in content based indexing and search of multimedia documents
1996
Publication Details
  • Proceedings Knowledge Representation for Interactive Multimedia Systems: Research and Experience (Budapest, Hungary, August 1996), ECAI, pp. 57-65.
  • Aug 1, 1996

Abstract

Close
An approach to semantics based on traditional paradigms of knowledge representation (e.g., developing reductionist models of video document genres), while not entirely of the mark, may be significantly misdirected. Understanding the semantics of video and multimedia must begin with understanding how video (and film) are "read" and "written." The purpose of this paper is to set an agenda for coming to an understanding of reading and writing multimedia and to address the representationalist implications of achieving that end. Further, we illustrate this research agenda, showing how we have applied concepts and methods from film theory (to the reading process) and interaction analysis (to the work of multimedia production), and what our preliminary findings might mean for computational support and knowledge representation in multimedia.
Publication Details
  • IEEE Computer Society Multimedia Newsletter, 4, 1 (August 1996), pp. 45-48.
  • Aug 1, 1996

Abstract

Close
The fundamental objective of Extended Media research is the empowering of documents through technology. We see our goal as that of inventing documents which communicate more effectively. Furthermore, we need to make it easier for the writer to record what must be communicated and for the reader to access it. Thus, we anticipate that one cannot invent new documents without also inventing new processes for both writing and reading. Our primary research thrust is thus in the authoring of hypermedia documents, supplemented by a secondary thrust concerned with the problem of managing archives and libraries where more than text is involved.
Publication Details
  • Proceedings of the 16th International Conference on Computational Linguistics (Copenhagen, Denmark, August 1996).
  • Aug 1, 1996

Abstract

Close
We have developed a technique that categorizes document images based on their content. Unlike conventional methods that use optical character recognition (OCR), we convert document images into word shape takens, a shape-based representation of words. Because we have only to recognize simple graphical features from image, this process is much faster than OCR. Although the mapping between word shape tokens and words is one-to-many, they are a rich source of information for content characterization. Using a vector space classifier with a scanned document image database, we show that the word shape token-based approach is quite adequate for content-oriented categorization in terms of accuracy compared with conventional OCR-based approaches.
Publication Details
  • Proceedings Interface Conference (Sydney, Australia, July 1996).
  • Jul 1, 1996

Abstract

Close
Online digital audio is a rapidly growing resource, which can be accessed in rich new ways not previously possible. For example, it is possible to listen to just those portions of a long discussion which involve a given subset of people, or to instantly skip ahead to the next speaker. Providing this capability to users, however, requires generation of necessary indices, as well as an interface which utilizes these indices to aid navigation. We describe algorithms which generate indices from automatic acoustic segmentation. These algorithms use hidden Markov models to segment audio into segments corresponding to different speakers or acoustics classes (e.g. music). Unsupervised model initialization using agglomerative clustering is described, and shown to work as well in most cases as supervised initialization. We also describe a user interface which displays the segmentation in the form of a timeline, which tracks for the different acoustic classes. The interface can be used for direct navigation through the audio.

Integrating Information via Matchmaking

Publication Details
  • Journal of Intelligent Information Systems, 1996.
  • Jun 1, 1996

Abstract

Close
Trends such as the massive increase in information available via electronic networks, the use of on-line product data by distributed concurrent engineering teams, and dynamic supply chain integration for electronic commerce are placing severe burdens on traditional methods of information sharing and retrieval. Sources of information are far too numerous and dynamic to be found via traditional information retrieval methods, and potential consumers are seeing increased need for automatic notification services. Matchmaking is an approach based on emerging information integration technologies whereby potential producers and consumers of information send messages describing their information capabilities and needs. These descriptions, represented in rich, machine-interpretable description languages, are unified by the matchmaker to identify potential matches. Based on the matches, a variety of information brokering services are performed. We introduce matchmaking, and argue that it permits large numbers of dynamic consumers and providers, operating on rapidly-changing data, to share information more effectively than via traditional methods. Two matchmakers are described, the SHADE matchmaker, which operates over logical and structured text languages, and the COINS matchmaker, which operates over free text. These matchmakers have been used for a variety of applications, most significantly, in the domains of engineering and electronic commerce. We describe our experiences with the SHADE and COINS matchmaker, and we outline the major observed benefits and problems of matchmaking.
Publication Details
  • International Journal of Cooperative Information Systems 6(2), 1996.
  • Feb 1, 1996

Abstract

Close
As agents see more use in dynamic, distributed information networks, information sharing facilitators, such as the SHADE matchmaker, and underlying knowledge-based agent communication protocols, such as the Knowledge Query and Manipulation Language, will see increased use. We have created several communities of agents collaborating via KQML and matchmaking within the domains of collaborative engineering and satellite image retrieval. Based on these experiences, matchmaking has proven to be very beneficial for multi-agent systems, but we have also identified a number of issues and extensions that are not only vital to KQML-based matchmaking, but to inter-agent protocols in general. These include representational approaches to advertising complex databases, approaches to error recovery and response timing, maintaining consistency among information providers, scalability, security, persistent requests in information brokering, and the dilemma between explicit vs. implicit brokering.
1995

Multimedia Document System for Temporal and Spatial Structuring

Publication Details
  • In Hypermedia Design, Montpellier 1995. S. FranssT, F. Garzotto, T. Isakowitz, J. Nardard, and M. Narard (eds.), Springer Verlag, 1996, pp. 39-58.
  • Oct 31, 1995

Abstract

Close
Structuring temporal relationships among multimedia information elements is one of the most important facilities for editing and creating multimedia documents. We have developed a multimedia system named MediaPreview which provides facilities for structuring and creating multimedia documents. In this paper, we present a document model which is adopted in MediaPreview. This model has been designed to realize spatial and temporal structuring for the documents. The main feature of this model is the concept of a "Multimedia Paragraph" (MMP) which is introduced to reduce the complexity of the temporal structuring, such as the asynchronous interactive operation among documents. The concept of "MMP" provides an explicit and basic unit which is used to create a document in a "top-down" manner. This paper also presents the system architecture and implementation of MediaPreview in a distributed environment including database system facilities. This system realizes "static" and "dynamic" integration schemes for multimedia information elements. Our system includes a parallel database engine which manipulates multimedia information elements as streams. This database engine is effectively used for creating multimedia documents.