User interfaces for digital libraries tend to focus on retrieval: users retrieve documents online, but then print them out and work with them on paper. One reason for printing documents is to annotate them with freeform ink while reading. Annotation can help readers to understand documents and to make them their own. In addition, annotation can reveal readers' interests with respect to a particular document. In particular, it is possible to construct full-text queries based on annotated passages of documents. We describe an experiment that tested the effectiveness of such queries, as compared to relevance feedback query techniques. For a set of TREC topics and documents, queries derived from annotated passages produced significantly better results than queries derived from subjects' judgments of relevance.

How will we read digital library materials? This paper describes the reading practices of an on-going reading group, and how these practices changed when we introduced XLibris, a digital library reading appliance that uses a pen tablet computer to provide a paper-like interface. We interviewed group members about their reading practices, observed their meetings, and analyzed their annotations, both when they read a paper document and when they read using XLibris. We use these data to characterize their analytic reading, reference use, and annotation practices. We also describe the use of the Reader's Notebook, a list of clippings that XLibris computes from a reader's annotations. Implications for digital libraries stem from our findings on reading and mobility, the complexity of analytic reading, the social nature of reference following, and the unselfconscious nature of readers' annotations.

The Palette is a digital appliance designed for intuitive control of electronic slide shows. Current interfaces demand too much of our attention to permit effective computer use in situations where we can not give the technology our fullest concentration. The Palette uses index cards that are printed with slide content that is easily identified by both humans and computers. The presenter controls the presentation by directly manipulating the cards. The Palette design is based on our observation of presentations given in a real work setting. Our experiences using the system are described, including new practices (e.g., collaborative presentation, enhanced notetaking) that arise from the affordances of this new approach. This system is an example of a new interaction paradigm called tacit interaction that supports users who can spare very little attention to a computer interface.

NotePals is a lightweight note sharing system that gives group members easy access to each other's experiences through their personal notes. The system allows notes taken by group members in any context to be uploaded to a shared repository. Group members view these notes with browsers that allow them to retrieve all notes taken in a given context or to access notes from other related notes or documents. This is possible because NotePals records the context in which each note is created (e.g., its author, subject, and creation time). The system is "lightweight" because it fits easily into group members' regular note-taking practices, and uses informal, ink-based user interfaces that run on portable, inexpensive hardware. In this paper we describe NotePals, show how we have used it to share our notes, and present our evaluations of the system.

Recent work on the social nature of human-computer interactions [3] has prompted research on animated, anthropomorphic characters in user interfaces. Such interfaces may simplify user interactions by allowing them to use and interpret natural face-to-face communication techniques such as speech, gestures and facial expressions. We describe our initial implementation, a character that controls the A/V facilities in a state-of-the-art conference room, and outline the goals of our ongoing project.

We explored a new type of user interface, interactive cover sheets: computer forms laid out on the banner pages of print jobs that people can mark on, scan back into a multifunction printer/scanner, and use as input to applications. Cover sheets are commonly strewn around printer rooms; with interactivity, they can let people see what others have to say, add their own comments, or play games, all while waiting for their print jobs. We designed three prototype applications and deployed them briefly in our research lab. We found that interactive cover sheets can be very appealing, that the sheets must be designed so that people can still identify these pages as cover sheets, and that the slow interaction cycle favors asynchronous applications.

This paper presents methods of generating compact pictorial summarizations of video. By calculating a measure of shot importance video can be summarized by de-emphasizing or discarding less important information, such as repeated or common scenes. In contrast to other approaches that present keyframes for each shot, this measure allows summarization by presenting only the most important shots. Selected keyframes can also be resized depending on their relative importance. We present an efficient packing algorithm that constructs a pictorial representation from differently-sized keyframes. This results in a compact and visually pleasing summary reminiscent of a comic book.

This paper describes a technique for automatically creating an index for handwritten notes captured as digital ink. No text recog-nition is performed. Rather, a dictionary of possible index terms is built by clustering groups of ink strokes corresponding roughly to words. Terms whose distribution varies significantly across note pages are selected for the index. An index page containing the index terms is created, and terms are hyper-linked back to their original location in the notes. Further, index terms occurring in a note page are highlighted to aid browsing.

This paper describes techniques for classifying video frames using statistical models of reduced DCT or Hadamard transform coefficients. When decimated in time and reduced using truncation or principal component analysis, transform coefficients taken across an entire frame image allow rapid modeling, segmentation, and similarity calculation. Unlike color-histogram metrics, this approach models image composition and works on grayscale images. Modeling the statistics of the transformed video frame images gives a likelihood measure that allows video to be segmented, classified, and ranked by similarity for retrieval. Experiments are presented that show an 87% correct classification rate for different classes. Applications are presented including a content-aware video browser.

This paper describes a method for finding segments in video-recorded meetings that correspond to presentations. These segments serve as indexes into the recorded meeting. The system automatically detects intervals of video that correspond to presentation slides. We assume that only one person speaks during an interval when slides are detected. Thus these intervals can be used as training data for a speaker spotting system. An HMM is automatically constructed and trained on the audio data from each slide interval. A Viterbi alignment then resegments the audio according to speaker. Since the same speaker may talk across multiple slide intervals, the acoustic data from these intervals is clustered to yield an estimate of the number of distinct speakers and their order. This allows the individual presentations in the video to be identified from the location of each presenter's speech. Results are presented for a corpus of six meeting videos.

S. Bly and E.F. Churchill.

In many hierarchical companies, reports from several independent groups must be merged to form a single, company-wide report. This paper describes a process and system for creating and structuring such reports and for propagating contributions up the organization. The system has been in regular use, in-house, by about 30 users for over a year to create monthly status reports. Our experiences indicate that it is possible to change a monthly reporting practice so that the system is easy to use, improves the quality of the written report, fosters collaboration across projects and creates a corporate memory for the company. These results were achieved as a consequence of our design effort to directly support the hierarchical and collaborative process of creating and assembling the report within the organization. User feedback has led to many improvements in the usability and functionality of the system. Further enhancements using information retrieval and text summarization techniques are in progress.

This is a critical view of the hypothesis that better access to a broader repertoire of media resources will significantly enhance our ability to communicate more effectively. It begins by laying down a foundation of some basic principles concerning the nature of knowledge creation. This foundation is framed in a manner that involves the potential relevance of two particularly creative activities, storytelling and making jazz. This foundation provides the basis for a critical examination of several media-rich presentations that were delivered at the Institute for the Future Outlook Exchange in November of 1997, since these presentations actually pertained to the practices of digital storytelling and jamming. This critique is followed by a more detailed examination of what we may learn from jazz if we wish to invoke it as a metaphor for knowledge creation. The report then concludes by discussing the implications of these observations for a new world of work experiences in which knowledge creation is a critical element.

The genre of mathematics writing has several distinctive features that point to some of the weaknesses of current digital documents. Some of these weaknesses are surprising. While it might be expected that the importance of formatting and special symbols in mathematics writing would pose challenges for digital documents, the linked, chunked style of mathematics writing, with its Theorems, Lemmas, Corollaries and Remarks explicitly referring to each other, resembles standard hypertext so closely that one would expect that mathematics writing would take well to online hypertext form. It does not. This failure points to deficiencies in our understanding of the true strengths and weaknesses of digital documents. This paper describes mathematics writing, with particular emphasis on features of interest with respect to digital documents. The difficulties in producing effective digital mathematics documents are then examined and used as a basis for talking about general challenges for digital documents. The paper then discusses strengths of digital documents and some of the problems that need to be overcome before digital documents can live up to claims made for them. It also examines some of the misguided claims, such as superior support for non-linearity, that are commonly made for digital documents, explains why these claims are unwarranted, and speculates on why the claims have been made anyway. Suggestions are then given as to what the true benefits of digitization are, including performing computations on text, flexible control of time, and better support for hiding information. The paper concludes with a list of questions whose answers are critical to understanding the capabilities, and therefore the future, of digital documents.

As computer networks improve, more social and work interactions are carried out "virtually" by geographically separated group members. In this paper we discuss the design of a tool, PAVE, to support remote work interactions among colleagues in different time zones. PAVE extends a 2D graphical MOO and supports synchronous and asynchronous interactions. PAVE logs and indexes activities in the space. This capture facility enables playback and augmentation of meeting interactions by non-collocated group members. Thus, members can participate asynchronously in meetings they could not attend in real time, not just review them.

The Digestor system automatically converts web-based documents designed for desktop viewing into formats appropriate for handheld devices with small display screens, such as Palm-PCs, PDAs, and cellular phones. Digestor employs a heuristic planning algorithm and a set of structural page transformations to produce the "best" looking document for a given display size. Digestor can also be instructed, via a scripting language, to render portions of documents, thereby avoiding navigation through many screens of information. Two versions of Digestor have been deployed, one that re-authors HTML into HTML for conventional browsers, and one that converts HTML into HDML for Unwired Planet's micro-browsers. Digestor provides a crucial technology for rapidly accessing, scanning and processing information from arbitrary web-based documents from any location reachable by wired or unwired communication.

In accessing large collections of digitized videos, it is often difficult to find both the appropriate video file and the portion of the video that is of interest. This paper describes a novel technique for determining keyframes that are different from each other and provide a good representation of the whole video. We use keyframes to distinguish videos from each other, to summarize videos, and to provide access points into them. The technique can determine any number of keyframes by clustering the frames in a video and by selecting a representative frame from each cluster. Temporal constraints are used to filter out some clusters and to determine the representative frame for a cluster. Desirable visual features can be emphasized in the set of keyframes. An application for browsing a collection of videos makes use of the keyframes to support skimming and to provide visual summaries.

Reading appliances allow people to work on electronic documents much as they would on paper. They therefore provide an alternative to the standard "browse or search and then print" model of reading online. By integrating a wide variety of document activities, such as searching, organizing, and skimming, and by allowing fluid movement among them, reading appliances eliminate disruptive transitions between paper and digital media.

Reading appliances or e-books hold substantial promise to help us collaborate. In this paper, we use a study of a group activity - a reading group that meets to discuss articles of mutual interest - to explore four scenarios for collaborating with e-books: (1) meetings and face-to-face discussions; (2) serendipitous sharing of annotations, as when we borrow a document from a colleague or buy a used book; (3) community-wide use of anonymous annotations to guide future readers; and (4) e-books as a basis for initiating interaction between people. In so doing, we describe some methods for implementing these facilities, and introduce design guidelines.

In this paper, we describe a technique for dynamically grouping digital ink and audio to support user interaction in freeform note-taking systems. For ink, groups of strokes might correspond to words, lines, or paragraphs of handwritten text. For audio, groups might be a complete spoken phrase or a speaker turn in a conversation. Ink and audio grouping is important for editing operations such as deleting or moving chunks of ink and audio notes. The grouping technique is based on hierarchical agglomerative clustering. This clustering algorithm yields groups of ink or audio in a range of sizes, depending on the level in the hierarchy, and thus provides structure for simple interactive selection and rapid non-linear expansion of a selection. Ink and audio grouping is also important for marking portions of notes for subsequent browsing and retrieval. Integration of the ink and audio clusters provides a flexible way to browse the notes by selecting the ink cluster and playing the corresponding audio cluster.

NotePals is an ink-based, collaborative note taking application that runs on personal digital assistants (PDAs). Meeting participants write notes in their own handwriting on a PDA. These notes are shared with other participants by synchronizing later with a shared note repository that can be viewed using a desktop-based web browser. NotePals is distinguished by its lightweight process, interface, and hardware. This demonstration illustrates the design of two different NotePals clients and our web-based note browser.

Many techniques can extract information from an multimedia stream, such as speaker identity or shot boundaries. We present a browser that uses this information to navigate through stored media. Because automatically-derived information is not wholly reliable, it is transformed into a time-dependent "confidence score." When presented graphically, confidence scores enable users to make informed decisions about regions of interest in the media, so that non-interesting areas may be skipped. Additionally, index points may be determined automatically for easy navigation, selection, editing, and annotation and will support analysis types other than the speaker identification and shot detection used here.