FlyAbout

Spatially indexed panoramic video

The FlyAbout system uses spatially indexed panoramic video to create an interactive, photographic “virtual reality” system.

By recording both FlyCam panoramic video and location data along interesting paths, the video can be accessed by spatial location using a map or “driving” interface.

Panoramic video is captured by a moving a 360° camera along continuous paths (see below for details), and replayed to give users the experience of moving along those trajectories, and the ability to view any interesting object or direction. Spatially indexed video gives the ability to travel along paths or roads with a map-like interface. At intersections, users can chose which path to follow as well as which direction to look, allowing interaction not available with conventional video. Combining the spatial index with a spatial database of maps or objects allows users to navigate to specific locations or interactively inspect particular objects.

Here are two videos that show FlyAbout navigation. The first video shows interactively navigating a courtyard at Stanford University campus. Dragging the arrow changes your view direction, while dragging the red dot moves your view location. The second video shows a larger-scale FlyAbout system where you can virtually “drive” around Lake Tahoe. The red location dot moves with your viewpoint. Notice that you can even “drive backwards” by reversing your viewpoint and the play direction!

FlyAbout Figure

FlyAbout Capture

We have automated the FlyAbout video production system to a large extent. Panoramic video is recorded by moving a six-camera FlyCam along interesting paths. Filming panoramic video can be challenging, as it is difficult for the operator to avoid being in the picture! For our “walking tour” productions, the operator crouches underneath a tripod dolly. We have also constructed a roof-mounted camera system that lets the operator drive the camera around in a car (a much more comfortable solution). Each of the six cameras is recorded by a handheld DV camcorder used as a video tape recorder (VTR). These are battery-powered and quite small which makes for a reasonably  portable recording solution. SMPTE time code is synchronously recorded on one audio track of each of the six VTRs for later synchronization. Concurrently, time-stamped GPS location data is recorded synchronously on a laptop computer.

This is our roof-mounted panoramic camera system:

project-flyabout-carmount

And this is our camera system, including video and GPS capture hardware. Fairly portable, if you don’t mind the spaghetti.  Note the six-camera FlyCam at far left.

project-flyabout-recordrig

FlyAbout Post-Production

Once captured, the six video files are individually transferred to a PC via a firewire interface. The component images are corrected for lens distortion and color matching using patent-pending algorithms. The pictures below show the raw video (from an earlier 4-camera system).

project-flyabout-sourcecams

These images are unwarped into a cylindrical projection, corrected for camera tilt, offset, and brightness, and stitched into a sequence of composite panoramic images like that shown below. This is done in a batch process, but works at several times real time. The result is a motion JPEG of continuous cylindrical panoramas at the video frame rate of 30 per second.

project-flyabout-roiview

 

Technical Contact

Related Publications

2002

Detecting Path Intersections in Panoramic Video

Publication Details
  • IEEE International Conference on Multimedia and Expo 2002
  • Aug 26, 2002

Abstract

Close
Given panoramic video taken along a self-intersecting path, we present a method for detecting the intersection points. This allows "virtual tours" to be synthesized by splicing the panoramic video at the intersection points. Spatial intersections are detected by finding the best-matching panoramic images from a number of nearby candidates. Each panoramic image is segmented into horizontal strips. Each strip is averaged in the vertical direction. The Fourier coefficients of the resulting 1-D data capture the rotation-invariant horizontal texture of each panoramic image. The distance between two panoramic images is calculated as the sum of the distances between their strip texture pairs at the same row positions. The intersection is chosen as the two candidate panoramic images that have the minimum distance.
2001
Publication Details
  • Proc. ACM Multimedia 2001, Ottawa,CA, Oct. 2001.
  • Sep 30, 2001

Abstract

Close
We describe a system called FlyAbout which uses spatially indexed panoramic video for virtual reality applications. Panoramic video is captured by moving a 360° camera along continuous paths. Users can interactively replay the video with the ability to view any interesting object or choose a particular direction. Spatially indexed video gives the ability to travel along paths or roads with a map-like interface. At junctions, or intersection points, users can chose which path to follow as well as which direction to look, allowing interaction not available with conventional video. Combining the spatial index with a spatial database of maps or objects allows users to navigate to specific locations or interactively inspect particular objects.