The Virtual Factory: Industrial Collaboration Environments
The FX Palo Alto Laboratory in Palo Alto, California (FXPAL) is investigating virtual and mixed-reality systems for collaboration, particularly in industrial settings. In collaboration with TCHO, a chocolate maker start-up in San Francisco, we are building virtual mirror world representations of a real-world chocolate factory and its processes, and importing real-time sensor data into the resulting virtual factory. This research is an exploration into an area sometimes called "Serious Games" - combining high-end game engines with robust media and data importation for real-world purposes.
We run several parallel projects in our exploration of mixed reality applications for manufacturing:
Virtual Factory 3D world: This 3D environment is designed for simulation, visualization, and collaboration, using a set of interlinked, real-time 3D and 2D layers of information about the TCHO chocolate factory and its processes.
In the Virtual Factory environment, users can view real-time high-definition video streaming from a number of camera viewpoints in the factory, click on a machine's "DataSpot" to read its sensor status, or move an avatar closer to trigger an in-world video overview of its function.
The virtual environment imports real-time data from a network of sensors on the factory floor and displays it when appropriate in the virtual world. The system will be able to track processes in detail over extended time periods and play them back on request.
The virtual factory is also multi-user, allowing remote collaboration and communication for collaborative tasks such as factory observation, virtual inspections, customer visits, employee education and training, process monitoring, and inventory tracking. The Virtual Factory is now deployed on a large display at the real-world TCHO factory, where it is in daily use. FXPAL sees this mashup of real and virtual factories as a way to enhance collaboration and communication between physical-plant operators and engineers and remote managers; for example, factories in China with managers in Japan. It also creates efficiencies for local operators.
Mobile Mixed Reality: an iPhone/iPad app for laboratory monitoring and control
Integrating mobile with virtual and mixed reality systems is a logical extension for industrial smart environments. With this in mind, we have built an experimental app for iPhone and iPad that provides mobile laboratory monitoring and control.
In the TCHO recipe development lab, accurate tracking of time and temperature are essential. The mobile application gives the lab users a real-time view into this lab (via PTZ or Pan-Tilt-Zoom steerable camera) and imports real-world sensor data as well.
Using the mobile app on either iPhone or iPad (or Android devices), machines can be turned on/off, temperatures and timer settings can be read and changed, and lab environmental controls can be changed.
The mobile system is integrated with both the multi-camera surveillance system and the database underlying the virtual factory worlds. It has been in daily use at TCHO since 2009.
Our Mobile Virtual Factory shows the Virtual Factory's 3D environment on iPhone and Android mobiles, along with real-time data from the hundreds of sensors on the factory floor.
Web-based Virtual Factory tours allow visitors to see the chocolate making processes as they run in the online environment. Online visitors with the right permissions can see the live data streams from the factory floor.
Damon Hernandez of Metaverse One in an interview with Maribeth Back, Senior Research Scientist at FXPAL research lab in Palo Alto, California.
For more information, contact Maribeth Back at firstname.lastname@example.org.