CamaLeon: Smart Camera for Conferencing in the Wild

Abstract

Despite work on smart spaces, nowadays a lot of knowledge work happens in the wild: at home,
in coffee places, trains, buses, planes, and of course in crowded open office cubicles. Conducting
web conferences in these settings creates privacy issues, and can also distract participants, leading to a perceived lack of professionalism from the remote peer(s).
To solve this common problem, we implemented CamaLeon, a browser-based tool that uses real-time machine vision powered by deep learning to change the webcam stream sent by the remote peer: specifically, CamaLeon dynamically changes the “wild” background into one that resembles that of the office workers.
In order to detect the background in wild settings, we designed and trained a fast UNet model on head and shoulder images. CamaLeon also uses a face detector to determine whether it should stream the person’s face, depending on its location (or lack of presence). It uses face recognition to make sure it streams only a face that belongs to the user who connected to the meeting.
The system was tested during a few real video conferencing calls at our company where 2 workers are remote. Both parties felt a sense of enhanced co-presence, and the remote participants felt more professional with their background replaced.