Our research laboratory is called HIMMEL (Harvard Intelligent Multi-Media Environments Laboratory). As a group, we have been developing an automatic videoconferencing system which employs acoustic source localization, video face tracking and head pose estimation. I was involved in the development of the audio portion of the system which uses a set of microphone arrays to provide the initial localization of the talkers. Beyond this, the microphone array can detect a new source outside the current video frame and our digital signal processing algorithms are used to provide an enhanced audio stream to accompany the video signal. The video component (the work of Ce Wang) tracks the talker by utilizing source motion, contour geometry, color data, and simple facial features. The resulting system is capable of tracking multiple people in complex backgrounds and robustly discriminating faces from similar objects. Decisions involving which camera to use are based on an estimate of the head's orientation as well as the estimated signal-to-noise ratios (SNR) of the spatially diverse microphones.

Wang, C., Griebel, S., and Brandstein, M., "Robust Automatic Video-Conferencing with Multiple Cameras and Microphones," Proceedings of the IEEE International Conference on Multimedia and Exposition, New York, NY, July 2000. [download PDF]