Real-time Virtual Reality 3D Scanning

Abstract

We propose a head-mounted device and accompanying software for 3D scanning of static scenes that provides real-time feedback during scanning and allows for geometry-aware augmented reality applications with intuitive human-computer interaction. This method allows viewing the current scan from multiple viewpoints at different angles, while also providing a feeling of presence through a virtual reality interface. Holes and other artifacts can be immediately identified and fixed. The device uses accurate camera pose tracking to achieve registration at interactive speeds and also fix the loop-closure problem. We demonstrate two augmented reality applications: painting on scanned surfaces by pointing with the finger and a physics simulation where virtual rigid bodies interact with the scanned scene.