Training of Vehicle Perception via Stochastic Simulation

Abstract

In order to train perception systems for autonomous and semi-autonomous vehicles, one of the most difficult challenges is constructing large and accurately labeled datasets. In order to ease the cost of data collection (and more importantly, data labeling), this paper focuses on the construction of a modular, open simulator for automating the creation and collection of realistic and information-rich synthetic data. By emphasizing modularity, we allow the user to easily extend the simulator's generation process, to add new textures/objects, and to add new sensors in order to match their needs. By also emphasizing variation through randomization, we allow such changes to be propogated throughout the generated scenes, resulting in much more robust datasets. For implementation, we utilized the Unity3D game engine, a variety of free assets available online, and map data from OpenStreetMap and NaturalEarth, therefore allowing the user to make these changes with relative ease, able to consult a large repertoire of online sources. Lastly, we show that our simulation is able to train models that accomplish a variety of vehicle perception tasks such as estimating the location of lane markings, road curvature, distances to vehicles in each lane, distances to stop signs and stop lights, and vehicle bounding box proposals. Lastly, we show that those models are able to transfer their knowledge to their relevant domains, and utilize the KITTI Object Detection dataset to demonstrate domain transfer to the real world.