Extracting Cognition out of Images for the Purpose of Autonomous Driving

Abstract

Autonomous driving is a broadly recognized solution to serious traffic problems such as accidents and congestions. It is a very broad topic that extends across cognition, artificial intelligence, and control. While this thesis primarily focuses on the cognition aspect, others are also considered. Here, the thesis proposes several computer vision algorithms for autonomous driving, encompassing three major parts:

In part one, experiments on motion-based object recognition are presented. The proposed method differentiates objects according to their speed. In part two, the artificial intelligence aspect of autonomous driving is considered. Research on training an autonomous driving AI agent through reinforcement learning is introduced.

In part three, the key part of the thesis, a direct perception approach is proposed to drive a car in a highway environment. In this approach, an input image is mapped to a small number of key perception indicators that directly relate to the affordance of a road/traffic state for driving. This representation provides a set of compact yet complete descriptions of the scene to enable a simple controller to drive autonomously on highways. Using synthetic images from a virtual environment, a deep convolutional neural network (ConvNet) is trained for direct perception. Experiments show that the model can effectively drive a car in a very diverse set of virtual environments, and it provides good estimation of affordance indicators from real driving images. To further improve the performance of the direct perception-based system, the issue of temporal information is considered by studying the Long Short Term Memory (LSTM) unit and its influence on the affordance indicator estimation. Quantitative results show that adding the LSTM unit does help to improve the system's performance.

Finally, as object detection is closely related to autonomous driving, in Appendix A a deep learning-based small object detection approach is proposed. The applicability of the state-of-the-art object detection algorithms to the small object detection task is studied.