Reference / Literaturverweis [Bre21a]

Müller, F., Krekel, N., Navarro, P., Kaehn, R., Bredereke, J.:
Applying Binarized Neural Networks on FPGAs to an Autonomous Driving Problem.
Technical Report. City University of Applied Sciences Bremen, Germany (Mar. 2021).

Abstract / Zusammenfassung

Autonomy of a moving vehicle requires continuous observations of the environment through a large number of sensors. This leads to a substantial amount of data that has to be processed as close to real time as possible. Artificial neural networks are a popular choice for problems like this, as they can find important patterns in data quickly and at the same time interpret these findings in the required context. Types of networks that use parameters with lower, fixed precision have been shown to require less computational resources and memory, while remaining accurate. Combined with specialised hardware like FPGAs, these networks promise reductions in latency and energy consumption.

Space exploration could benefit hugely from these improvements, as autonomy promises to reduce the effects of signal delays introduced by huge distances between vehicle and operator. More confidence could be put in a space probe's decisions, reducing situations in which time is wasted by waiting for control inputs from an operator. However, as space vehicles are among the most resource-constrained, harsh and expensive computing environments imaginable, it is impractical to add a lot of computing power. Quantized neural networks on FPGA hardware are therefore emerging as one solution to bring autonomy to more space vehicles.

This work presents a real life approximation of an autonomous roving scenario. It uses existing research results into binarized neural networks on FPGA hardware and combines it with actual low-power hardware that approximates what would currently be available to a space mission. The same hardware package also supplies the neural network with image data from a camera and translates its outputs into steering commands. Those are then used in a control scheme for the actual vehicle. The whole setup is able to find and identify a target object, work out its relative location and approach it, even if the target is not stationary. This demonstrates the potential of using FPGAs for complex deep learning inference tasks, reducing the requirements for computing power and energy consumption and therefore the applicability of this research to resource-constrained environments like space exploration.

Full Text / Volltext

PDF (11 MB)

Source Code / Quelltexte

Please just ask me for a ZIP file with the source code, if you need it. / Bei Bedarf fragen Sie mich bitte einfach nach einer ZIP-Datei mit den Quelltexten.

Video

You may also ask me for a video of the integration tests (see Sec. 8.1). / Sie können mich auch nach einem Video der Integrationstests (siehe Kap. 8.1) fragen.