In this text we present the real-time implementation of a Bayesian framework for robotic multisensory perception on a graphics processing unit (GPU) using the Compute Unified Device Architecture (CUDA). As an additional objective, we intend to show the benefits of parallel computing for similar problems (i.e. probabilistic grid-based frameworks), and the user-friendly nature of CUDA as a programming tool. Inspired by the study of biological systems, several Bayesian inference algorithms for artificial perception have been proposed. Their high computational cost has been a prohibitory factor for real-time implementations. However in some cases the bottleneck is in the large data structures involved, rather than the Bayesian inference per se. We will demonstrate that the SIMD (single-instruction, multiple-data) features of GPUs provide a means for taking a complicated framework of relatively simple and highly parallelisable algorithms operating on large data structures, which might take up to several minutes of execution with a regular CPU implementation, and arrive at an implementation that executes in the order of tenths of a second. The implemented multimodal perception module (including stereovision, binaural sensing and inertial sensing) builds an egocentric representation of occupancy and local motion, the Bayesian Volumetric Map (BVM), based on which gaze shift decisions are made to perform active exploration and reduce the entropy of the BVM. Experimental results show that the real-time implementation successfully drives the robotic system to explore areas of the environment mapped with high uncertainty.
In summary, our artificial active perception system contributions are twofold:
- It provides a rather complete framework for active multimodal perception—introducing an approach which, while not strictly neuromimetic, finds its roots in the role of the dorsal perceptual pathway of the human brain—of which the proof of concept and relevance to robotic active perception is presented elsewhere.
- By providing a real-time implementation of a probabilistic grid-based framework for multimodal perception, shown in Sect. 2 to be a relatively unexplored subject, it demonstrates that the SIMD features of GPUs provide a means of dealing with the scalability of highly parallelisable algorithms operating on large data structures, therefore allowing for real-time performances not possible using a CPU implementation.
Further details on ongoing work can be found at http://paloma.isr.uc.pt/~jfilipe/BayesianMultimodalPerception
João Filipe Ferreira, Jorge Lobo, and Jorge Dias. 2011. Bayesian real-time perception algorithms on GPU. J. Real-Time Image Process. 6, 3 (September 2011), 171-186. [DOI: 10.1007/s11554-010-0156-7]
Category: Computer Science