Improving Time-of-flight and Other Depth Images: Super-resolution and Denoising Using Variational Methods
Abstract
Depth information is a new important source of perception for machines, which allow them to have a better representation of the surroundings. The depth information provides a more precise map of the location of every object and surfaces in a space of interest in comparison with conventional cameras. Time of flight (ToF) cameras provide one of the techniques to acquire depth maps, however they produce low spatial resolution and noisy maps. This research proposes a framework to enhance and up-scale depth maps by using two different regularization terms: Total Generalized Variation (TGV) and Total Generalized Variation with a Structure Tensor (TTGV). Furthermore, the proposed technique implements the Alternating Direction Method of Multipliers (ADMM) not just to solve a denoising problem as previous efforts, it employs this numerical method to inpaint, reduce impulsive noise and up-scales the low-resolution observation from ToF camera frames by means of fusion of them instead of getting help for the super-resolution process from a second RGB color camera as others have considered. The proposed technique’s performance relies on the precision of the multi-frame registration and the denoising capability. The registration performance is addressed with the iterative motion estimation proposed by Lukas-Kanade while the noise elimination is based on an impulsive noise detection scheme recently adapted to the multi-frame super-resolution problem. The proposed algorithm of this dissertation is objectively validated using simulated depth maps and toy images while the subjective evaluation is performed using real ToF depth image sequences. In general, the algorithm shows reduction of staircasing phenomena and enhancement of simulated and real depth maps under the presence of Gaussian noise and two types of impulsive nose: salt and pepper and random value.
Subject Area
Applied Mathematics|Computer Engineering|Electrical engineering
Recommended Citation
Andrade, Salvador Canales, "Improving Time-of-flight and Other Depth Images: Super-resolution and Denoising Using Variational Methods" (2018). ETD Collection for University of Texas, El Paso. AAI10815216.
https://scholarworks.utep.edu/dissertations/AAI10815216