Variational Optical Flow Algorithms for Motion Estimation

Abstract

Motion is an intrinsic character of the world and an inherent part of our visual experience, which gives essential source of information to a wide variety of visual tasks, and directly affects the subsequent image processing and other related applications. Since Horn and Schunck (HS) proposed the optical flow method to estimate motion field in 1981, it becomes one of the best approaches for motion estimation. The research described in this dissertation focuses on improving the accuracy of the optical flow estimation by proposing some novel techniques to handle the unsolved challenges. The main contributions of this work are the following: 1) We modify the balance between the data term and the smoothness term in two aspects: First, proposing a weighted root mean square (WRMS) error measure to automatically select an optimal smoothness parameter λ globally. Which achieves by setting a reference λ0 based on the quality of the frame, and then proposes an efficient brute-force approach to assign a group of λ to reduce the number of candidates. The optimal λ corresponds to the minimal RMS. Second, presenting an effective fusion method to fuse multiple flows of different smoothness parameter λ to compose a single superior flow. 2) We handle large displacements by integrating matching information into the continuous variational flow field according to a novel weighted local intensity fusion (WLIF) method. The variational methods usually combines with a coarse-to-fine strategy. However, the sub-sampling scheme that the coarse-to-fine framework used to reduce the size of the images and the motion within, leads to a loss of motion details that any algorithm can recover. Hence, the variational methods perform poorly on image structures with motions larger than their own size. The matching field supplies correct large displacements, which is helpful to recover the lost motion information (both large and small displacements) of traditional variational methods. 3) We remove outliers in either of the intermediate flow fields or the warped interpolation image by designing new filters. In particular, three filters are designed to achieve this goal. First, we present a novel combined post-filtering (CPF) method, which uses a weighted median filter (WMF), a bilateral filter (BF) and a fast MF to post-smooth the detected edges, occlusions, and flat regions of the intermediate flow fields respectively. Second, we present a PatchWMF method, which denpends on an improved color patch similarity measure (ICPSM), to modify the robustness of the WMF of to noise. Third, we present an adaptive guided image filter (AGIF) to correct these errors in the warped interpolation image. Due to this contribution, the accuracy of the discrete temporal derivative It is modified. 4) We achieve to estimate accurate optical flow in the presence of spatially-varying motion blur and, improve the flow accuracy by computing optical flow and restoring images with preserved edges simultaneously.