Inpainting with robust loss function

Reset to Original vs Set as Left Image

IID Gaussian + L2 IID Gaussian + Huber AR Sparse + L2 AR Sparse + Huber Telea04 Telea04 > Huber λ=0.01 λ=0.001

Example illustrating inpainting and robust (Huber) loss function. The goal is to remove the mesh and replace these missing pixels with inpainted (interpolated) pixels. We use a very crude method to detect the mesh: any pixel whose grayscale value (scaled to lie between 0 and 1) is larger than a threshold (0.65 or 0.7) is treated as missing. Pixels within a small neighbourhood of these pixels are also treated as missing.

This crude procedure does not completely remove the mesh. When inpainting with squared error loss for image error (i.e., Gaussian noise for the conditional distribution of the image), these show up as artifacts in the reconstructed image. Increasing the regularization parameter λ helps with this, but changing the squared error loss to a robust loss function also helps. Here we use the Huber loss function with scale parameter set to the MAD / 0.6745 (robust estimate of the standard deviation) of the residuals in each IRLS step. Results are only shown for the IID Gaussian and AR Sparse image gradient priors.

For comparison, results using the algorithm of Telea (2004), An image inpainting technique based on the fast marching method, is also shown, as implemented in OpenCV. The result of further denoising the result with a robust loss function is also shown.