Learning-Based Signal Recovery: Sparse-to-Dense Interpolation and Multi-View 3D Reconstruction

Reconstructing dense signals from sparse or complete measurements is a long-standing challenge in computer vision with applications across science and engineering. This thesis addresses four problems under this theme: depth completion, sea surface height (SSH) interpolation, reflectance transformation imaging (RTI), and multi-view 3D reconstruction. These are grouped into two categories: (1) data interpolation from sparse inputs, including depth completion, SSH, and RTI; and (2) reconstruction from complete data, represented by multi-view 3D geometry. For sparse-to-dense interpolation, we propose a hybrid model that combines classical interpolation with deep networks for unguided depth completion, diffusion-based generative modeling for SSH maps from sparse ocean measurements, and a Neural Reflectance Field (NRF) for RTI, enabling realistic relighting from limited illumination conditions. For complete-data reconstruction, we present ENHs3R, a unified framework that integrates coarse pose estimation with a multi-task transformer to jointly predict 3D point maps, surface normals, and image outpainting. Overall, this work contributes scalable and robust solutions for dense signal recovery, advancing the state of the art in computer vision, geoscience, and digital cultural preservation.