Deep neural network use to help improve vision-based depth estimation in computer vision

Though no tech can quite compare to the nuances of human vision, that doesn’t mean researchers won’t try. In this comprehensive overview, researchers look for ways to get computer vision depth estimation close to the capabilities of human perception.

Three-dimensional computer vision is becoming more prevalent in everyday lives as advances in technology and its accessibility to the general population increase. This type of tech is seen in autonomous driving vehicles, augmented reality (AR), virtual reality (VR) and robotics, to name a few. However, historically, these types of technologies have been limited by their methods employed: LiDAR (light detection and ranging, a remote sensing tech used to measure distances) is limited by its cost and low-resolution to be as useful as it once was in the burgeoning field that is 3D computer vision. Researchers are looking towards something else: vision-based methods using depth foundation models.

Researchers published their results in Computational Visual Media in January 2026.

Vision-based methods avoid the use of active depth-sensing hardware, such as LiDAR, in exchange for the readily available technology of cameras, which are commonly found on everyday devices. Vision-based methods are more cost effective and are not as affected by environmental conditions, expanding their use in the real-world, while also maintaining higher resolution imaging than the traditional method. Despite the improvements vision-based methods have, there are still kinks to work out to make this method truly useful and practical.

Depth foundation models appear to be the solution to the problems of vision-based depth estimation methods, researchers believe. Current issues with vision-based methods include generalization and stability issues, mostly attributed to a lack of large-scale data sets and low capacity for the type and amount of data needed to be processed by the existing artificial intelligence models. Depth foundation models are deep neural networks that are trained on large datasets with the ability to execute complex tasks without relying on previous training examples, instead using general knowledge and information to assess how to handle the task at hand.

This study focuses on the evolution of this deep learning architecture for depth estimation in the following categories: monocular (use of images from a single camera), monocular video (use of video from a single camera), stereo (using multiple points from a 2D image to estimate 3D depth) and multiview (using multiple 2D images from multiple perspectives to get 3D information).

Results indicate monocular depth estimation models have increasingly stronger generalization capabilities, which covers a lot of the groundwork for other foundation models. Stereo foundation models utilize the framework achieved by monocular datasets to learn, making effective learning from a small-scale dataset achievable. Monocular video depth estimation integrates temporal and spatial depth cues for a consistent and scalable result. Finally, multi-view stereo (MVS) shows great robustness and accuracy, most likely due to the additional points where data can be gathered.

While all of these methods appear to lay out a promising path, there are numerous areas for improvement in the field of foundation models for depth estimation tasks, such as a lack of relevant training data, inconsistencies in depth estimation, difficulties adjusting to complex materials such as reflective or transparent surfaces, and data efficiency and consistency for both spatial and temporal inputs.

“We believe that as foundation models continue to evolve and depth estimation tasks advance, we will witness an increasing number of sophisticated and practical applications in the future,” said Hujun Bao, researcher and author of the study.

Though these problems seem plentiful, there is significant room for improvement in the technology and datasets, leading to imperative and inevitable advancements in depth estimation for use in 3D computer vision.

Zhen Xu, Hongyu Zhou, Sida Peng, Haotong Lin, Jiahao Shao, Peishan Yang, Qinglin Yang, Shen Miao, Xingyi He, Yifan Wang, Yue Wang, Yiyi Liao, Xiaowei Zhou and Hujun Bao of Zhejiang University, Haoyu Guo of the Shanghai AI Lab and Ruizhen Hu of Shenzhen University contributed to this research.

The Zhejiang Provincial Natural Science Foundation of China and the Information Technology Center and State Key Lab of CAD&CG supported this study.

DOI Link:

https://doi.org/10.26599/CVM.2025.9450517