Monocular depth estimation has made large strides with models like Depth Anything, but those models treat each frame independently and often produce flickering, inconsistent depth when applied to video. Prior attempts to fix this have leaned on video generation models or on priors from optical flow and camera poses. Those methods, however, are limited to short clips of under ten seconds and force a trade-off between output quality and computational cost.

Video Depth Anything takes a different route. The authors keep the Depth Anything V2 backbone but replace its head with an efficient spatial-temporal head, and they introduce a temporal consistency loss that constrains the temporal depth gradient. This removes the need for extra geometric priors. The model is trained on a joint dataset of video depth and unlabeled images, similar to the original Depth Anything V2, and a novel key-frame-based strategy is used for long video inference.

According to the paper, the model can be applied to arbitrarily long videos without sacrificing quality, consistency, or generalization ability. Evaluations across multiple video benchmarks report a new state-of-the-art in zero-shot video depth estimation. The authors also offer models at different scales, with the smallest one capable of real-time performance at 30 frames per second.