Generating LiDAR point clouds from camera images could enable controllable data synthesis, cross-modal simulation, and downstream perception tasks. Yet most existing methods are not built for this task: they tend to rely on a single monocular image or to use only semantic cues, which limits the quality of the generated LiDAR.
LiFi, introduced by Sizhuo Zhou and colleagues, is a dedicated framework that takes multi-view images as input. It guides a latent diffusion model through two complementary branches. A geometry branch constructs scene features using a reverse sampling algorithm and an uncertainty-aware geometric encoder combined with DepthAnything3, while a semantic branch supplies high-level representations through a cross-view interaction module.
The authors also propose a dual-branch balanced classifier-free guidance (CFG) strategy. This enhances the model's ability to follow the conditioning information while preserving independent learning and collaborative controllability of the two branches. In experiments, LiFi outperformed state-of-the-art methods at generating high-fidelity LiDAR scenes. The paper was accepted to NeurIPS 2026, and the code is planned to be released.