Video instance segmentation in low-light conditions is notoriously difficult because noise and blur obscure object boundaries, and models trained on normal-light footage transfer poorly to dark scenes. The authors of ELVIS note that existing synthetic training pipelines fail to capture temporal degradations, and large annotated low-light video datasets are scarce.
ELVIS tackles this with three components: an unsupervised synthetic low-light video pipeline that models both spatial and temporal degradations, a calibration-free degradation profile estimation network called VDP-Net, and an enhancement decoder head that disentangles degradation information from content features. Together, these allow domain adaptation of state-of-the-art video instance segmentation models to low-light scenarios without manual annotations.
In experiments, ELVIS improved performance by up to +3.7 AP on the synthetic low-light YouTube-VIS 2019 dataset and outperformed two-stage baselines by at least +2.8 AP on real low-light videos. The authors have released the code and dataset for further research.