Software composition analysis tools surface far more vulnerable dependencies than teams actually fix. In production data from Snyk Open Source, 94% of vulnerability findings have a known fixed version available, yet only about 10% are resolved. This gap suggests that simply knowing a fix exists is not enough—applying it without breaking the project remains a practical hurdle.
To study this problem, the authors introduce VulnBump, a benchmark for fixing vulnerable dependencies in real projects while keeping the build and tests green. The benchmark is designed to test whether proposed dependency updates can be applied without introducing regressions, providing a standardized way to measure progress in automated or assisted vulnerability remediation.
By focusing on real projects and the requirement that builds and tests remain green, VulnBump moves beyond simple version matching. It highlights the need for fixes that are not only available but also safe to adopt, which may help explain why so many known vulnerabilities stay unresolved in practice.