Standard cameras and vision models can tell you an object is red, but not whether that red is paint, plastic, or fabric. Hyperspectral imaging captures more detail, and unmixing aims to decompose each pixel's spectrum into a set of material signatures. In 3D, however, this is usually done after rendering or with slow implicit neural fields, and there has been no multi-view dataset with known ground-truth composition to evaluate such methods.
The Spectral World Renderer (SWR) takes a different route. It uses a coding agent to build executable 3D worlds, which are rendered with measured spectra so every hyperspectral image comes with exact material labels and sub-pixel proportions. These worlds form SWR-Bench, eight indoor and outdoor scenes where decompositions can be verified. SWR itself performs unmixing inside the rendering pipeline: each Gaussian in a compact 3D representation is decoded into material abundances before compositing, so a single render yields both spectra and material maps.
Because the material assignment happens before pixels are combined, SWR can separate surfaces that share the same colour but are made of different materials. Users can query any surface by material name, a clicked pixel, or a reference spectrum. In experiments, SWR identified materials best in all eight benchmark worlds without material labels, achieved the best mean PSNR on real captures, trained 3.7–12.9 times faster than unmixing baselines, and rendered 34.9 labelled views per second.