Model merging combines several fine-tuned expert models into a single network without retraining. Closed-form methods do this by computing each merged layer from statistics of its input activations, which are measured on one expert model.

The paper argues this is a weak point: statistics gathered from a single expert may not reflect the activations the merged model will actually see. CrossMean addresses this by pairing merged and expert activations when computing the statistics, so the merged layers are derived from a more representative signal.

The arXiv abstract describes the method but does not include experimental results, so any reported performance gains are not covered here.