According to the arXiv abstract, synthetic data is playing a growing role in language-model pretraining. The paper proposes AFCS (Atomic-Fact-Conditioned Synthesis), a pipeline designed to improve factual knowledge in pretraining.
Existing knowledge-corpus synthesis pipelines, such as WRAP and Nemotron-CC, typically take a source passage or document segment as input. The abstract suggests AFCS differs by conditioning synthesis on atomic facts, though the full description is cut off in the provided text.
The paper is listed as a new announcement on arXiv. Because only the abstract snippet is available, details on the method's architecture, training setup, and evaluation results are not yet accessible from this source.