A paper posted on arXiv (2508.17771) examines the tokenizers and vocabularies of GPT models including 4o, 4o-mini, o1, o3, 4.5, 4.1, and o4-mini. The authors argue that tokens are the basic elements of training data, so the presence of certain Chinese phrase tokens can reveal what content was in the training corpus. They report that many such tokens are indicative of pornography or similar explicit material.
The approach is indirect: rather than inspecting training data directly, the authors speculate about pollution based on which tokens exist. This offers a low-cost way to audit proprietary models whose training sets are not public. However, the paper's claims are explicitly speculative—a token's presence does not prove that explicit text was used, nor does it measure how such content affects model outputs.
Because this is a single source, there are no conflicting findings to compare. The significance lies in the method: token-level analysis could become a practical screening tool for data quality issues in large language models, even when full training data is unavailable.