GitHub has adopted a new credential classifier for secret scanning, replacing a larger LLM pipeline. The system first uses regular expressions to find strings that look like passwords, tokens, or other credentials, then a fine-tuned ModernBERT model scores each candidate using context such as file path, credential type, variable names, and surrounding code. This helps distinguish real secrets from "hard negatives" like sample values, build identifiers, and test fixtures.
The change brought major efficiency gains: median post-push scan execution time fell by 86% and model serving costs dropped by more than 98%. The speedup also let GitHub extend generic secret detection to push protection, which can block a push before a credential leak is accepted. In offline evaluation on public traffic, the new pipeline achieved 13% higher precision and 19% higher recall than the previous LLM pipeline, and a customer A/B experiment recorded an 11.5% increase in repository owner-weighted alert acceptance.
GitHub built its training and evaluation data from product feedback, repository examples, and synthetic cases, with human validation and annotation. The model was exported to ONNX format with 8-bit quantization for faster inference. GitHub plans to use production feedback to catch missed secrets and false positives, and is exploring a single-step architecture that combines secret detection and extraction.