In a recent essay, Eugene Yan lays out what he calls the first rule of machine learning: start without it. The idea is to build a simple, explainable baseline—such as a rule of thumb or a straightforward statistical method—before reaching for a model. This approach forces teams to clarify the actual problem, surface data quirks, and establish a performance floor that any ML system must beat.
The article argues that these heuristics are not just a starting point but often a strong contender. Because they are transparent and cheap to run, they can be deployed quickly and debugged easily. If a heuristic already meets the business need, adding machine learning may only add complexity and maintenance cost without meaningful gains.
Yan suggests moving on to machine learning when the heuristic stops improving or when the problem clearly requires learning from patterns that simple rules cannot capture. Even then, the baseline remains useful as a benchmark. The essay is a practical reminder that ML is a tool, not a default answer—and that starting small is often the most direct route to a working solution.