According to Eugene Yan, testing machine learning code and systems can be approached through three complementary checks: correct implementation, expected learned behavior, and satisfactory performance. Each check targets a different layer of the ML lifecycle, from code correctness to model outcomes.
The first check asks whether the implementation matches the intended design — catching bugs in data processing, training logic, or inference code. The second moves beyond code to behavior, verifying that the trained model actually learns what it should. The third evaluates results against performance expectations, ensuring the system is good enough for its intended use.
Yan’s framing is useful because it separates concerns that are often conflated. A model can be correctly implemented yet behave unexpectedly, or behave as expected yet underperform. Testing all three dimensions provides a more complete picture of system readiness.