Building AI that interacts with people often requires testing at a scale that would be impractical with human subjects alone. User simulators promise a way around this bottleneck, acting as stand-ins for people in evaluation and training. The authors of a new arXiv paper argue that such simulators can only fulfill this role if they mimic human behavior closely enough to be trustworthy.
The paper focuses on memory-constrained simulators and asks whether they reproduce two well-known human cognitive tendencies: repetition and interference. In people, limited memory can lead to repeating previous statements and to confusing or blending similar pieces of information. A simulator that does not show these effects may behave too smoothly compared to real users.
That leads to the paper's central suggestion: evaluation should measure behavioral parity with people. Rather than merely producing plausible dialogue, a simulator should be tested for the same patterns of errors and constraints that characterize human interaction. How well memory-limited simulators actually match human data remains the question the study investigates.