Computer-use agents (CUAs) are typically judged on how well they complete a fixed task specification. But real users do not always behave that way: they may leave out requirements, add extra constraints, or demand approval before consequential actions. A new arXiv paper argues that completing the benchmark instruction alone is not enough to capture these realistic scenarios.
The paper introduces PersonaCUABench, a benchmark designed to serve diverse personas that reflect these variations. By incorporating omitted requirements, added constraints, and approval checkpoints, the benchmark aims to provide a more faithful test of how well CUAs adapt to different user needs. The authors position this as a step beyond conventional fixed-spec evaluation, though the full details of the benchmark design and results are not covered in the abstract.