Mixed reality interfaces face a fundamental tension: they must continuously adapt to what a user is doing and what is happening around them, yet the policies governing those adaptations are typically fixed ahead of time. According to a new arXiv preprint, these policies are either preset by the system or manually configured by the user, leaving little room for in-the-moment adjustment.
The paper, titled "Say It or Grab It? Conversational Control of UI Adaptations in Mixed Reality," suggests natural language as an alternative. Rather than relying on static rules, users could simply state what they need, offering a more flexible way to handle dynamic tasks and environments.
The abstract is brief and does not include experimental details or results, so the claims are limited to the proposed direction. As a single source, there are no other findings to compare against, but the idea highlights a growing interest in making mixed reality interfaces more responsive to user intent through conversation.