Proactive systems in augmented reality, virtual reality, and embodied applications need to anticipate how users will interact with their surroundings. The new arXiv paper Gaze2HOI addresses this challenge by focusing on egocentric hand-object interaction forecasting.
The authors note that prior work usually forecasts hand-object interactions directly, without accounting for the user's gaze. Gaze2HOI instead incorporates gaze as a signal to disambiguate which of several possible interactions the user intends, according to the abstract.
This matters because gaze can reveal intent before a hand movement is completed, potentially making assistance more timely. The paper is available as arXiv:2610.27877v1.