RSS Google AI Blog
Follow
Sensible Agent: A framework for unobtrusive interaction with proactive AR agents
Sensible Agent is a framework designed for unobtrusive interaction with proactive AR agents. It uses multimodal sensing to anticipate user needs and provide contextually appropriate assistance, addressing the limitations of voice-command-based systems. The system comprises two modules: one determines what assistance is needed, and the other decides how to deliver it considering social context. The prototype uses a context parser, proactive query generator, interaction module, and response generator, all running on Android XR and WebXR. A user study compared Sensible Agent to a voice-controlled baseline across various scenarios. The study revealed that Sensible Agent significantly reduced cognitive workload and increased user preference. Interaction time was slightly longer, but the preference for Sensible Agent suggests the trade-off was acceptable. Proactivity reshapes the user's relationship with the agent, fostering a collaborative experience. Future directions include personalization, scaling across devices, and applications in smart homes and robotics. The research team integrated multimodal sensing and real-time adaptation to improve human-agent interaction. The authors acknowledge their collaborators, feedback, and contributions from multiple teams at Google.