Building a Mixed-Reality Tour ... Note

Building a Mixed-Reality Tour Guide with Android XR, the Geospatial API, and Gemini

Google announced the preview of the Geospatial API in ARCore for Jetpack XR, bringing its Visual Positioning System (VPS) to Android XR. This technology enables digital content to anchor to the physical world with high accuracy and orientation in supported regions. To demonstrate its potential, a team developed the XR Geospatial Tour, an immersive, hands-free walking tour experience. This demo combines the Geospatial API, Gemini API with Firebase AI Logic, Google Maps Grounding, and the Jetpack XR SDK.The Geospatial API leverages computer vision to provide more accurate location and heading than GPS. The application monitors accuracy thresholds, prompting users to move to recognized outdoor areas if indoors or in unsupported locations. To create the tour itinerary, the Gemini API is prompted with user coordinates to generate structured JSON responses detailing nearby walking tours. Google Maps Grounding is used to anchor the AI's generated locations, preventing inaccuracies.Dynamic voiceovers are implemented using the gemini-2.5-flash-tts model, which can directly return audio data. This adds a present and engaging element to the virtual tour guide. Jetpack Compose for XR is used to render spatial components and bring the tour to life in 3D. The team built custom composables like InfoSphere to display points of interest interactively.By combining Jetpack Compose for XR with the Geospatial API and Gemini, developers can create spatial experiences that understand both user location and context. The availability of the Geospatial API lowers the barrier for building world-scale spatial applications on Android. Developers can apply for the Android XR Developer Catalyst Program to gain access to XREAL Project Aura devkits. This initiative aims to foster innovation in Android XR development.