How Fullstory uncovers user insights with Vertex AI serving Gemini 1.5 Pro

Fullstory is a behavioral data analytics platform that helps organizations identify pain points and optimize digital experiences by reproducing user sessions and sharing strong analytics highlighting areas for improvement in the customer's journey. The platform's comprehensive AI-powered autocapture technology, Fullcapture, removes the need for manual instrumentation and uncovers hidden patterns that might otherwise be missed. Fullstory leverages Vertex AI serving Gemini 1.5 Pro to strengthen their autocapture technology, enabling customers to effortlessly analyze extensive data and focus on what truly matters. The integration of Fullcapture with Vertex AI allows for faster data discovery by highlighting important elements and automatically categorizing user interactions into semantic components, providing even deeper levels of analysis. Fullstory's AI agent, powered by Gemini 1.5 Pro, automates the critical task of identifying and labeling key elements on websites, making intelligent decisions at various stages and ensuring businesses can focus on deriving actionable insights. The platform's approach to optimizing for accurate element identification involves strategic prompt engineering, pre-filtering with heuristics, validation with Vertex AI with Gemini, and contextualized suggestions. This process ensures effective and efficient use of Gemini's AI capabilities, resulting in accurate and valuable element suggestions. Fullstory captures every user interaction, generating a wealth of data, and continuously monitors unrecognized components, prioritizing new feature discovery, power user behavior, error signals, and content analysis. The platform's deep indexing, semantic relevance, and powerful search capabilities enable efficient searching through countless variations of CSS selectors. Accurate labeling of elements is crucial, involving a clear and concise name, a detailed explanation of the element's purpose and behavior, and assigning a predefined role from Fullstory's library. Vertex AI with Gemini 1.5 Pro provides comprehensive context, including visual representation, textual analysis, and location tracking, to ensure high-quality and consistent element labeling. The collaboration between Fullstory and Google Cloud has yielded tens of thousands of element suggestions generated for customers, with Gemini 1.5 Pro intelligently filtering out a significant portion of irrelevant suggestions. The model has also identified numerous error elements that were previously unrecognized, and the mapping between element configuration and screenshots has opened up new opportunities for improving site configuration and enhancing analytics.