AI Experiment: Hacking HubSpot Chat to Improve the Customer Experience

HubSpot integrated AI into their website chat and saw a 43% increase in chat conversion rates, a >50% improvement in the value per chat, and CSAT scores matched human-led interactions. The experiment involved integrating an AI chatbot on high-traffic, low-risk pages, and tracking KPIs such as conversion rates, value per chat, and CSAT scores. Initially, CSAT scores declined, but after manually annotating chat transcripts to improve the AI's responses, the results improved. The next test is happening on the pricing page, aiming to handle product-related questions and help customers understand packages and pricing. The team is also developing an annotation user interface to speed up the AI's training. Four tips for using AI to improve the customer experience include starting experimentation now, aiming for dramatic results, being transparent with customers, and starting with chat.