1 million documents to 300+agents: Building an enterprise-scale Microsoft 365 Copilot connector
Microsoft built a large-scale Microsoft 365 Copilot connector to make internal documentation accessible as grounded context within Copilot experiences. This connector aims to bridge the gap where trusted enterprise knowledge often resides in underutilized portals. The connector's true power lies not in ingestion, but in enabling content availability across various Copilot features like Copilot Chat and Copilot in Microsoft 365 apps. It also empowers users to build custom Copilot Studio agents scoped to the connector's content without additional development effort. Transitioning from a prototype to a production-ready connector revealed critical gaps in tenant configuration, security and compliance readiness, discoverability, and shared throttling limits. Early validation of tenant configuration is essential, addressing issues like cross-tenant data flow and ownership gaps. Security and compliance readiness, including threat modeling and privacy reviews, required significant upfront effort. Content discoverability is a challenge as connector content may not be prioritized over emails and user files by default. Designing for throttling and shared limits is crucial, requiring robust retry logic and monitoring for ingestion slowdowns. The architecture emphasizes separating source content from ingestion compute for easier retries, monitoring, and evolution. Iterative schema design significantly improved retrieval quality, and secure authentication and deployment patterns were implemented for consistency.