Bring your own model to GitLab... Note
GitLab

Bring your own model to GitLab Duo Self-Hosted with Microsoft Foundry

Organizations with data sovereignty concerns can adopt AI coding tools like GitLab Duo by ensuring control over where their code resides. GitLab Duo Self-Hosted allows administrators to connect its features to models hosted on their chosen infrastructure, including Microsoft Foundry. This provides control over hosting, region, network path, and credentials, addressing data residency and regulatory needs. Microsoft Foundry serves as a versatile platform for various AI models, including OpenAI GPT, Anthropic Claude, and Meta Llama. GitLab's flexibility enables assigning specific models to individual Duo features, optimizing for different workloads and cost efficiencies. The setup process is consistent across model families, with variations only in the specific model deployed and configuration details. It is crucial to verify model support from both GitLab and Foundry, as newer versions from Foundry are not automatically compatible with GitLab's matrix. GitLab Duo Self-Hosted is ideal for organizations operating within Azure, offering centralized management of deployments, access, and networking. The architecture involves a self-managed GitLab instance, an AI Gateway for request routing, and model endpoints hosted in Microsoft Foundry. Inference data, such as code inputs and model responses, remains within the user's network in a fully self-hosted configuration. When selecting models, consider their ratings across different capability areas and prioritize current, well-supported options. Prerequisites include a GitLab Premium or Ultimate Self-Managed instance and an Azure subscription with access to Microsoft Foundry. The implementation involves deploying models in Foundry, installing the AI Gateway, configuring GitLab to connect to the gateway, and then adding each Foundry deployment to GitLab.