Own your AI: Self-Hosted GitLa... Note
GitLab

Own your AI: Self-Hosted GitLab Duo models with AWS Bedrock

GitLab Duo Self-Hosted allows organizations to leverage AI in software development while maintaining data control. This solution integrates GitLab with AI models hosted on platforms like AWS Bedrock, ensuring data remains within the organization's infrastructure. Key benefits include complete data privacy, model flexibility, and compliance adherence for regulated industries. The architecture involves a self-managed GitLab instance, an AI Gateway for routing requests, and an LLM backend. To implement, organizations need a GitLab Premium or Ultimate instance with a Duo Enterprise add-on license and AWS credentials. The process begins with installing the AI Gateway, followed by configuring GitLab to connect to it. Users then gain access to desired models on AWS Bedrock and configure a specific self-hosted model within GitLab. Finally, AI-powered features like Code Suggestions and GitLab Duo Chat are assigned to the configured self-hosted models. Verification involves running health checks, testing features, and reviewing AI Gateway logs. While the guide focuses on AWS Bedrock, GitLab Duo Self-Hosted also supports on-premises deployments with vLLM and Azure OpenAI Service. This approach offers a balance between AI innovation and robust data security.