Secure AI agent deployment to GKE

5 months ago 107

Building AI agents is

exciting, but deploying them securely to production shouldn't be

complicated. In this tutorial, you will learn how GitLab's native Google Cloud integration makes it straightforward to deploy AI agents to Google Kubernetes Engine (GKE) — with built-in scanning and zero service account keys.

Why choose GKE to deploy your AI agents?

GKE provides enterprise-grade orchestration that connects seamlessly with GitLab CI/CD pipelines through OIDC authentication. Your development team can deploy AI agents while maintaining complete visibility, compliance, and control over your cloud infrastructure. This guide uses Google's Agent Development Kit (ADK) to build the app, so you can expect increased seamlessness as this is deployed using GitLab.

Three key advantages to this approach:

Full infrastructure control - Your data, your rules, your environment. You maintain complete control over where your AI agents run and how they're configured.

Native GitLab integration - No complex workarounds. Your existing pipelines work right out of the box thanks to GitLab's native integration with Google Cloud.

Production-grade scaling - GKE automatically handles the heavy lifting of scaling and internal orchestration as your AI workloads grow.

The key point is that GitLab with GKE provides the enterprise reliability your AI deployments demand without sacrificing the developer experience your teams expect.

Prerequisites

Before you start, make sure you have these APIs enabled:

  • GKE API

  • Artifact Registry API

  • Vertex AI API

Also make sure you have:

  • GitLab project created

  • GKE cluster provisioned

  • Artifact Registry repository created

The deployment process

1. Set up IAM and permissions on GitLab

Navigate to your GitLab integrations to configure Google Cloud authentication (IAM).

Go to Settings > Integrations and configure the Google Cloud integration. If you're using a group-level integration, notice that default settings are already inherited by projects. This means you configure once at the group level, and all projects benefit and inherit this setting.

To set this up from scratch, provide:

  • Project ID

  • Project Number

  • Workload Identity Pool ID

  • Provider ID

Once configured, GitLab provides a script to run in Google Cloud Console, via Cloud Shell. The outcome of running this script is a Workload Identity Federation pool with the necessary service principal to enable the proper access.

2. Configure Artifact Registry integration

Still in GitLab's integration settings, configure Artifact Management:

  1. Click Artifact Management.

  2. Select Google Artifact Registry.

  3. Provide:

    • Project ID
    • Repository Name (created beforehand)
    • Repository Location

GitLab provides another script to run in Google Cloud Console.

Important: Before proceeding, add these extra roles to the Workload Identity Federation pool:

  • Service Account User

  • Kubernetes Developer

  • Kubernetes Cluster Viewer

These permissions allow GitLab to deploy to GKE in subsequent steps.

3. Create the CI/CD pipeline

Now for the key part — creating the CI/CD pipeline for deployment.

Head to Build > Pipeline Editor and define your pipeline with four stages:

  • Build - Docker creates the container image.

  • Test - GitLab Auto DevOps provides built-in security scans to ensure there are no vulnerabilities.

  • Upload - Uses GitLab's built-in CI/CD component to push to Google Artifact Registry.

  • Deploy - Uses Kubernetes configuration to deploy to GKE.

Here's the complete .gitlab-ci.yml:

default: tags: [ saas-linux-2xlarge-amd64 ] stages: - build - test - upload - deploy variables: GITLAB_IMAGE: $CI_REGISTRY_IMAGE/main:$CI_COMMIT_SHORT_SHA AR_IMAGE: $GOOGLE_ARTIFACT_REGISTRY_REPOSITORY_LOCATION-docker.pkg.dev/$GOOGLE_ARTIFACT_REGISTRY_PROJECT_ID/$GOOGLE_ARTIFACT_REGISTRY_REPOSITORY_NAME/main:$CI_COMMIT_SHORT_SHA GCP_PROJECT_ID: "your-project-id" GKE_CLUSTER: "your-cluster" GKE_REGION: "us-central1" KSA_NAME: "ai-agent-ksa" build: image: docker:24.0.5 stage: build services: - docker:24.0.5-dind before_script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY script: - docker build -t $GITLAB_IMAGE . - docker push $GITLAB_IMAGE include: - template: Jobs/Dependency-Scanning.gitlab-ci.yml - template: Jobs/Container-Scanning.gitlab-ci.yml - template: Jobs/Secret-Detection.gitlab-ci.yml - component: gitlab.com/google-gitlab-components/artifact-registry/upload-artifact-registry@main inputs: stage: upload source: $GITLAB_IMAGE target: $AR_IMAGE deploy: stage: deploy image: google/cloud-sdk:slim identity: google_cloud before_script: - apt-get update && apt-get install -y kubectl google-cloud-sdk-gke-gcloud-auth-plugin - gcloud container clusters get-credentials $GKE_CLUSTER --region $GKE_REGION --project $GCP_PROJECT_ID script: - | kubectl apply -f - <<EOF apiVersion: apps/v1 kind: Deployment metadata: name: ai-agent namespace: default spec: replicas: 2 selector: matchLabels: app: ai-agent template: metadata: labels: app: ai-agent spec: serviceAccountName: $KSA_NAME containers: - name: ai-agent image: $AR_IMAGE ports: - containerPort: 8080 resources: requests: {cpu: 500m, memory: 1Gi} limits: {cpu: 2000m, memory: 4Gi} livenessProbe: httpGet: {path: /health, port: 8080} initialDelaySeconds: 60 readinessProbe: httpGet: {path: /health, port: 8080} initialDelaySeconds: 30 --- apiVersion: v1 kind: Service metadata: name: ai-agent-service namespace: default spec: type: LoadBalancer ports: - port: 80 targetPort: 8080 selector: app: ai-agent --- apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: ai-agent-hpa namespace: default spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: ai-agent minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: {type: Utilization, averageUtilization: 70} EOF kubectl rollout status deployment/ai-agent -n default --timeout=5m EXTERNAL_IP=$(kubectl get service ai-agent-service -n default -o jsonpath='{.status.loadBalancer.ingress[0].ip}') echo "Deployed at: http://$EXTERNAL_IP" only: - main

The critical configuration for GKE

What makes this work — and why we need this extra configuration for GKE— is that we must have a Kubernetes Service Account in the cluster that can work with Vertex AI. We need that service account to be permitted to access the AI capabilities of Google Cloud.

Without this, we can deploy the application, but the AI agent won't work. We need to create a Kubernetes Service Account that can access Vertex AI.

Run this one-time setup:

#!/bin/bash PROJECT_ID="your-project-id" GSA_NAME="ai-agent-vertex" GSA_EMAIL="${GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" KSA_NAME="ai-agent-ksa" CLUSTER_NAME="your-cluster" REGION="us-central1" # Create GCP Service Account gcloud iam service-accounts create $GSA_NAME \ --display-name="AI Agent Vertex AI" \ --project=$PROJECT_ID # Grant Vertex AI permissions gcloud projects add-iam-policy-binding $PROJECT_ID \ --member="serviceAccount:${GSA_EMAIL}" \ --role="roles/aiplatform.user" # Get cluster credentials gcloud container clusters get-credentials $CLUSTER_NAME \ --region $REGION --project $PROJECT_ID # Create Kubernetes Service Account kubectl create serviceaccount $KSA_NAME -n default # Link accounts kubectl annotate serviceaccount $KSA_NAME -n default \ iam.gke.io/gcp-service-account=${GSA_EMAIL} gcloud iam service-accounts add-iam-policy-binding ${GSA_EMAIL} \ --role=roles/iam.workloadIdentityUser \ --member="serviceAccount:${PROJECT_ID}.svc.id.goog[default/${KSA_NAME}]" \ --project=$PROJECT_ID

4. Deploy to GKE

Once you're done, push this change to the pipeline and you're good to go.

You can see the pipeline has just deployed. Go to CI/CD > Pipelines and you'll see the four stages:

  • Build

  • Test (with all defined security scans)

  • Upload to Artifact Registry (successful)

  • Deploy to Kubernetes in GKE (success)

Summary

With GitLab and Google Cloud together, you're able to deploy your AI agent to GKE with ease and security. We didn't have to go through a lot of steps — we were able to do that thanks to GitLab's native integration with Google Cloud.

Watch this demo:

Use this tutorial's complete code example to get started now. Not a GitLab customer yet? Explore the DevSecOps platform with a free trial. Startups hosted on Google Cloud have a special perk to try and use GitLab.

Read Entire Article