SIE AKS Terraform Module

August 2, 2026 · View on GitHub

One command to get a GPU-ready AKS cluster for SIE (Search Inference Engine). The module creates everything you need - VNet, AKS, GPU pools, container registry, autoscaling - so you can focus on running inference, not managing infrastructure.

What you get

AKS cluster with Workload Identity + OIDC issuer + AAD-RBAC
GPU node pool - pick your GPU via gpu_class: t4 (NC4as_T4_v3), a10 (NV6ads_A10_v5), a100 (NC24ads_A100_v4), or h100 (NC40ads_H100_v5)
Scale-to-zero - GPU pools scale down to zero when idle, so you only pay when running inference
Built-in cluster autoscaler - per-pool, configured directly on the AKS node pools (no separate Helm chart to deploy or upgrade)
NVIDIA device plugin - installed via Helm so GPU pods schedule immediately
ACR repository (opt-in) - private Premium-SKU container registry; image paths <acr>.azurecr.io/<project>/{sie-server,sie-gateway,sie-config}
Workload Identity - pods authenticate to Azure without stored credentials
Private endpoints (opt-in) - private connectivity to ACR + Storage via privatelink.* DNS zones
Managed CSI - persistent volumes work out of the box

Quick start

cd examples/dev-nc4ast4-spot
az login
az account set --subscription "<subscription_id>"
terraform init
terraform plan
terraform apply

That's it. After apply, configure kubectl and deploy SIE via Helm:

# Point kubectl at the new cluster
$(terraform output -raw kubectl_config_command)

# Deploy SIE (gateway, workers, KEDA, Prometheus, Grafana). The -f flag pulls
# the AKS overlay (values-aks.yaml) directly from the chart's source repo -
# it wires up KEDA, the t4 + a10 machine profiles, and the
# azure.workload.identity/use=true pod label the AKS Workload Identity webhook
# keys off of. Pin to a release tag instead of `main` for reproducible installs.
helm upgrade --install sie-cluster oci://ghcr.io/superlinked/charts/sie-cluster --version 0.6.26 \
  -f https://raw.githubusercontent.com/superlinked/sie/main/deploy/helm/sie-cluster/values-aks.yaml \
  --namespace sie --create-namespace \
  --set "serviceAccount.annotations.azure\.workload\.identity/client-id=$(terraform output -raw sie_workload_identity_client_id)" \
  $(terraform output -raw model_cache_helm_args)

Examples

Costs shown are approximate West Europe spot list prices at the time of writing - check the Azure pricing calculator for the current rate in your region.

Example	GPU	Cost	Description
`dev-nc4ast4-spot`	T4 (NC4as_T4_v3)	~$0.15/hr	Spot VMs, scale 0-5 nodes, minimal cost for development
`dev-nv6adsa10-spot`	A10 (NV6ads_A10_v5)	~$0.35/hr	Spot VMs, scale 0-5 nodes, 24 GiB VRAM for larger embedding bundles

Prerequisites

This is a per-cluster product module. It does not ship its own state-backend or CI-identity bootstrap - those are subscription-wide / landing-zone concerns that you (or your platform team) own once and reuse across every cluster.

Azure subscription + credentials. az login locally (an account with Contributor on the target subscription is sufficient), or a federated service principal via the GitHub Actions Azure OIDC flow for CI.
GPU quota in your target region. Request from the Azure portal Quotas blade - e.g. Standard NCASv3_T4 Family vCPUs for T4, Standard NVADSA10v5 Family vCPUs for A10, Standard NCadsH100v5 Family vCPUs for H100. H100 quota is the slowest to approve; file the request before the first apply.
Terraform >= 1.14.

CI authentication (GitHub Actions)

If you're running this from CI, the recommended path is the federated Azure OIDC flow - no long-lived secrets. Create a service principal with Contributor and User Access Administrator (the latter provisions role assignments, needed for the default create_model_cache=true and when create_acr=true), then set three repo variables (not secrets):

permissions:
  id-token: write
  contents: read
steps:
  - uses: azure/login@v3
    with:
      client-id: ${{ vars.AZURE_CLIENT_ID }}
      tenant-id: ${{ vars.AZURE_TENANT_ID }}
      subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}

For tightening the SP's RBAC to the minimum set this module actually needs (Network Contributor, AKS Contributor, AKS RBAC Cluster Admin, AcrPush, Monitoring Contributor, Key Vault Contributor, RBAC Admin), see the Azure docs on scoped role assignments.

Remote state

Each example ships a commented backend "azurerm" {} stub and a backend.hcl.example template. Provision a Storage Account + blob container once per subscription (any standard Azure remote-state pattern works), fill the placeholders in backend.hcl.example, then init with:

terraform init -backend-config=backend.hcl

Per-cluster only the key field changes.

Variables

Required

No variables are strictly required - all have sensible defaults. Override these for your environment:

Variable	Default	Description
`location`	`westeurope`	Azure region to deploy in
`project_name`	`sie`	Name prefix for all resources
`owner`	(required)	UPN of the human accountable for this cluster. Populates the CAF `Owner` tag - useful for cost attribution and required if your subscription has a CAF tag-baseline policy. Example: `alice@example.com`.

Provider configuration

Callers MUST set storage_use_azuread = true on their azurerm provider block when create_model_cache = true (the default in every shipped example). The module disables shared-access-key auth on the model-cache Storage Account, so the provider's post-create blob-service probe needs an AAD token instead of a SAS key. Without this, apply fails with 403 KeyBasedAuthenticationNotPermitted immediately after the storage account is created.

provider "azurerm" {
  features {}
  storage_use_azuread = true
}

Every example in this module already sets this.

Region zone constraints

A handful of Azure regions don't expose availability zone 1 (francecentral, southafricawest, brazilsoutheast, swedensouth, norwaywest, switzerlandwest, westcentralus). The module catches the combination of "restricted region + zone 1 in pool zones" at plan time with a clear error. If you deploy to one of these regions, override system_node_pool.zones and every gpu_node_pools[*].zones to a subset of the supported zones (typically ["2", "3"]).

CAF baseline tags

Azure landing zones aligned with Microsoft's Cloud Adoption Framework typically enforce four taxonomy tags (Environment, Owner, CostCenter, Workload) via subscription-level Audit/Deny policy. This module populates all four automatically from input variables so the cluster doesn't get flagged at apply time:

Variable	Default	CAF tag	Allowed values
`environment`	`nonprod`	`Environment`	`prod`, `nonprod`, `sandbox`, `shared`
`owner`	(required)	`Owner`	UPN, e.g. `alice@example.com`
`cost_center`	`sie-platform`	`CostCenter`	free-form string
`workload`	`sie`	`Workload`	free-form string

If your subscription has no such policy, the tags are still applied (and are useful for cost attribution) but cause no apply failures.

GPU configuration

Variable	Default	Description
`gpu_node_pools`	`[]`	List of GPU pool definitions. Each entry needs `name`, `gpu_class` (`t4` / `a10` / `a100` / `h100`), optional `spot`, `node_count`, `max_count`

Adding A100 or H100 once Azure quota is granted is a values-only change - append a new entry to gpu_node_pools:

gpu_node_pools = [
  { name = "t4spot", gpu_class = "t4",  spot = true, node_count = 0, max_count = 5 },
  { name = "a100",   gpu_class = "a100",              node_count = 0, max_count = 2 },
]

The default gpu_class mapping uses one-GPU VM sizes. For a multi-GPU worker pod, override gpu_node_pools[*].vm_size to a SKU with enough GPUs, then set the matching Helm workers.pools.<name>.gpu.count to the number of GPUs the pod should consume on one node.

GPU SKU cheat sheet:

Hourly prices are approximate West Europe on-demand list prices at the time of writing - region, term, and Reserved/Savings Plan commitments all materially change them. Check the Azure pricing calculator for the current rate.

`gpu_class`	VM size	GPU	VRAM	Approx. on-demand/hr	Best for
`t4`	Standard_NC4as_T4_v3	1x T4	16 GB	~$0.55	Development, small models
`a10`	Standard_NV6ads_A10_v5	1x A10	24 GB	~$1.10	Development, medium models
`a100`	Standard_NC24ads_A100_v4	1x A100	80 GB	~$3.50	Large models, production
`h100`	Standard_NC40ads_H100_v5	1x H100	80 GB	~$7.00	Maximum throughput

Networking

Variable	Default	Description
`vnet_cidr`	`10.0.0.0/16`	CIDR block for the cluster VNet
`system_subnet_cidr`	`10.0.0.0/22`	System pool subnet
`gpu_subnet_cidr`	`10.0.4.0/22`	GPU pool subnet
`private_endpoint_subnet_cidr`	`10.0.8.0/24`	Private-endpoint subnet
`enable_private_cluster`	`false`	Toggle a private API endpoint
`api_server_authorized_ip_ranges`	`[]`	CIDRs allowed to reach the API server
`create_ingress_public_ip`	`false`	Provision a static public IP for the ingress controller in the cluster RG so DNS survives a cluster destroy/recreate
`deletion_protection`	`true`	Place a CanNotDelete management lock on the AKS cluster (set false for dev)
`automatic_upgrade_channel`	`stable`	AKS auto-upgrade channel (`patch` / `rapid` / `stable` / `node-image` / `none`)

Container registry

Variable	Default	Description
`server_acr_repository_name`	`sie-server`	Repository path within the ACR
`gateway_acr_repository_name`	`sie-gateway`
`config_acr_repository_name`	`sie-config`
`create_acr`	`false`	Whether this module manages the ACR. Default `false` matches the chart's GHCR-by-default behaviour. Set `true` to opt in. `acr_*_repository_url` outputs are emitted regardless.
`acr_repository_prefix`	`null` -> `<project_name>`	Namespace prefix for ACR repos. Set to `""` to disable prefixing.

Workload Identity

Variable	Default	Description
`sie_namespace`	`sie`	Kubernetes namespace for SIE workloads
`sie_service_account_name`	`sie-server`	K8s SA federated to the workload UAMI

Outputs

After terraform apply, use these outputs to connect and deploy:

Output	Description
`kubectl_config_command`	Run this to configure kubectl
`cluster_name`	AKS cluster name
`cluster_endpoint`	AKS API FQDN (sensitive)
`sie_workload_identity_client_id`	Pass to Helm for workload identity
`acr_login_server`	ACR login server
`acr_server_repository_url`	Where to push `sie-server` images
`acr_gateway_repository_url`	Where to push `sie-gateway` images
`acr_config_repository_url`	Where to push `sie-config` images
`model_cache_bucket_url`	`abfs(s)://`-style URL - pass to Helm as `workers.common.clusterCache.url`
`model_cache_helm_args`	Pre-composed Helm `--set` flags for the cache
`ingress_public_ip`	Static ingress IP address (when `create_ingress_public_ip = true`)
`ingress_helm_args`	Pre-composed Helm `--set` flags for ingress-nginx (loadBalancerIP + LB-RG annotation)
`gpu_node_pool_vm_sizes`	Effective VM SKU per pool (resolved from `gpu_class`)

Architecture

                         ┌────────────────────────────────────────────────────┐
                         │                  Azure subscription                │
                         │                                                    │
┌──────────┐             │  ┌──────────────────────────────────────────────┐  │
│          │   HTTPS     │  │              VNet (10.0.0.0/16)              │  │
│  Client  │────────────▶│  │                                              │  │
│          │             │  │  ┌─────────────────────────────────────────┐ │  │
└──────────┘             │  │  │   AKS Cluster (AAD-RBAC + Workload ID)  │ │  │
                         │  │  │                                         │ │  │
                         │  │  │  ┌────────────┐    ┌─────────────────┐  │ │  │
                         │  │  │  │   Gateway  │───▶│  GPU Workers    │  │ │  │
                         │  │  │  │            │    │ (T4/A10/A100/H) │  │ │  │
                         │  │  │  └─────┬──────┘    └─────────────────┘  │ │  │
                         │  │  │        │                    │           │ │  │
                         │  │  │  ┌─────┴──────┐              │           │ │  │
                         │  │  │  │ sie-config │ (config control plane)  │ │  │
                         │  │  │  └────────────┘              │           │ │  │
                         │  │  │  ┌─────────────────────────────────────┐ │ │  │
                         │  │  │  │  KEDA · Prometheus · Grafana        │ │ │  │
                         │  │  │  └─────────────────────────────────────┘ │ │  │
                         │  │  │  ┌──────────────┐   ┌─────────────────┐  │ │  │
                         │  │  │  │ System pool  │   │  GPU pools      │  │ │  │
                         │  │  │  │ (B4ms)       │   │ (NC*/NV*)       │  │ │  │
                         │  │  │  └──────────────┘   └─────────────────┘  │ │  │
                         │  │  └─────────────────────────────────────────┘ │  │
                         │  │                                              │  │
                         │  │  ┌───────────┐  ┌───────────┐  ┌──────────┐  │  │
                         │  │  │    ACR    │  │  Storage  │  │   NAT    │  │  │
                         │  │  │ (images)  │  │  (cache)  │  │  GW      │  │  │
                         │  │  └───────────┘  └───────────┘  └──────────┘  │  │
                         │  └──────────────────────────────────────────────┘  │
                         └────────────────────────────────────────────────────┘

Pushing images to ACR

This is optional, because the official images are available under ghcr.io/superlinked/.

Requires create_acr = true (or an ACR managed by another stack - see acr_repository_prefix).

After terraform apply, push your SIE Docker images:

# Authenticate Docker to ACR
az acr login --name $(terraform output -raw acr_name)

# Push server image
docker tag sie-server:latest $(terraform output -raw acr_server_repository_url):latest
docker push $(terraform output -raw acr_server_repository_url):latest

# Push gateway image
docker tag sie-gateway:latest $(terraform output -raw acr_gateway_repository_url):latest
docker push $(terraform output -raw acr_gateway_repository_url):latest

# Push sie-config image
docker tag sie-config:latest $(terraform output -raw acr_config_repository_url):latest
docker push $(terraform output -raw acr_config_repository_url):latest

Model cache and payload store

SIE clusters benefit from two object-store-backed features that share a single blob container:

Model cache: pre-staged model weights at abfs://sie-cache@.../models/, so workers cold-start from blob storage rather than re-downloading from Hugging Face on every pod spin-up.
Payload store: large work-item payloads (images, long documents that exceed the 1 MiB NATS in-band budget) at abfs://sie-cache@.../payloads/, written by the gateway and read once by the worker. Garbage-collected by a runtime TTL plus a blob lifecycle rule.

Because the payload store is required for >1 MiB work items, the shared blob container is created by default (create_model_cache = true). With it enabled, the module:

Provisions a managed StorageV2 account with versioning, soft delete, and a lifecycle rule that deletes blobs under sie-cache/payloads/ after one day.
Attaches two ABAC-scoped role assignments to the SIE workload UAMI: Storage Blob Data Reader constrained to models/ and Storage Blob Data Contributor constrained to payloads/.
Optional CMEK via model_cache_kms_key_id (Key Vault key resource ID).
Locks the storage account's network ACL to Deny by default and allows only the cluster's system + GPU subnets (via the Microsoft.Storage service endpoint). Operators populating the cache from outside the VNet (e.g. running sie-admin cache populate from a laptop or CI runner) must add their egress IP to storage_allowed_ip_ranges. To allow additional subnets (e.g. a bastion), set storage_allowed_subnet_ids to an explicit list. Both knobs are ignored when enable_private_endpoints = true (private link disables the public path entirely).

After apply, pass the cache URL into Helm with one terraform output:

helm upgrade --install sie-cluster oci://ghcr.io/superlinked/charts/sie-cluster --version 0.6.26 \
  --set "serviceAccount.annotations.azure\.workload\.identity/client-id=$(terraform output -raw sie_workload_identity_client_id)" \
  $(terraform output -raw model_cache_helm_args)

The chart auto-derives payloadStore.url from workers.common.clusterCache.url, so a single --set for the cache covers both the optional weights cache (models/) and the payload store (payloads/); the payload_store_url output is exposed for visibility and can be wired explicitly via --set payloadStore.url=... for the rare override case. On the chart side payloadStore.enabled defaults to true, decoupled from the optional workers.common.clusterCache. Operators who bring their own storage can opt out (create_model_cache = false) and wire payloadStore.url themselves; skipping the payload store entirely means work items larger than 1 MiB (e.g. images) fail.

See infra/storage.tf and infra/identity.tf for the resource definitions.

Security features

This module follows Azure security best practices out of the box:

AAD-RBAC - no local admin users; cluster authn/authz through Azure AD
Workload Identity - pods exchange projected SA tokens for AAD tokens; no static credentials
TLS 1.2 minimum - enforced on Storage + ACR
NAT gateway egress - predictable outbound IPs for allowlisting
AcrPull on kubelet UAMI - image pulls without registry passwords
NVIDIA GPU taints - GPU nodes are tainted so only GPU workloads schedule on them
Container Insights - control-plane and node logs to Log Analytics (opt-in via enable_cloud_logging)
Model-cache storage on-VNet by default - when create_model_cache = true, the storage account's network ACL defaults to Deny, allowing only the cluster's system + GPU subnets (via Microsoft.Storage service endpoint). Add caller IPs through storage_allowed_ip_ranges or override the subnet allowlist via storage_allowed_subnet_ids.
Optional private endpoints - ACR + Storage on Private Link (when enable_private_endpoints = true, the network ACL is omitted because public access is already disabled).

Bring-your-own components

Some pieces of a production deployment are intentionally not turnkey:

Container registry - optional. Default create_acr = false matches the chart's GHCR default. Set true to opt in. To use an external registry, point the Helm chart at it via gateway.image.repository, workers.common.image.repository, and config.image.repository.
TLS certificate - BYO by default. Set ingress.tlsConfig.mode to one of: byo (supply your own kubernetes.io/tls Secret), cert-manager (annotates Ingress for Let's Encrypt HTTP-01; requires cert-manager in the cluster), self-signed (chart bootstraps a self-signed root CA - for air-gapped / on-prem), or disabled (no TLS resources; TLS terminated upstream of the Ingress).
DNS / domain - always BYO. The module does not provision Azure DNS zones or records. After terraform apply, take the ingress controller's LoadBalancer IP and create an A/AAAA record under a domain you control.
OIDC provider - BYO. When auth.enabled: true in the chart, set auth.oauth2Proxy.oidcIssuerUrl and the corresponding client ID / secret to your existing identity provider (Okta, Auth0, Google Workspace, Azure AD, ...).

Cleanup

terraform destroy

Important: GPU VMs can be expensive. Always destroy dev/test clusters when not in use. Spot pools (spot = true) can reduce cost significantly but can be evicted with no warning (eviction_policy = "Delete" - Azure Spot does not have an EC2-style 2-minute interruption notice).