dynamo-graph-deployment-request.mdx

Allowed values: constant arima kalman prophet

Prometheus metrics

These fields default from environment variables and are excluded from serialized output.

Advisory and diagnostics

Scheduling and plugin pipeline

  <span className="enum-values"><span className="enum-label">Allowed values:</span> <Badge intent="note" minimal>static_secret</Badge> <Badge intent="note" minimal>allow_unauthenticated</Badge></span>
</ParamField>
<ParamField path="static_secrets" type="object" default="{}">
  Map of `secret_value` to `subject_label`. Populate from a mounted Kubernetes Secret rather than hard-coding in the ConfigMap.
</ParamField>

Generated DGD overrides

spec.overrides.dgd is a partial, versioned DGD. It does not define or extend the profiler's candidate topology. Thorough profiling merges the override into each generated benchmark candidate before measurement and into the interpolation deployment. Every search strategy merges it into the final generated DGD. Use apiVersion: nvidia.com/v1beta1 and kind: DynamoGraphDeployment for new overrides. Components, pod-template containers, and container environment variables merge by name; atomic lists such as graph-level spec.env and container args replace their generated values.

To permit custom model repository code without depending on the selected topology, set spec.overrides.trustRemoteCode: true. The profiler applies --trust-remote-code to every generated vLLM or SGLang worker after the component topology is known. Enable this setting only for model repositories you trust.

Generated component names

An entry in spec.overrides.dgd.spec.components must use the exact, case-sensitive component name from the generated DGD. These are DGD component names, not Kubernetes Service resource names. The current profiler templates generate these components:

Setting spec.features.mocker.enabled: true replaces the inference workers with decode and prefill mocker components. Setting spec.features.planner adds a Planner component.

Add overrides before profiling starts. The DGDR spec becomes immutable during profiling, so you cannot inspect selectedConfig and then add overrides to the same resource. To inspect the names generated by a specific Dynamo release, follow the discovery workflow, then recreate the DGDR with the required overrides.

For example, set an environment variable on one generated component:

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeploymentRequest
metadata:
  name: qwen3-sglang
spec:
  model: Qwen/Qwen3-30B-A3B
  backend: sglang
  overrides:
    dgd:
      apiVersion: nvidia.com/v1beta1
      kind: DynamoGraphDeployment
      spec:
        components:
        - name: decode
          podTemplate:
            spec:
              containers:
              - name: main
                env:
                - name: TRITON_PTXAS_PATH
                  value: /usr/local/cuda/bin/ptxas

Overrides can modify only components present in the generated DGD. An unresolved spec.components or spec.services name causes DGD materialization to fail. A recognized deprecated name may be translated to the corresponding component in the selected topology with a warning. An override cannot add deployment topology. Labels and annotations merge, metadata.name selects the final DGD name, and other identity metadata is ignored. status, null values for typed fields, and field deletion are not supported.

To add flags to a container already present in the generated DGD, set $patch.args to append on that named container:

spec:
  overrides:
    dgd:
      apiVersion: nvidia.com/v1beta1
      kind: DynamoGraphDeployment
      spec:
        components:
        - name: Frontend
          podTemplate:
            spec:
              containers:
              - name: main
                $patch:
                  args: append
                args:
                - --router-mode
                - kv

The only supported modifier is args: append, and its args list must contain at least one non-empty string. The target container and its args list must already be present in the generated blueprint. The profiler blueprints define the complete default CLI for Frontend/main, so the example preserves python3 -m dynamo.frontend and appends the KV router flags. The override helper combines the argument lists and removes $patch before returning the final DGD. Without $patch, specifying args retains the standard replacement behavior and requires the complete desired list.

For worked examples, see DGDR Templates. For router configuration and EPP topologies, see the Router Guide and Gateway API Inference Extension.

SKU format

hardware.gpuSku uses lowercase underscore format (h100_sxm, not H100-SXM5-80GB).

Allowed values: gb200_sxm gb10 b200_sxm h200_sxm h100_sxm h100_pcie a100_sxm a100_pcie a30 l40s l40 l4 v100_sxm v100_pcie t4 mi200 mi300

Status

The operator maintains observed state under status. See Lifecycle for how phase and profilingPhase progress, and for the full condition-type list.

<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/runtime#RawExtension" target="_blank">runtime.RawExtension</a>

# Watch phase transitions
kubectl get dgdr my-model -n $NAMESPACE -w

# Detailed status, conditions, and events
kubectl describe dgdr my-model -n $NAMESPACE

# Profiling sub-phase
kubectl get dgdr my-model -n $NAMESPACE -o jsonpath='{.status.profilingPhase}'

# Profiling job logs
PROFILING_JOB=$(kubectl get dgdr my-model -n "$NAMESPACE" -o jsonpath='{.status.profilingJobName}')
kubectl logs -f "job/${PROFILING_JOB}" -c profiler -n "$NAMESPACE"

# View generated DGD spec (when autoApply: false)
kubectl get dgdr my-model -n $NAMESPACE \
  -o jsonpath='{.status.profilingResults.selectedConfig}' | python3 -m json.tool