Testing

This document describes how to run tests for obs-mcp. Run make help to see all available targets.

Linting

Run golangci-lint to check code quality:

make lint        # check
make lint-fix    # auto-fix

Unit Tests

make test-unit

Manual Testing

OpenShift — via kubeconfig (route auto-discovery):

make run             # auto-discovers Thanos Querier route (default backend)
make run-prometheus  # auto-discovers Prometheus route (--metrics-backend prometheus)
make run-no-guardrails  # auto-discovers Thanos route, guardrails disabled (use for Thanos < v0.40.0)

OpenShift — via port-forward (header auth, useful when kubeconfig lacks a bearer token):

make run-openshift-pf-prometheus     # port-forwards prometheus-k8s-0:9090 + alertmanager-main-0:9093

kube-prometheus or any other backend — set URLs explicitly:

PROMETHEUS_URL=http://localhost:9090 ALERTMANAGER_URL=http://localhost:9093 AUTH_MODE=header make run

Note: AUTH_MODE=header is required for Kind clusters because their kubeconfig uses client certificates instead of bearer tokens. The default kubeconfig auth mode will fail with a "kubeconfig doesn't contain a bearer token" error.

Override other defaults as needed:

LISTEN_ADDR=:8080 LOG_LEVEL=info make run

Kind-based E2E Tests

Tests obs-mcp against a local Kind cluster with kube-prometheus.

make test-e2e-full          # setup + deploy + test + teardown in one command

Or step by step:

make test-e2e-setup         # create Kind cluster
make test-e2e-deploy        # build and deploy obs-mcp
make test-e2e               # run tests
make test-e2e-teardown      # cleanup

OpenShift E2E Tests

Validates route auto-discovery (pkg/openshift) and tool correctness against OpenShift monitoring.

TestRouteDiscovery_* exercises pkg/openshift directly using the kubeconfig — no running obs-mcp needed. TestOpenShiftMetricsPresent requires OBS_MCP_URL and is skipped when not set. In CI, OBS_MCP_URL is set automatically by the step registry to point at the deployed obs-mcp instance.

Authentication: TestRouteDiscovery_URLsAreReachable queries monitoring routes directly. It uses OPENSHIFT_TOKEN if set (required in CI where the kubeconfig uses client certs that the OAuth proxy rejects), otherwise falls back to the kubeconfig bearer token (works with oc login).

# CI: export a token for route authentication
export OPENSHIFT_TOKEN=$(oc create token prometheus-k8s -n openshift-monitoring)

Route discovery only

Verifies route auto-discovery, URL shape, and that each route responds HTTP 200 when accessed with a bearer token against a real /api endpoint.

make test-e2e-openshift

Full suite including MCP tool smoke tests

Start obs-mcp in one terminal, then run the tests in another:

make run             # Thanos Querier route (default)
make run-prometheus  # or Prometheus route

OBS_MCP_URL=http://localhost:9100 make test-e2e-openshift   # OpenShift route discovery + metrics
OBS_MCP_URL=http://localhost:9100 make test-e2e             # full MCP tool smoke tests

Note: make test-e2e without OBS_MCP_URL will attempt a port-forward to a Kind/k8s cluster. It will fail if no obs-mcp pod is running in the obs-mcp namespace.

MCPChecker Evals

Validates that AI agents can discover and correctly use obs-mcp tools. See evals/mcpchecker/README.md for installation, environment setup, and detailed usage.

Quick start:

make run-mcpchecker-eval                          # run all tasks in parallel (1 run each)
make run-mcpchecker-eval CATEGORY=queries          # run by category (metrics, labels, queries, alerts, traces, logs, otelcol)
make run-mcpchecker-eval CATEGORY=logs             # Loki log evals
make run-mcpchecker-eval TASK=cpu-usage             # single task, verbose
make run-mcpchecker-eval RUNS=3                     # multiple runs for consistency testing

See evals/mcpchecker/README.md for installation, environment setup, and detailed usage.