AI Services GPU Validation
December 6, 2025 ยท View on GitHub
Generated: 2025-12-06
This document validates GPU detection and acceleration for local AI services running on the AMD Radeon RX 7700 XT.
Service Overview
| Service | Status | Port | GPU Acceleration |
|---|---|---|---|
| Ollama (ROCm) | Running | 11434 | Yes |
| Whisper (ROCm) | Running | 9000 | Yes |
| Edge-TTS | Running | 8880 | CPU (not GPU-accelerated) |
Docker Images
| Image | Tag | Size |
|---|---|---|
| ollama/ollama | rocm | 6.4 GB |
| whisper-rocm | latest | 29.7 GB |
| chatterbox-tts-rocm | latest | 19.5 GB |
| rocm/pytorch | latest | 29.3 GB |
| travisvn/openai-edge-tts | latest | 189 MB |
Ollama (LLM Inference)
Container Details
- Container Name: ollama-rocm
- Image: ollama/ollama:rocm
- Port: 0.0.0.0:11434 -> 11434/tcp
GPU Detection
Ollama successfully detects the GPU with ROCm:
inference compute: id=GPU-[REDACTED] library=ROCm compute=gfx1101
name=ROCm0 description="AMD Radeon Graphics"
driver=60342.13 pci_id=0000:03:00.0 type=discrete
total="12.0 GiB" available="11.8 GiB"
Environment Variables (Container)
HIP_VISIBLE_DEVICES=0
HSA_OVERRIDE_GFX_VERSION=11.0.1
OLLAMA_CONTEXT_LENGTH=4096
OLLAMA_DEBUG=INFO
OLLAMA_HOST=http://0.0.0.0:11434
Installed Models
| Model | Size | Last Modified |
|---|---|---|
| danielorpheus:latest | 3.5 GB | 7 days ago |
| legraphista/orpheus:latest | 2.4 GB | 7 days ago |
| nomic-embed-text:latest | 274 MB | 2 weeks ago |
| mistral:7b | 4.4 GB | 3 weeks ago |
| qwen3-vl:8b | 6.1 GB | 3 weeks ago |
| gemma3:4b | 3.3 GB | 3 weeks ago |
| qwen2.5-coder:3b | 1.9 GB | 3 weeks ago |
| llama3.2:3b | 2.0 GB | 3 weeks ago |
Whisper (Speech-to-Text)
Container Details
- Container Name: whisper-rocm
- Image: whisper-rocm:latest (custom build)
- Port: 0.0.0.0:9000 -> 9000/tcp
- Image Size: 29.7 GB
GPU Detection
Device set to use cuda:0
Note: ROCm uses HIP which provides CUDA compatibility layer, so cuda:0 indicates successful GPU detection.
TTS Services
Edge-TTS (CPU-based)
- Container Name: edge-tts
- Image: travisvn/openai-edge-tts:latest
- Port: 0.0.0.0:8880 -> 5050/tcp
- GPU: Not used (cloud-based TTS via Microsoft Edge API)
Chatterbox TTS (ROCm)
- Image: chatterbox-tts-rocm:latest
- Size: 19.5 GB
- Status: Image available, not currently running
Note: Chatterbox TTS is built with ROCm support for GPU-accelerated local TTS inference.
ComfyUI (Image Generation)
Installation Location
/home/daniel/programs/media-production/image-editing/image-to-video/ComfyUI
Launch Script
The ComfyUI run script configures ROCm environment:
#!/bin/bash
cd ~/programs/media-graphics/image-to-video/ComfyUI
# Set ROCm environment variables
export HSA_OVERRIDE_GFX_VERSION=11.0.1
export ROCM_PATH=/opt/rocm
# Disable xformers (not compatible with ROCm)
export XFORMERS_DISABLED=1
export DISABLE_XFORMERS=1
python main.py "$@"
Validation Commands
Test Ollama GPU
docker exec ollama-rocm ollama list
docker logs ollama-rocm 2>&1 | grep -i gpu
Test PyTorch ROCm
python3 -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
Check Container GPU Access
docker exec <container> rocminfo | grep -i "gfx"