OCR Configuration Guide for NVIDIA RAG Blueprint

June 2, 2026 ยท View on GitHub

This guide explains the OCR (Optical Character Recognition) services available in the NVIDIA RAG Blueprint, including configuration and deployment options.

Overview

The NVIDIA RAG Blueprint supports two OCR services:

  1. Nemotron OCR (Default) - High-performance OCR service offering 2x+ faster performance
  2. Paddle OCR (Legacy) - General-purpose OCR service maintained for compatibility

:::{tip} Nemotron OCR is now the default OCR service and is recommended for all new deployments due to its superior performance and efficiency. :::

Nemotron OCR (Default)

Nemotron OCR is the default and recommended OCR service for the NVIDIA RAG Blueprint, providing:

  • 2x+ faster performance compared to Paddle OCR
  • Optimized text extraction from documents and images
  • Enhanced accuracy for modern document layouts
  • Better resource efficiency

Key Features

  • High-performance optical character recognition
  • Support for various document types and image formats
  • GPU-accelerated processing
  • Production-ready with model control capabilities

Default Configuration

By default, the NVIDIA RAG Blueprint is configured to use Nemotron OCR with the following settings. For Nemotron OCR, use OCR_MODEL_NAME=pipeline (the previous default scene_text_ensemble applied to the legacy NeMo Retriever OCR service).

VariableDefault ValueDescription
OCR_GRPC_ENDPOINTnemotron-ocr:8001gRPC endpoint for OCR service
OCR_HTTP_ENDPOINThttp://nemotron-ocr:8000/v1/inferHTTP endpoint for OCR service
OCR_INFER_PROTOCOLgrpcCommunication protocol (grpc or http)
OCR_MODEL_NAMEpipelineOCR model to use (use pipeline for Nemotron OCR; legacy scene_text_ensemble was used with NeMo Retriever OCR)

Hardware Requirements

For detailed hardware requirements and GPU support, refer to the Nemotron OCR Support Matrix.

Docker Configuration

The Nemotron OCR service is configured in the Docker Compose file with the following key settings:

  • Image: nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0
  • GPU Memory: 8192 MB (default)
  • Max Batch Size: 32 (default)
  • Ports: 8012 (HTTP), 8013 (gRPC), 8014 (Metrics)

You can customize the GPU allocation by setting:

export OCR_MS_GPU_ID=0  # Specify which GPU to use
export OCR_CUDA_MEMORY_POOL_MB=8192  # Adjust memory allocation
export OCR_BATCH_SIZE=32  # Configure batch size
export OCR_OMP_NUM_THREADS=8  # Set OpenMP threads

Paddle OCR (Legacy)

Paddle OCR is maintained as a legacy option for compatibility with existing workflows. While still functional, it is recommended to migrate to Nemotron OCR for better performance.

When to Use Paddle OCR

Consider using Paddle OCR if you:

  • Need compatibility with existing Paddle OCR-based workflows
  • Have specific requirements that work better with Paddle OCR
  • Are migrating from an older deployment

Hardware Requirements

Docker Configuration

The Paddle OCR service configuration:

  • Image: nvcr.io/nim/baidu/paddleocr:1.5.0
  • GPU Memory: 3072 MB (default)
  • Ports: 8009 (HTTP), 8010 (gRPC), 8011 (Metrics)

:::{note} Legacy Service: Paddle OCR is maintained as a legacy option. For new deployments, we recommend using the default Nemotron OCR service for better performance. :::

Deployment Instructions

Docker Compose Deployment

Using Nemotron OCR (Default)

Nemotron OCR is deployed by default when you follow the standard deployment guide. No additional configuration is required.

  1. Prerequisites: Follow the deployment guide for standard setup.

  2. Start Services:

    USERID=$(id -u) docker compose -f deploy/compose/nims.yaml up -d

    :::{tip} Nemotron OCR is included in the default profile and will start automatically. :::

  3. Verify Service Status:

    watch -n 2 'docker ps --format "table {{.Names}}\t{{.Status}}"'

Switching to Paddle OCR

If you need to use Paddle OCR instead:

  1. Prerequisites: Follow the deployment guide up to and including the step labelled "Start all required NIMs."

  2. Configure Environment Variables:

    export OCR_GRPC_ENDPOINT=paddle:8001
export OCR_HTTP_ENDPOINT=http://paddle:8000/v1/infer
export OCR_INFER_PROTOCOL=grpc
export OCR_MODEL_NAME=paddle

  3. Stop Nemotron OCR if running:

    USERID=$(id -u) docker compose -f deploy/compose/nims.yaml down nemotron-ocr

  4. Deploy Paddle OCR Service:

    USERID=$(id -u) docker compose -f deploy/compose/nims.yaml --profile paddle up -d

  5. Restart Ingestor Server and NeMo Retriever Library Runtime:

    docker compose -f deploy/compose/docker-compose-ingestor-server.yaml up -d

  6. Test Document Ingestion: Use the ingestion API usage notebook to verify functionality.

NVIDIA-Hosted Deployment

Using Nemotron OCR (Default)

Follow the standard NVIDIA-hosted deployment guide - Nemotron OCR is the default configuration.

Using Paddle OCR with NVIDIA-Hosted Deployment

  1. Prerequisites: Follow the deployment guide up to and including the step labelled "Start the vector db containers from the repo root."

  2. Configure API Endpoints:

    export OCR_HTTP_ENDPOINT=https://ai.api.nvidia.com/v1/cv/baidu/paddleocr
export OCR_INFER_PROTOCOL=http
export OCR_MODEL_NAME=paddle

  3. Deploy Services: Continue with the remaining steps in the deployment guide to deploy ingestion-server and rag-server containers.

  4. Test Document Ingestion: Use the ingestion API usage notebook to verify functionality.

Helm Deployment

Using Nemotron OCR (Default)

Nemotron OCR is deployed by default with Helm installations. Follow the standard Helm Deployment Guide - no additional OCR configuration is required.

Using Paddle OCR with Helm

To use Paddle OCR instead of the default Nemotron OCR:

Modify values.yaml to override the OCR service image:

nv-ingest:
  nimOperator:
    # Override the OCR service to use PaddleOCR image
    ocr:
      enabled: true
      image:
        repository: nvcr.io/nim/baidu/paddleocr
        tag: 1.5.0
  
  # Update OCR model name
  envVars:
    OCR_MODEL_NAME: paddle

:::{note} The service endpoints (OCR_GRPC_ENDPOINT and OCR_HTTP_ENDPOINT) remain the same and do not need to be changed. The service name nemotron-ocr-v1 is retained even when using the PaddleOCR image. :::

After modifying values.yaml, apply the changes as described in Change a Deployment.

For detailed Helm deployment instructions, see Helm Deployment Guide.

OCR Configuration Reference

Environment Variables

VariableDescriptionNemotron DefaultPaddle DefaultRequired
OCR_GRPC_ENDPOINTgRPC endpoint for OCR servicenemotron-ocr:8001paddle:8001Yes (on-premises)
OCR_HTTP_ENDPOINTHTTP endpoint for OCR servicehttp://nemotron-ocr:8000/v1/inferhttp://paddle:8000/v1/inferYes
OCR_INFER_PROTOCOLCommunication protocolgrpcgrpcYes
OCR_MODEL_NAMEOCR model to usepipelinepaddleYes
OCR_MS_GPU_IDGPU device ID to use00No
OCR_CUDA_MEMORY_POOL_MBCUDA memory pool size81923072No
OCR_BATCH_SIZEMax batch size (Nemotron only)32N/ANo
OCR_OMP_NUM_THREADSOpenMP thread count88No

Advanced Configuration

For deployments with NIMs on different workstations or outside the nvidia-rag docker network:

export OCR_GRPC_ENDPOINT="workstation_ip:8001"

Replace workstation_ip with the actual IP address of the machine running the OCR service.

Switching Between OCR Services

Migrating from Paddle OCR to Nemotron OCR

To switch to the default Nemotron OCR service:

  1. Stop Paddle OCR:

    USERID=$(id -u) docker compose -f deploy/compose/nims.yaml down paddle

  2. Configure Nemotron OCR environment variables:

    export OCR_GRPC_ENDPOINT=nemotron-ocr:8001
export OCR_HTTP_ENDPOINT=http://nemotron-ocr:8000/v1/infer
export OCR_INFER_PROTOCOL=grpc
export OCR_MODEL_NAME=pipeline

  3. Start Nemotron OCR:

    USERID=$(id -u) docker compose -f deploy/compose/nims.yaml up -d nemotron-ocr

  4. Restart Ingestor Server:

    docker compose -f deploy/compose/docker-compose-ingestor-server.yaml up -d

Migrating from Nemotron OCR to Paddle OCR

Follow the steps in Switching to Paddle OCR above.

Performance Comparison

FeatureNemotron OCRPaddle OCR
Performance2x+ fasterBaseline
GPU Memory8 GB (default)3 GB (default)
Batch ProcessingUp to 32Limited
StatusRecommended (Default)Legacy
Use CaseAll new deploymentsLegacy compatibility

Troubleshooting

Common Issues

  1. OCR Service Not Starting

    • Check GPU availability: nvidia-smi
    • Verify NGC API key is set correctly
    • Check logs: docker logs <container-name>

  2. Connection Errors

    • Verify the OCR endpoint variables are set correctly
    • Ensure the OCR service is running: docker ps
    • Check network connectivity between services

  3. Performance Issues

    • Consider increasing OCR_CUDA_MEMORY_POOL_MB
    • Adjust OCR_BATCH_SIZE for Nemotron OCR
    • Verify GPU has sufficient memory

Getting Logs

# Nemotron OCR logs
docker logs nemotron-ocr

# Paddle OCR logs
docker logs paddle