Multi-Core Support in Synapse

Overview

Synapse has full multi-core support built-in. The application automatically detects available CPU cores and configures the Tokio runtime to utilize all available processing power.

Implementation Details

Automatic CPU Detection

The application detects available CPU cores at startup using Rust's standard library:

let num_cpus = std::thread::available_parallelism()
    .map(|n| n.get())
    .unwrap_or(1);

Runtime Configuration

Synapse uses a multi-threaded Tokio runtime configured to match the number of available CPU cores:

let mut builder = tokio::runtime::Builder::new_multi_thread();
builder.worker_threads(num_cpus);
builder.enable_all();
let runtime = builder.build()?;

Location: src/main.rs lines 60-227

Environment Variable Support

The TOKIO_WORKER_THREADS environment variable is automatically set if not already configured:

if std::env::var("TOKIO_WORKER_THREADS").is_err() {
    unsafe {
        std::env::set_var("TOKIO_WORKER_THREADS", num_cpus_early.to_string());
    }
}

This ensures that:

• The main Tokio runtime uses all cores
• Pingora proxy subsystem inherits the multi-core configuration
• All async operations can execute concurrently

Dependencies

Multi-core support is enabled through Tokio's rt-multi-thread feature:

tokio = { version = "1", features = [
  "rt-multi-thread",  # Enables multi-threaded runtime
  "macros",
  "net",
  "io-util",
  "signal",
  "sync",
] }

Location: Cargo.toml lines 18-25

Performance Characteristics

Benchmark Results

Based on test results on a 16-core system:

CPU-Intensive Workload

• Single-threaded: 81.00 ms for 8 tasks
• Multi-threaded (8 threads): 15.92 ms for 8 tasks
• Speedup: ~5.1x faster

High Throughput

• Throughput: 179,713 requests/second with 8 worker threads
• Latency: <6 microseconds per request

Realistic Proxy Workload

• Throughput: 3,555 requests/second
• 50 concurrent requests: completed in 14.06 ms

Thread Distribution

The runtime effectively distributes work across multiple threads:

• IO-bound tasks: Scheduled across worker threads
• CPU-bound tasks: Dispatched to blocking thread pool via spawn_blocking

Testing

Comprehensive test suites verify multi-core functionality:

Unit Tests (tests/multicore_test.rs)

• test_multiple_workers_are_used: Verifies tasks run on different threads
• test_concurrent_task_execution: Validates concurrent execution
• test_cpu_intensive_tasks_on_multiple_cores: Tests CPU work distribution
• test_parallel_work_distribution: Measures concurrent task execution

Integration Tests (tests/integration_multicore_test.rs)

• test_app_detects_multiple_cores: Validates CPU detection
• test_runtime_configuration_matches_main: Tests runtime setup
• test_high_concurrency_scenario: Simulates proxy workload
• test_mixed_cpu_and_io_workload: Tests realistic scenarios

Benchmarks (tests/multicore_benchmark.rs)

• Single-threaded vs multi-threaded comparisons
• CPU-intensive workload tests
• Realistic proxy workload simulation
• Memory allocation concurrency tests

Running Tests

# Run all multi-core tests
cargo test --test multicore_test --test integration_multicore_test -- --nocapture

# Run benchmarks
cargo test --test multicore_benchmark -- --nocapture --test-threads=1

# Run specific test
cargo test test_multiple_workers_are_used -- --nocapture

Configuration

Default Behavior

By default, Synapse uses all available CPU cores. No configuration required.

Override Worker Threads

To manually set the number of worker threads:

# Use 4 worker threads
export TOKIO_WORKER_THREADS=4
./synapse --config config.yaml

# Or inline
TOKIO_WORKER_THREADS=4 ./synapse --config config.yaml

Verification

Check the log output at startup to verify multi-core configuration:

[DEBUG] main() started, num_cpus=16, TOKIO_WORKER_THREADS=Ok("16")

Architecture Components Utilizing Multi-Core

1. HTTP Proxy (Pingora)

• Multiple worker threads handle incoming connections
• Each thread can process requests independently
• Location: src/http_proxy/start.rs

2. Background Workers

All workers run concurrently:

• Certificate Worker: TLS certificate management
• Config Worker: Configuration refresh
• Log Sender Worker: Event batching and transmission
• Threat MMDB Worker: Threat intelligence updates
• GeoIP MMDB Worker: GeoIP database updates

Location: src/worker/

3. BPF Statistics Collection

• Multiple interfaces processed concurrently
• Statistics aggregation runs in parallel
• Location: src/bpf_stats.rs

4. TCP Fingerprinting

• Concurrent fingerprint collection across interfaces
• Parallel event processing
• Location: src/utils/tcp_fingerprint.rs

5. Access Rules Enforcement

• XDP programs on multiple interfaces
• Concurrent rule evaluation
• Location: src/access_rules.rs

Performance Recommendations

For High Throughput

• Recommended: Let the system auto-detect (uses all cores)
• Manual tuning: Set TOKIO_WORKER_THREADS to match CPU core count

For Low Latency

• Recommended: Use all cores for parallel processing
• Consider: CPU affinity for dedicated worker threads (advanced)

For Resource-Constrained Systems

• Manually limit worker threads to conserve resources:

  TOKIO_WORKER_THREADS=2 ./synapse --config config.yaml
  

For Container Deployments

The application respects container CPU limits:

• Automatically detects available CPUs in cgroup
• No special configuration needed for Docker/Kubernetes

Monitoring Multi-Core Utilization

System Level

# Monitor per-thread CPU usage
top -H -p $(pgrep synapse)

# Detailed thread information
htop  # Press 'H' to show threads

Application Level

The application logs worker thread activity:

[DEBUG] Runtime worker thread active, thread_id=ThreadId(42), worker_index=0
[DEBUG] Runtime worker thread active, thread_id=ThreadId(43), worker_index=1

Metrics

• BPF statistics show per-interface packet processing
• Worker logs indicate concurrent task execution
• Thread IDs in logs demonstrate multi-thread activity

Troubleshooting

Issue: "Only 1 thread detected"

Cause: Running in severely constrained environment Solution: Check container/VM CPU allocation

Issue: "Tasks not running concurrently"

Cause: TOKIO_WORKER_THREADS=1 is set Solution: Unset the environment variable or increase value

Issue: "High CPU but low throughput"

Cause: CPU-bound tasks blocking async threads Solution: Ensure CPU-intensive operations use spawn_blocking

Issue: "Thread count exceeds CPU count"

Expected: Blocking thread pool is separate from worker threads Normal: Additional threads for blocking I/O are expected

Future Enhancements

Potential optimizations:

• Per-worker metrics and load balancing
• NUMA-aware thread pinning
• Dynamic worker thread adjustment based on load
• Work-stealing queue optimizations
• CPU affinity for critical paths

References

• Tokio Runtime Documentation
• Multi-threaded Runtime
• Work-stealing Scheduler

Summary

✅ Multi-core support is fully implemented and tested ✅ Automatic CPU detection and configuration ✅ Performance scales with CPU core count ✅ Comprehensive test coverage with benchmarks ✅ Production-ready for high-throughput deployments