🪱 Quantum Nematode

This project simulates a simplified nematode (C. elegans) navigating dynamic foraging environments to find food while managing satiety, using either a quantum variational circuit or a classical neural network as its decision-making brain. It leverages Qiskit to simulate quantum behavior and integrates classical logic for realistic foraging dynamics.

🧪 Features

• ✅ Dynamic Foraging Environment: Realistic multi-food foraging with satiety management and distance efficiency tracking
• ✅ Predator Evasion: Multi-objective learning with pursuit/stationary predators and gradient-based danger perception
• ✅ Temporal Sensing: Biologically-accurate sensing modes replacing oracle spatial gradients — scalar concentration (Mode A), derivative (Mode B), and klinotaxis head-sweep (Mode C) with Short-Term Associative Memory (STAM) buffers
• ✅ Thermotaxis: Temperature-guided navigation with comfort/discomfort/danger zones and scattered hot/cold spots
• ✅ Aerotaxis: Oxygen-guided navigation with asymmetric hypoxia/hyperoxia danger zones (5-12% O2 comfort range, URX/BAG neuron-inspired)
• ✅ Multi-Agent Simulations: Cooperative and competitive foraging with pheromone communication (food-marking, alarm, aggregation), social feeding (npr-1 mediated), food competition policies, and collective behavior metrics
• ✅ Modular Quantum Brain: Parameterized quantum circuits with 2+ qubits for decision-making
• ✅ Classical ML Alternatives: REINFORCE, PPO, DQN, LSTM/GRU PPO, and spiking neural network brain architectures
• ✅ Quantum Learning: Parameter-shift rule for gradient-based optimization
• ✅ Evolutionary Optimization & Inheritance: CMA-ES, genetic algorithms, and TPE hyperparameter search, plus Lamarckian weight inheritance, Baldwin-effect, and predator-prey co-evolution
• ✅ Connectome Substrate: Connectome-constrained brains on the real C. elegans wiring diagram (302 neurons, Cook et al. 2019) with chemical synapses and gap junctions
• ✅ Hardware Support: Classical simulation (AerSimulator) and real quantum hardware (IBM QPU)
• ✅ Comprehensive Tracking: Per-run and session-level metrics, plots, and CSV exports
• ✅ Interactive Workflows: CLI scripts with flexible configuration
• ✅ Multi-Agent Visualization: Real-time Pygame rendering of multi-agent simulations with per-agent colored sprites, viewport agent switching, and pheromone concentration overlays
• ✅ Pluggable Architecture Interface: Self-registering @register_brain plug-in registry for adding new brain architectures as comparable rows in one experimental sweep
• 🚧 Expandable Framework: Modular design for research and experimentation

🧠 Brain Architectures

Choose from 25 brain architectures spanning quantum, classical, hybrid, and biologically-inspired approaches:

Quantum:

• QVarCircuitBrain (qvarcircuit): Quantum variational circuit with modular sensory processing and parameter-shift rule gradients
• QRCBrain (qrc): Quantum reservoir computing with data re-uploading circuits and classical readout
• QRHBrain (qrh): Quantum reservoir hybrid with C. elegans-inspired structured topology, X/Y/Z+ZZ feature extraction, and PPO-trained classical readout
• QSNNReinforceBrain (qsnnreinforce): Quantum spiking neural network (QLIF neurons) with surrogate gradient REINFORCE
• QSNNPPOBrain (qsnnppo): QLIF quantum spiking network with PPO training
• QLIFLSTMBrain (qliflstm): Quantum-enhanced LSTM with QLIF gates for temporal memory, trained via recurrent PPO with chunk-based truncated BPTT
• QQLearningBrain (qqlearning): Hybrid quantum-classical Q-learning with experience replay
• QEFBrain (qef): Quantum entangled features — parameterized quantum circuit with configurable cross-modal entanglement topology (modality-paired, ring, random), Z+ZZ+cos/sin feature extraction, and PPO-trained classical readout
• QRHQLSTMBrain (qrhqlstm): QRH quantum reservoir with QLIF-LSTM temporal readout — reservoir feature extraction + recurrent PPO with truncated BPTT
• CRHQLSTMBrain (crhqlstm): CRH classical reservoir with QLIF-LSTM temporal readout — classical reservoir ablation companion to QRH-QLSTM
• EquivariantQuantumPPOBrain (equivariantquantum): Z2-equivariant parameterized quantum circuit with data re-uploading and odd/even-parity latent split, PPO-trained — ships with classical-equivariant and symmetry-prior ablation controls to isolate the equivariance and quantum contributions

Hybrid (Quantum + Classical):

• HybridQuantumBrain (hybridquantum): QSNN reflex + classical cortex MLP + classical critic with mode-gated fusion and 3-stage curriculum (96.9% on pursuit predators)
• HybridClassicalBrain (hybridclassical): Classical ablation control for HybridQuantum — replaces QSNN reflex with small classical MLP (96.3% on pursuit predators)
• HybridQuantumCortexBrain (hybridquantumcortex): QSNN reflex + QSNN cortex (grouped sensory QLIF) + classical critic with 4-stage curriculum (halted — 40.9% on 2-predator)

Classical:

• CRHBrain (crh): Classical reservoir hybrid — Echo State Network reservoir with configurable feature channels (raw, cos_sin, squared, pairwise) and PPO-trained classical readout; quantum ablation control for QRH
• MLPPPOBrain (mlpppo): Classical actor-critic with Proximal Policy Optimization (clipped objective, GAE)
• LSTMPPOBrain (lstmppo): LSTM/GRU-augmented PPO with chunk-based truncated BPTT, separate actor/critic optimizers, and entropy decay — designed for temporal sensing tasks where memoryless MLP processing is insufficient. GRU variant recommended (outperforms LSTM across all evaluated environments)
• MLPReinforceBrain (mlpreinforce): Classical multi-layer perceptron with policy gradients (REINFORCE)
• MLPDQNBrain (mlpdqn): Classical MLP with Deep Q-Network (DQN) learning
• CfCPPOBrain (cfcppo): CfC (Closed-form Continuous-time) liquid neural network with AutoNCP wiring and continuous-time recurrent dynamics, PPO-trained — an alternative recurrent substrate for temporal sensing
• TransformerPPOBrain (transformerppo): Transformer self-attention encoder over a temporal window of recent sensory features, PPO-trained — an attention-based temporal-memory comparator to the recurrent (LSTM/CfC) substrates
• FeedforwardGABrain (feedforwardga): Feed-forward network whose weights are evolved by the genetic-algorithm optimizer (gradient-free), with graded episodic-progress fitness for sparse-reward cells

Biologically-Inspired:

• SpikingReinforceBrain (spikingreinforce): Biologically realistic spiking neural network with LIF neurons and surrogate gradient learning
• SpikingPPOBrain (spikingppo): Recurrent adaptive-LIF spiking network with a configurable MLP actor head, trained via PPO
• ConnectomePPOBrain (connectomeppo): Connectome-constrained PPO on the real C. elegans connectome (Cook et al. 2019 hermaphrodite — chemical synapses + gap junctions) with biologically-faithful sensor→interneuron→motor projections and multi-hop recurrence

For full architecture comparison and benchmarks, see quantum-architectures.md and logbook 008.

Select the brain architecture when running simulations:

uv run ./scripts/run_simulation.py --brain hybridquantum     # Best quantum (96.9% pursuit predators)
uv run ./scripts/run_simulation.py --brain mlpppo            # Best classical (PPO actor-critic)
uv run ./scripts/run_simulation.py --brain lstmppo           # LSTM/GRU PPO (temporal sensing)
uv run ./scripts/run_simulation.py --brain crh               # Classical reservoir hybrid (QRH ablation control)
uv run ./scripts/run_simulation.py --brain spikingreinforce  # Biologically realistic (LIF spiking)
uv run ./scripts/run_simulation.py --brain qvarcircuit       # Quantum variational circuit

🚀 Quick Start

1. Install Dependencies

Install uv for dependency management and Git LFS for large file storage:

brew install uv git-lfs
git lfs install

Install the project (choose one based on your needs):

# For CPU simulation (recommended for beginners)
uv sync --extra cpu --extra pixel --extra torch

# For quantum hardware access (requires IBM Quantum account)
uv sync --extra qpu --extra pixel

# For GPU acceleration (local installation)
uv sync --extra gpu --extra pixel --extra torch

# For GPU acceleration (Docker with NVIDIA GPU support)
docker compose up --build

Docker GPU Requirements: For the Docker setup, you need Docker with NVIDIA Container Toolkit installed for GPU acceleration.

2. Configure Environment (Optional)

If using quantum hardware, set up your IBM Quantum API key:

cp .env.template .env
# Edit .env to add your IBM_QUANTUM_API_KEY

3. Run a Simulation

Command Line Examples:

# Hybrid quantum brain — QSNN reflex + classical cortex (best quantum: 96.9% on pursuit predators)
uv run ./scripts/run_simulation.py --log-level DEBUG --show-last-frame-only --track-per-run --runs 50 --config ./configs/scenarios/foraging/hybridquantum_small_oracle.yml --theme emoji

# Classical PPO brain (best classical: actor-critic with GAE)
uv run ./scripts/run_simulation.py --log-level DEBUG --show-last-frame-only --track-per-run --runs 50 --config ./configs/scenarios/foraging/mlpppo_medium_oracle.yml --theme emoji

# Spiking neural network brain (biologically realistic LIF neurons)
uv run ./scripts/run_simulation.py --log-level DEBUG --show-last-frame-only --track-per-run --runs 50 --config ./configs/scenarios/foraging/spikingreinforce_small_oracle.yml --theme emoji

# Quantum variational circuit brain
uv run ./scripts/run_simulation.py --log-level DEBUG --show-last-frame-only --track-per-run --runs 50 --config ./configs/scenarios/foraging/qvarcircuit_medium_oracle.yml --theme emoji

# Multi-agent cooperative foraging (5 agents with social feeding)
uv run ./scripts/run_simulation.py --log-level INFO --runs 10 --config ./configs/scenarios/multi_agent_foraging/mlpppo_medium_5agents_social_oracle.yml --theme pixel

# Quantum hardware (IBM QPU) with dynamic foraging
uv run ./scripts/run_simulation.py --log-level DEBUG --show-last-frame-only --track-per-run --runs 1 --config ./configs/scenarios/foraging/qvarcircuit_small_oracle.yml --theme emoji --device qpu

Docker GPU Examples:

# Run dynamic foraging with MLP brain and GPU acceleration
docker-compose exec quantum-nematode uv run ./scripts/run_simulation.py --log-level DEBUG --show-last-frame-only --track-per-run --runs 50 --config ./configs/scenarios/foraging/mlpreinforce_medium_oracle.yml --theme emoji

# Interactive Docker shell for development
docker-compose exec quantum-nematode bash

❓ How It Works

Dynamic Foraging Environment

1. State Perception: The nematode perceives its environment through modular sensory inputs — oracle gradients, temporal concentration scalars, or derivative signals (dC/dt) depending on sensing mode, plus proprioception, mechanosensation, and STAM temporal memory
2. Brain Processing: The selected brain architecture processes the state
3. Action Selection: Brain outputs action probabilities (forward, left, right, stay)
4. Environment Update: Agent moves, satiety decays, and receives reward signal
5. Food Collection: When reaching food, satiety is restored and new food spawns
6. Learning: Brain parameters are updated based on reward feedback
7. Repeat: Process continues until all foods are collected, satiety reaches zero (starvation), or maximum steps reached

Quantum Learning Process

The project supports multiple quantum learning approaches:

• Quantum Feature Encoding: Environmental data encoded as qubit rotations
• Parameterized Quantum Circuits: Trainable quantum gates for decision-making
• Surrogate Gradient Descent: Differentiable QLIF (Quantum LIF) neurons enabling backpropagation through quantum spiking layers — used by the highest-performing hybrid architectures
• Parameter-Shift Rule: Analytical quantum gradient computation for variational circuits
• Evolutionary Optimization: CMA-ES and genetic algorithms as gradient-free alternatives
• Entanglement: Quantum correlations between different sensory modules

Spiking Neural Network

The spiking brain architecture provides biologically realistic neural computation with modern gradient-based learning:

• Leaky Integrate-and-Fire (LIF) Neurons: Membrane potential dynamics with spike generation
• Surrogate Gradient Descent: Differentiable spike approximation enabling backpropagation
• Policy Gradient Learning (REINFORCE): Same proven algorithm as MLPBrain
• Population Coding: Gaussian tuning curves for improved input discrimination

Key Features:

• Biologically plausible temporal dynamics with LIF neurons
• Effective gradient-based learning through surrogate gradients
• Configurable network architecture (timesteps, hidden layers, hidden size)
• Achieves 100% success on foraging tasks, 63% on predator evasion

Predator Evasion

The predator evasion system adds a challenging multi-objective learning task where agents must balance food collection with survival:

Predator Mechanics:

• Random movement patterns with configurable speed (default 1 unit/step)
• Detection radius (default 8 units) creating danger zones
• Kill radius (default 0 units) for lethal collisions
• Multiple predators with independent movement

Perception Modes:

• Oracle sensing (default): Directional gradients — food attraction and predator repulsion via spatial gradient vectors
• Klinotaxis sensing (Mode C): Scalar concentration + lateral head-sweep gradient + temporal derivative (dC/dt) — most biologically complete, models C. elegans ASE neuron head sweeps with STAM temporal memory
• Derivative sensing (Mode B): Scalar concentration + temporal derivative (dC/dt) — biologically plausible, models sensory neuron temporal detection
• Temporal sensing (Mode A): Scalar concentration only — biologically honest klinokinesis, agent infers direction from LSTM/GRU memory of movement-concentration correlations
• STAM (Short-Term Associative Memory): Exponential-decay buffer storing recent sensory readings, position deltas, and action entropy for temporal integration

Learning Dynamics:

• Proximity penalty: Continuous negative reward when in danger zone (detection radius)
• Death penalty: Large negative reward (default -10.0) on predator collision
• Multi-objective optimization: Agents learn to collect food while avoiding threats
• Predator metrics: Track encounters, successful evasions, and survival strategies

🏆 Top Benchmarks

Track and compare performance across different brain architectures and optimization strategies. The benchmark system helps identify effective approaches and advances the state-of-the-art in quantum navigation.

Quick Start with Benchmarks

# Run 10+ independent training sessions
for session in {1..10}; do
    uv run scripts/run_simulation.py \
        --config configs/your_config.yml \
        --track-experiment \
        --runs 50
done

# Submit all sessions together
uv run scripts/benchmark_submit.py \
    --experiments experiments/* \
    --category foraging_small/classical \
    --contributor "Your Name"

# Regenerate leaderboards
uv run scripts/benchmark_submit.py regenerate

Current Leaders

Foraging Small - Classical

Brain	Score	Success Rate	Learning Speed	Stability	Distance Efficiency	Sessions	Contributor	Date
mlpppo	0.835 ± 0.007	96.7% ± 1.3%	0.93 ± 0.01	0.95 ± 0.05	0.47 ± 0.02	12	@chrisjz	2025-12-28
mlpreinforce	0.810 ± 0.014	95.1% ± 1.9%	0.91 ± 0.02	0.99 ± 0.03	0.39 ± 0.04	12	@chrisjz	2025-12-29

Foraging Small - Quantum

Brain	Score	Success Rate	Learning Speed	Stability	Distance Efficiency	Sessions	Contributor	Date
qvarcircuit	0.835 ± 0.006	99.8% ± 0.6%	0.80 ± 0.00	0.99 ± 0.04	0.46 ± 0.01	12	@chrisjz	2025-12-29

Predator Small - Classical

Brain	Score	Success Rate	Learning Speed	Stability	Distance Efficiency	Sessions	Contributor	Date
mlpppo	0.728 ± 0.029	83.3% ± 2.9%	0.92 ± 0.02	0.62 ± 0.05	0.51 ± 0.02	12	@chrisjz	2025-12-29
mlpreinforce	0.624 ± 0.123	73.4% ± 10.9%	0.84 ± 0.09	0.52 ± 0.19	0.39 ± 0.07	12	@chrisjz	2025-12-29

Predator Small - Quantum

Brain	Score	Success Rate	Learning Speed	Stability	Distance Efficiency	Sessions	Contributor	Date
qvarcircuit	0.611 ± 0.054	76.1% ± 2.1%	0.93 ± 0.04	0.47 ± 0.04	0.45 ± 0.01	12	@chrisjz	2025-12-29

See BENCHMARKS.md for complete leaderboards and submission guidelines.

📊 Simulation Visualization

The default Pixel theme renders the simulation in a Pygame window with biologically accurate sprites inspired by real C. elegans ecology.

Single-Agent Mode

Multi-Agent Mode

Multi-agent simulations render all agents with distinct colors. The viewport follows one agent at a time, with keyboard controls to switch between agents and toggle pheromone concentration overlays.

Controls:

Key	Action
← →	Cycle between agents
1-9	Jump to agent by number
P	Toggle pheromone concentration overlay (food=green, alarm=red, aggregation=blue)

Continuous-2D Substrate

The continuous-2D substrate renders with a dedicated fidelity renderer (--theme pixel_continuous): the worm moves at sub-cell resolution on a full-arena plate view, with a concentration-field heatmap, gradient/sensor overlays, and the adaptive-sensor readout. Run it with:

uv run ./scripts/run_simulation.py \
  --config configs/scenarios/foraging/mlpppo_small_continuous2d_fick_adaptive_klinotaxis.yml \
  --theme pixel_continuous

Controls:

Key	Action
H	Toggle the concentration-field heatmap
F	Cycle the heatmap field (food → predator → temperature → oxygen → pheromone)
G	Toggle the gradient quiver (up-gradient arrows; off by default)
C	Toggle the camera (full-arena plate ↔ agent-following zoom)

Entities

Entity	Visual	Biological Basis
Nematode head	Translucent rounded head with pharynx bulb, directional facing	C. elegans head morphology
Nematode body	Connected tan/cream segments with tapered tail	C. elegans body coloring
Multi-agent colors	8-color palette (cream, blue, green, red, orange, purple, cyan, yellow)	Visual differentiation for 2+ agents
Dead agent	Gray overlay with red X marker	Agent terminated (starved, killed, frozen)
Food	Green clustered dots	E. coli / OP50 bacterial lawns
Random predator	Purple branching tendrils	Nematode-trapping fungi (Arthrobotrys oligospora)
Stationary predator	Purple ring/net structure with toxic zone	Constricting ring traps (Drechslerella)
Pursuit predator	Orange-red arachnid shape	Predatory mites

Environment Layers

Layer	Description
Soil	Dark earth background with subtle texture
Temperature zones	Blue (cold) through neutral to red/orange (hot) overlays based on thermal gradient
Oxygen zones	Red (hypoxia) through neutral to cyan (hyperoxia) overlays based on O2 concentration
Toxic zones	Purple overlay around stationary predators indicating damage radius
Pheromone overlay	Togglable colored overlay showing pheromone concentration (green=food, red=alarm, blue=aggregation)

Status Bar

The status bar displays session-level information (run progress, cumulative wins, total food eaten, average steps) and run-level information (current step, food collected, health, satiety, danger status, temperature zone, oxygen zone). In multi-agent mode, it additionally shows the followed agent indicator, per-agent food counts, and alive/dead status.

Alternative Themes

Console-based themes (ASCII, Emoji, Rich, etc.) are available for terminal rendering in single-agent mode. A dedicated headless theme skips all rendering entirely for maximum performance in batch training and CI. Multi-agent simulations support pixel and headless themes only.

# Headless mode — no rendering overhead, fastest for training
uv run ./scripts/run_simulation.py --config ./configs/scenarios/foraging/mlpppo_small_oracle.yml --runs 50 --theme headless

Available themes: pixel (default), ascii, emoji, unicode, colored_ascii, rich, emoji_rich, headless.

Session Summary

After all runs complete, a summary report is printed to the console:

Total runs completed: 50
Successful runs: 30 (60.0%)
Failed runs - Starved: 2 (4.0%)
Failed runs - Health Depleted: 15 (30.0%)
Failed runs - Max Steps: 3 (6.0%)
Average foods collected per run: 8.18
Average steps per run: 300.20
Average reward per run: 1.93
Average distance efficiency: 0.32
Average survival score: 0.72
Average temperature comfort: 0.68
Success rate: 60.00%

🧰 Built With

• Qiskit: Quantum computing framework
• PyTorch: Classical neural networks
• uv: Modern Python dependency management
• Pydantic: Data validation and settings
• Rich: Beautiful terminal output

🔬 Research Applications

This project serves as a platform for exploring:

• Quantum Machine Learning: Investigating quantum advantages in learning tasks
• Biological Modeling: Simplified models of neural decision-making
• Hybrid Algorithms: Combining quantum and classical computation
• NISQ Applications: Near-term quantum computing applications

🗺️ Roadmap

See docs/roadmap.md for the comprehensive project roadmap.

Recently Completed

• Evolution & Inheritance: CMA-ES, genetic-algorithm, and TPE optimization plus Lamarckian weight inheritance across generations (the headline-positive Phase 5 result), with Baldwin-effect, predator-prey co-evolution arms-race, and transgenerational-memory studies
• Pluggable Architecture Interface: Self-registering @register_brain plug-in registry admitting MLP, recurrent, spiking, reservoir, quantum, hybrid, GA-evolved, and connectome-constrained brains as comparable rows in one experimental sweep
• Multi-Agent Simulations: Cooperative and competitive foraging with pheromone communication (food-marking, alarm, aggregation), social feeding (npr-1 mediated satiety modulation), food competition policies, collective behavior metrics (aggregation index, alarm evasion, food sharing), and real-time Pygame visualization with per-agent colored sprites and pheromone overlays
• Temporal Sensing: Biologically-accurate sensing replacing oracle spatial gradients — scalar concentration (Mode A) and derivative (Mode B) with STAM temporal memory buffers
• LSTM/GRU PPO Brain: Recurrent architecture with chunk-based truncated BPTT for temporal sensing tasks — achieves oracle-level converged performance with scalar-only sensing
• Aerotaxis: Oxygen sensing with asymmetric 5-zone system (URX/BAG neuron-inspired), combined thermal+oxygen environments with orthogonal gradients
• Enhanced Sensory Systems: Thermotaxis with hot/cold zones, mechanosensation (touch response), health/damage systems
• Advanced Predator Behaviors: Stationary traps with toxic zones, pursuit patterns with configurable speed/detection

Upcoming Features

• Connectome Architecture Comparison (in progress): Closed-loop learning and evolution on the real C. elegans connectome (302 neurons, Cook et al. 2019) as a focal architecture, with NEAT topology search ranking the wild-type connectome against evolved alternatives on klinotaxis, thermotaxis, and predator evasion
• Plasticity & Cross-Species Transfer: Biologically-plausible plasticity (STDP + neuromodulator-modulated) on the connectome, and P. pacificus transfer using Cook et al. 2025 connectome data
• Continuous Physics: Continuous 2D movement and realistic locomotion
• Advanced Quantum Algorithms: VQE, QAOA, quantum error mitigation, and hardware deployment
• Real-World Validation: WormBot deployment, C. elegans lab collaborations, cross-organism transfer (Drosophila, zebrafish)

Research Applications

This platform enables research in:

• Quantum advantages in reinforcement learning and biologically-relevant navigation tasks
• Bio-inspired quantum algorithms for multi-objective decision-making
• Comparative analysis of quantum, classical, and spiking neural architectures
• Hybrid quantum-classical computation in ecologically-valid environments
• Near-term quantum device applications (NISQ algorithms with error mitigation)
• Theoretical foundations linking quantum mechanics to biological neural computation
• Universal computational principles transferable across organisms (C. elegans → Drosophila → zebrafish) and domains (foraging → robotics)

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for complete development setup instructions, code style guidelines, testing procedures, and pull request process.

Areas We Need Help With

• Quantum Algorithm Development: New quantum learning techniques for foraging
• Connectome & Plasticity: Connectome-constrained architectures, biologically-plausible plasticity (STDP + neuromodulation), and cross-species transfer
• Foraging Environment Extensions: Food quality variations, food spatial persistence, continuous action spaces
• Visualization Enhancements: Agent trail visualization, frame recording/video export, heatmaps
• Documentation: Tutorials and examples for dynamic environments
• Testing: Performance benchmarks and foraging strategy analysis

📄 License

This project is licensed under the Apache License 2.0. See LICENSE for details.

🙏 Acknowledgments

• Q-CTRL: For providing quantum hardware access with Fire Opal performance management tools to suppress quantum hardware errors and optimize quantum circuits
• OpenSpec: For providing the OpenSpec framework for structured, spec-driven AI development
• C. elegans Research Community: For inspiring this computational model
• Qiskit Team: For providing excellent quantum computing tools
• Quantum ML Community: For advancing the field of quantum machine learning