Test Data Generator for WarDragon Analytics

January 30, 2026 · View on GitHub

Overview

test_data_generator.py generates realistic fake drone detection data for testing the WarDragon Analytics platform. It simulates:

Multiple WarDragon kits at different GPS locations
Realistic drone tracks with GPS waypoints and flight characteristics
FPV signal detections (5.8 GHz analog and DJI Digital)
System health metrics (CPU, memory, disk, temps, GPS)
Time progression over a configurable duration

Installation

Dependencies

For SQL output mode (default):

# No additional dependencies required
python app/test_data_generator.py --mode=sql --duration=1h > test_data.sql

For database mode:

# Install psycopg2 for direct database writes
pip install psycopg2-binary

Usage

Basic Examples

# Generate 2 hours of test data for 3 kits, write directly to database
python app/test_data_generator.py --mode=db --duration=2h --kits=3 --drones=15

# Generate SQL INSERT statements (for manual import or inspection)
python app/test_data_generator.py --mode=sql --duration=1h --kits=1 --drones=5

# Save SQL to file
python app/test_data_generator.py --mode=sql --duration=30m > test_data.sql
psql -U wardragon -d wardragon -f test_data.sql

Command-Line Options

Option	Description	Default
`--mode`	Output mode: `sql` (print SQL) or `db` (write to database)	`sql`
`--duration`	Duration of data to generate (e.g., `2h`, `30m`, `1h30m`)	`2h`
`--kits`	Number of kits to simulate	`3`
`--drones`	Average number of drones per kit	`15`
`--db-url`	PostgreSQL connection URL (for `--mode=db`)	`postgresql://wardragon:wardragon@localhost:5432/wardragon`
`--signal-probability`	Probability of FPV signal per 5s interval (0.0-1.0)	`0.3`

Advanced Examples

# Small dataset for quick testing
python app/test_data_generator.py --mode=db --duration=15m --kits=1 --drones=3

# Large dataset with many signals
python app/test_data_generator.py --mode=db --duration=4h --kits=5 --drones=20 --signal-probability=0.6

# Custom database connection
python app/test_data_generator.py \
  --mode=db \
  --duration=1h \
  --db-url="postgresql://admin:secret@analytics.example.com:5432/wardragon"

# Generate SQL for specific time period
python app/test_data_generator.py --mode=sql --duration=2h30m --kits=2

Generated Data

Kits

Location: Realistic GPS coordinates from major US cities
Status: All kits marked as "online"
API URLs: Simulated local network addresses

Drone Tracks

Makes/Models: DJI (Mini 3 Pro, Mavic 3, Air 3, etc.), Autel, Skydio, Parrot
Flight patterns: Random waypoint-based flights (3-8 waypoints)
Duration: 5-45 minutes per flight
Speed: 5-20 m/s
Altitude: 30-120 meters
Remote ID: Realistic operator IDs, serial numbers, MAC addresses
Updates: Position every 5 seconds (matching DragonSync poll rate)

FPV Signals

Analog FPV: 5.8 GHz frequencies (Bands A, B, E, F, R)
DJI Digital: 5725-5865 MHz
Power levels: -85 to -45 dBm
Detection rate: Configurable probability (default 30% per 5s interval)

System Health

GPS: Stationary kit position with realistic GPS jitter
Metrics: CPU (20-60%), Memory (40-70%), Disk (30-52%)
Temperatures: CPU (40-70°C), GPU (35-60°C)
Updates: Every 30 seconds
Uptime: Starts at simulation start time

Database Schema

The generator expects TimescaleDB tables as defined in architecture.md:

kits - Kit registry
drones (hypertable) - Drone/aircraft tracks
signals (hypertable) - FPV signal detections
system_health (hypertable) - Kit health metrics

Performance

Typical Generation Rates

Duration	Kits	Drones	Records Generated	Time (db mode)
15m	1	5	~1,500	~5 seconds
1h	3	15	~20,000	~15 seconds
2h	3	15	~40,000	~30 seconds
4h	5	20	~120,000	~2 minutes

Batch Inserts

Uses psycopg2.extras.execute_batch() for efficient bulk inserts
Batch size: 1,000 records
ON CONFLICT handling prevents duplicate data

Troubleshooting

"psycopg2 not installed" error

pip install psycopg2-binary

Database connection failed

# Check TimescaleDB is running
docker ps | grep timescale

# Test connection manually
psql -U wardragon -h localhost -d wardragon

# Verify connection string
python app/test_data_generator.py --mode=db --db-url="postgresql://user:pass@host:port/db"

Too much data generated

# Reduce duration and number of drones
python app/test_data_generator.py --mode=db --duration=30m --kits=2 --drones=5

# Lower signal detection probability
python app/test_data_generator.py --mode=db --signal-probability=0.1

SQL output too large

# Generate smaller dataset
python app/test_data_generator.py --mode=sql --duration=15m --kits=1 > small_test.sql

# Or pipe directly to database
python app/test_data_generator.py --mode=sql --duration=1h | psql -U wardragon -d wardragon

Integration with Analytics UI

After generating test data:

Web UI: Navigate to http://localhost:8090 to see drone tracks on the map
Grafana: View pre-built dashboards at http://localhost:3000

Verify data: Query database directly

SELECT COUNT(*) FROM drones;
SELECT COUNT(*) FROM signals;
SELECT COUNT(*) FROM system_health;
SELECT * FROM kits;

Data Characteristics

Realistic Elements

GPS trajectories: Drones follow waypoint-based flight paths
Speed variation: Speeds vary during flight (50-100% of max speed)
Heading calculation: Calculated based on direction of travel
Pilot location: Stays fixed, within 5km of kit
Home point: Set at pilot location
RSSI: Realistic signal strength (-90 to -40 dBm)
Time progression: Data covers entire duration with 5s intervals

Limitations

No terrain awareness: Drones don't avoid obstacles
Simplified physics: Linear interpolation between waypoints
Random flight patterns: Not based on real-world mission profiles
Static pilot: Pilot doesn't move during flight

Future Enhancements

Potential improvements:

Aircraft (ADS-B) track generation
Geofence violations
Multi-day data generation
CSV export support
Configuration file for custom scenarios
Mission templates (survey, inspection, patrol)
Terrain-aware flight paths

License

Apache 2.0 (same as DragonSync and WarDragon Analytics)