---

duck_net

50+ network protocols as DuckDB SQL functions, written in pure Rust.

Query HTTP APIs, send emails, execute SSH commands, read from Redis, publish to Kafka, search LDAP directories — all from SQL.

Quick Start

-- Load the extension
LOAD 'target/release/libduck_net.so';

-- Query a JSON API
SELECT (http_get('https://api.github.com/repos/duckdb/duckdb')).body;

-- POST with custom headers
SELECT (http_post(
    'https://httpbin.org/post',
    MAP {'Content-Type': 'application/json'},
    '{"key": "value"}'
)).body;

-- Store credentials securely (never in query text)
SELECT duck_net_add_secret('my_api', 'http', '{"bearer_token": "sk-..."}');
SELECT (http_get_secret('my_api', 'https://api.example.com/data')).body;

-- Check security status
SELECT duck_net_security_status();
FROM duck_net_security_warnings();

Supported Protocols (49+)

Category	Protocols
Web	HTTP/S, GraphQL, SOAP, OData, WebDAV, WebSocket, gRPC, JSON-RPC, XML-RPC
DNS	DNS (A/AAAA/PTR/TXT/MX), DNS-over-HTTPS, mDNS/Bonjour
Email	SMTP/SMTPS, IMAP/IMAPS
File Transfer	FTP/FTPS, SFTP, SCP
Remote Execution	SSH (key + password auth)
Directory	LDAP/LDAPS (search, bind, add, modify, delete)
Cache/KV	Redis, Memcached
Cloud Storage	S3-compatible (AWS, MinIO, GCS, R2)
Messaging	MQTT, AMQP/RabbitMQ, Apache Kafka, NATS, ZeroMQ
Monitoring	Prometheus, Elasticsearch, InfluxDB
Service Discovery	HashiCorp Consul, HashiCorp Vault
Network	Ping, WHOIS, STUN, BGP Looking Glass
Time	NTP, PTP (Precision Time Protocol)
Infrastructure	SNMP (v2c), Syslog (RFC 5424), IPMI, RADIUS
Certificates	TLS Inspect, OCSP, CalDAV

Documentation

Full documentation is available in the docs/ directory:

• Getting Started — Installation and first queries
• Protocol Reference — All 50+ functions with signatures
• Security Architecture — Threat model and defense layers
• Secrets Management — In-memory credential store
• DuckDB Secrets Integration — Native CREATE SECRET support
• Security Warnings — Runtime insecure configuration alerts
• Hardening Guide — Production deployment checklist
• SSRF Protection — Private IP blocking

Security Highlights

Protection	Description
SSRF protection	Blocks all private/reserved IPv4 + IPv6 ranges (CWE-918); full coverage: CGN, NAT64, Teredo, 6to4
Redirect SSRF	Manual redirect following with per-hop SSRF check; HTTPS→HTTP downgrade warning
Header injection	RFC 7230 validation for header names and values; CRLF injection blocked (CWE-113)
Secrets manager	In-memory credentials with zeroize crate (compiler-resistant zeroing)
DuckDB secrets	Native CREATE SECRET for S3/HTTP/GCS/R2 with scoped, persistent, and STS support
AWS STS support	session_token in S3 secrets or duck_net_import_aws_env() for assumed roles
Security warnings	Runtime alerts for plaintext protocols, missing auth, S3 over HTTP, redirect downgrade
Input validation	URL length, hostname format, path traversal, LDAP filter, query size limits
TLS everywhere	Pure Rust rustls; no OpenSSL dependency
Response limits	256 MiB HTTP, 16 MiB gRPC/WS/Redis, 10 MiB IMAP, 1 MiB query payloads
CSPRNG	OS entropy via getrandom; panics instead of weak fallback
Credential scrubbing	Passwords, tokens, Authorization headers redacted in all error messages

-- Security audit
SELECT duck_net_security_status();

-- View any security warnings from this session
FROM duck_net_security_warnings();

-- Disable SSRF for local dev (re-enable for production!)
SELECT duck_net_set_ssrf_protection(false);

Secrets Management

Keep credentials out of SQL query text:

-- Store
SELECT duck_net_add_secret('mail', 'smtp',
    '{"host":"smtp.gmail.com","port":"587","username":"me@gmail.com","password":"app-pass"}');

-- Use
SELECT smtp_send_secret('mail', 'me@gmail.com', 'team@co.com', 'Alert', 'Error rate high');

-- List (values redacted)
FROM duck_net_secrets();

-- Rotate credentials atomically (old values zeroized before replacement)
SELECT duck_net_rotate_secret('mail', '{"host":"smtp.gmail.com","password":"new-pass"}');

-- Clear (zeroized in memory)
SELECT duck_net_clear_secret('mail');

For S3/HTTP, prefer DuckDB's native secrets:

CREATE SECRET my_s3 (TYPE s3, KEY_ID 'AKIA...', SECRET '...', REGION 'us-east-1');

See DuckDB Secrets Integration for details.

Building

git clone https://github.com/tomtom215/duck_net.git
cd duck_net
cargo build --release

Requires Rust 1.88+ (MSRV). The extension is built to target/release/libduck_net.so.

-- Load (unsigned)
LOAD 'target/release/libduck_net.so';

Protocol Opt-In

Only core web protocols are enabled by default (HTTP, DNS, TLS, GraphQL, OAuth2, WHOIS, secrets, audit log). Everything else must be explicitly enabled in a config file to reduce attack surface.

# Create ~/.config/duck_net/protocols and list what you need:
echo "ssh" >> ~/.config/duck_net/protocols
echo "smtp" >> ~/.config/duck_net/protocols
echo "redis" >> ~/.config/duck_net/protocols

-- From DuckDB: see all protocols and their enabled status
SELECT * FROM duck_net_protocols();

-- Generate a fully-commented config template
SELECT duck_net_generate_config();

See docs/src/installation.md for full details.

Audit Logging

-- Enable session audit log
SELECT duck_net_set_audit_logging(true);

-- Review all network operations made this session (credentials scrubbed)
SELECT * FROM duck_net_audit_log();

-- JSON summary
SELECT duck_net_audit_log_status();

Architecture

• Pure Rust — No C dependencies. Built on rustls for TLS, quack-rs for DuckDB FFI.
• Modular — Each protocol is a self-contained module with its own FFI bindings.
• Secure by default — Core web only; all other protocols opt-in via config file.
• Security-first — Centralized validation in security.rs, all credentials in secrets.rs.
• Bounded — Every buffer, cache, and response has enforced limits.

License

MIT