README.md

April 27, 2026 · View on GitHub

`MongoDB-ReplicaSet-Manager`

License Last Commit

Automated configuration, monitoring, and management of a MongoDB replica set in Docker Swarm.
Ensures continuous operation and adapts to topology changes for high availability and data consistency.

Read the full write-up: Building Production MongoDB Replica Sets in Docker Swarm — covers architecture decisions, failover benchmarks, and production lessons learned.

Quick Start

git clone https://github.com/BitWise-0x/MongoDB-ReplicaSet-Manager && cd MongoDB-ReplicaSet-Manager
./deploy.sh

Features

Deployment Intelligence

Intelligent Fresh Deployment: Configures and initiates a replica set from scratch with smart node detection, accounting for down or unavailable swarm nodes and deployment constraints.
Smart Redeployment Detection: Automatically detects container redeployments with changed IP addresses and immediately updates configuration without unnecessary delays or false initialization attempts.
Optimized Performance: Fast reconfiguration during full redeployments with intelligent primary detection using current task IPs instead of stale configuration data.
Advanced Startup Handling: Robust retry logic handles MongoDB transitional states (NotYetInitialized) during startup, distinguishing between temporary states and actual fresh deployments.

Runtime Management

Primary Node Management: Automatic primary designation, tracking, and failover handling with intelligent election timeout and forced reconfiguration when needed.
Dynamic Scaling: Real-time addition and removal of MongoDB nodes during runtime with automatic replica set reconfiguration and member management.
Comprehensive User Setup: Automated creation of MongoDB admin (root) accounts and initial application database users with proper authentication and permissions.
Continuous Topology Monitoring: Watches Docker Swarm changes and adjusts replica set configuration for IP changes, node additions/removals, and network topology updates.
Production-Ready Logging: Color-coded ANSI logging with contextual messages, eliminating misleading errors during normal operations.
Enterprise Scalability: Designed for multi-node Docker Swarm environments, tested against various outage scenarios and edge cases for maximum reliability.

Note: Primary discovery uses MongoDB's hello command and checks isWritablePrimary for accuracy across server versions.

Architecture

graph TD
    User["👤 Admin / App"]

    subgraph Docker Swarm
        db["<b>database</b><br>mongo:8.0.13<br>:27017 (internal)<br><i>global mode</i>"]
        ctrl["<b>dbcontroller</b><br>jackietreehorn/mongo-replica-ctrl<br><i>manager node only</i>"]
        nosql["<b>nosqlclient</b><br>mongoclient/mongoclient<br>:3030"]
    end

    subgraph Infrastructure
        net["<b>backend</b><br>encrypted overlay network"]
        secret["<b>mongo-keyfile</b><br>Docker secret"]
        vol_data["<b>mongodata</b><br>/data/db"]
        vol_cfg["<b>mongoconfigdb</b><br>/data/configdb"]
    end

    User -->|":3030"| nosql
    nosql --> db
    ctrl -->|"Docker socket"| db
    ctrl -->|"PyMongo<br>rs.reconfig()"| db
    db --- net
    ctrl --- net
    nosql --- net
    db ---|"keyfile auth"| secret
    db --- vol_data
    db --- vol_cfg

Requirements

Component	Version
MongoDB	8.0.x (recipe uses `8.0.13`; 7.0 compatible)
PyMongo Driver	4.15.x — pinned `>=4.15,<5`, included in controller image
Docker	>= `24.0.5`
OS	Linux >= Ubuntu 23.04 (`linux/amd64`, `linux/arm/v7`, `linux/arm64`)

Prerequisites

A Docker Swarm cluster (local or cloud) — tested on 6-node Swarm
Docker Stack recipe — see docker-compose-stack.yml
Environment variables — see mongo-rs.env
Deployment script — deploy.sh

How to Use

Set all required environment variables in mongo-rs.env (see Environment Variables below).

Modify docker-compose-stack.yml to include your application service. Set your app's MongoDB URI to:

mongodb://${MONGO_ROOT_USERNAME}:${MONGO_ROOT_PASSWORD}@database:27017/?replicaSet=${REPLICASET_NAME}

Deploy via ./deploy.sh — this will:
- Import environment variables
- Create a backend overlay network with encryption enabled
- Generate a keyfile for the replica set and add it as a Docker secret
- Spin up all stack services: mongo, dbcontroller, nosqlclient, and your application
- Run dbcontroller in global mode (one instance per Swarm node)
Monitor logs for output and errors (see Troubleshooting).
To remove the stack:
```
./remove.sh
# or
docker stack rm [stackname]
```
The _backend overlay network is not removed automatically (it is external). Leave it in place when redeploying to retain the original subnet and avoid connectivity issues.

Environment Variables

Defined in mongo-rs.env:

Variable	Default	Purpose
`STACK_NAME`	`myapp`	Docker stack name
`MONGO_VERSION`	`8.0.13`	MongoDB image tag
`REPLICASET_NAME`	`rs`	Replica set name
`BACKEND_NETWORK_NAME`	`${STACK_NAME}_backend`	Overlay network name
`MONGO_SERVICE_URI`	`${STACK_NAME}_database`	MongoDB service name
`MONGO_ROOT_USERNAME`	`root`	Admin username
`MONGO_ROOT_PASSWORD`	`password123`	Admin password
`INITDB_DATABASE`	`myinitdatabase`	Initial database name
`INITDB_USER`	`mydbuser`	Initial database user
`INITDB_PASSWORD`	`password`	Initial database password

How It Works

Smart Discovery: Identifies and assesses MongoDB services in the Docker Swarm with intelligent node state detection and constraint handling.
Deployment Intelligence: Distinguishes between fresh deployments, redeployments with IP changes, and dynamic scaling scenarios using advanced configuration analysis.
Optimized Configuration: Uses current task IPs for primary detection during redeployments, eliminating delays from stale configuration data.
Adaptive Management: Handles MongoDB startup transitional states with retry logic, ensuring reliable operation across various deployment scenarios.
Real-time Monitoring: Continuously adapts replica set configuration for network changes, node lifecycle events, and topology updates with minimal downtime.

The nosqlclient service included in the recipe provides a web UI at http://<any-swarm-node-ip>:3030 for browsing and managing the database.

The compose YML pulls the latest controller from DockerHub via jackietreehorn/mongo-replica-ctrl. You can also pull it manually (docker pull jackietreehorn/mongo-replica-ctrl:latest) or build locally using the included ./build.sh.

Troubleshooting / Additional Details

Logs

Check Docker service logs for the dbcontroller service. Enable DEBUG:1 in the compose YML for verbose output. The controller uses colored ANSI logging:

Color	Level
GREEN	INFO messages
YELLOW	WARNING messages and IP addresses
RED	ERROR messages
MAGENTA (bold)	PRIMARY nodes
CYAN	SECONDARY nodes and DEBUG messages
BOLD	ReplicaSet-related terms

docker service logs [servicename]_dbcontroller --follow

Example output:

Controller log output: member changes, config updates, primary election Controller log output: redeployment, config detection, primary failover

Common Issues

Environment — verify all variables are correctly set in mongo-rs.env.

Docker Stack Compose YML — dbcontroller must run as a single instance on a Swarm manager node. Multiple instances will perform conflicting actions. A restart policy is included in the service definition for error recovery.

IMPORTANT: The default MongoDB port 27017 is used internally only and is not published outside the Swarm by design. Publishing this port will break replica set management.

Firewalls / SELinux — Linux distributions using SELinux are known to cause issues with MongoDB. Check with sestatus and either disable or configure it for MongoDB. Also verify your firewall allows MongoDB traffic.

Networking — the _backend overlay network is assigned an address space automatically by Docker. To define a custom network space, uncomment the relevant section in deploy.sh. Do not remove this network between redeployments.

Persistent Data — the mongo service must run in global mode to prevent multiple instances sharing the same data directory. Volumes are defined as external to prevent accidental deletion between redeployments.

Swarm Nodes — for HA, run more than one Swarm manager so dbcontroller can restart on a different node if needed.

Healthchecks — the mongo-healthcheck script only verifies MongoDB process health (not cluster status). Cluster status is managed by dbcontroller. The script is POSIX sh compatible and uses mongosh ping.

MongoDB Configuration Check — run ./docker-mongodb_config-check.sh from any manager node to fetch rs.status() and rs.config() output:

./docker-mongodb_config-check.sh

members: [
    {
    _id: 1,
    name: '10.0.26.51:27017',
    health: 1,
    state: 2,
    stateStr: 'SECONDARY',
    ...
    },
    {
    _id: 2,
    name: '10.0.26.48:27017',
    health: 1,
    state: 1,
    stateStr: 'PRIMARY',     <--- should match dbcontroller log output
    ...
    }

Service Start-up — on initial deployment, image downloads and replica set initialization take time. Dependent services (e.g. nosqlclient, your application) may fail and restart until MongoDB is ready — this is expected behavior. MongoDB in replica set mode is not available until a primary is elected.

References & Versioning Notes

PyMongo driver docs: https://www.mongodb.com/docs/languages/python/pymongo-driver/current/
PyMongo API docs: https://pymongo.readthedocs.io/en/4.15.1/api/
Pinned versions:
- PyMongo: >=4.15,<5 (compatible with MongoDB 7.x/8.x and Python 3.13)
- Docker SDK for Python: >=7,<9

Contact

GitHub: github.com/BitWise-0x
DockerHub: hub.docker.com/u/jackietreehorn

License

MIT License — see LICENSE for details.