Reliability and Capacity Planning

July 16, 2026 · View on GitHub

Reliability is the ability to deliver the intended service over time. Capacity is the amount of traffic, storage, and work the system can handle while meeting that goal.

Define a Service Objective

Avoid saying only “the site should always work.” Define measurable outcomes:

99.9% of homepage requests succeed each calendar month.
95% of successful responses begin within 800 milliseconds.
Certificate expiration is detected at least 21 days in advance.
Website restoration completes within four hours.

Choose targets appropriate to user impact and available operations. A target creates monitoring and staffing obligations.

Availability Arithmetic

Approximate monthly downtime budgets:

AvailabilityDowntime in a 30-day month
99%About 7 hours 12 minutes
99.9%About 43 minutes
99.99%About 4 minutes 19 seconds

The calculation alone does not prove the user experience. Scheduled maintenance, measurement scope, partial failures, and dependency outages must be defined.

Error Budgets

An error budget is the permitted unreliability implied by a service objective. It helps balance feature changes against stability work.

If the service has already consumed its monthly budget, reduce risky changes and focus on causes of failure instead of declaring the target met by changing the measurement.

Estimate Capacity

For a static site, begin with:

Requests per second
Average and peak response size
Concurrent connections
Network transfer
Disk space
Log growth
Backup size

For a dynamic site, also measure:

Application latency
Database queries per request
Database connections
Queue depth
Cache hit rate
CPU and memory per process
External API limits

Average Is Not Peak

A site averaging one request per second may receive hundreds after an announcement. Plan for realistic bursts and define how the system fails when limits are reached.

Prefer controlled degradation:

  • Return a clear temporary response.
  • Limit expensive endpoints.
  • Queue work with bounded capacity.
  • Preserve administrative access.
  • Protect the database from unlimited concurrency.

Load Testing Safety

Load test only systems you own or are explicitly authorized to test. Begin in a nonproduction environment with small request rates.

Define:

  • Maximum request rate
  • Duration
  • Test source
  • Abort threshold
  • Expected server behavior
  • Person monitoring the test

An uncontrolled test can become a denial-of-service event.

Dependency Budgets

The website can be no more available than critical dependencies unless it can continue without them.

Classify each dependency:

  • Required for every request
  • Required only for a feature
  • Required only during deployment
  • Required only for administration

Cache or degrade optional features so one analytics, font, or external API failure does not blank the whole page.

Change Risk

Reliability problems often arrive through changes rather than raw traffic.

Safer deployment controls include:

  • Configuration validation
  • Automated tests
  • Small releases
  • Health checks
  • Observable rollout
  • Fast rollback
  • Database migration planning
  • Change windows for high-risk work

Capacity Review Worksheet

Record:

Normal requests/second:
Peak requests/second:
Largest response:
Monthly transfer:
Current disk usage:
Daily log growth:
Backup duration:
Restore duration:
Database size:
Largest dependency risk:
Next capacity threshold:

Reliability Exercise

For your first website:

  1. Write one availability objective.
  2. Define an external HTTP check.
  3. Define a certificate-expiration alert.
  4. Estimate a peak traffic event.
  5. Identify the first likely resource limit.
  6. Write a rollback for the next deployment.

Continue to Part 7: Capstone Project.