What's Here

This document provides an overview of Jepsen's various namespaces and how they work together.

Core Namespaces

These are the main namespaces involved in writing and running a test.

The main namespace is jepsen.core. The core function, jepsen.core/run! takes a test map, runs it, and returns a completed test map.

jepsen.cli supports command-line frontends to Jepsen tests. It provides commands for running a single test, a family of tests, and running a web server to browse results.

jepsen.os defines the protocol for an operating system, which is used to install basic packages on a node and prepare it to run a test. Most of the time you'll be using jepsen.os.debian, which supports Debian operating systems. There are also community-contributed, but generally unsupported jepsen.os.centos, jepsen.os.smartos, and jepsen.os.ubuntu.

jepsen.db defines the core protocols for database automation: installation, teardown, starting, killing, pausing, resuming, downloading logs, and so on, along with some basic utilities. jepsen.db.watchdog supports restarting databases.

jepsen.generator generates the operations Jepsen performs during a test. It provides (mostly) pure, composable data structures which yield effects. During a test, jepsen.generator.interpreter is responsible for launching worker threads, drawing operations from the generator, dispatching them, and journaling operations to the history. Two internal libraries, jepsen.generator.translation-table and jepsen.generator.context, help generators keep track of the time, what threads are doing, and workload-specific context. jepsen.generator.test helps with writing tests for generators.

Most operations from the generator are passed to a jepsen.client, which applies them to the database. Clients also support basic lifecycle management: opening and closing connections, and setting up and tearing down any state they need for the test.

Fault injection operations are passed to a jepsen.nemesis, which injects faults; this namespace provides some common faults and ways to compose nemeses together. jepsen.nemesis.combined provides composable packages of a nemesis, generators, and metadata. jepsen.nemesis.membership and its supporting protocol jepsen.nemesis.membership.state helps write nemeses that add and remove nodes from a cluster. jepsen.nemesis.time injects clock errors. jepsen.nemesis.file corrupts files.

The OS, DBs, nemeses, and more use jepsen.control, which runs commands via SSH (or other means) on DB nodes. jepsen.control.core defines the basic Remote protocol for running commands and transferring files, and some common functions for shell escaping and error management. The implementations are in jepsen.control.clj-ssh, jepsen.control.docker, jepsen.control.k8s, jepsen.control.retry, jepsen.control.scp, and jepsen.control.sshj. Two utility libraries, jepsen.control.util and jepsen.control.net, help with downloading URLs, making temporary files, getting IP addresses, running daemons, etc.

The interpreter produces a jepsen.history, which needs to be analyzed. jepsen.checker defines the protocol for checking a history and composing checkers together. It also provides a slew of basic checkers for common problems, like computing overall statistics, finding unhandled errors in the database and test itself, checking sets and queues, and so on. jepsen.checker.perf renders performance plots. jepsen.checker.timeline renders HTML timelines of a history.

jepsen.store writes tests to disk, as well as auxiliary data files. jepsen.store.format handles writing and reading Jepsen's binary file format. jepsen.store.fressian defines how Jepsen serializes data with Fressian.

Utilities

These supporting libraries round out Jepsen's capabilities.

jepsen.util is the "kitchen sink" of Jepsen. It provides a wealth of helpful utilities: everything from computing the majority of n nodes to timeouts to random values

jepsen.net, supported by jepsen.net.proto, provides pluggable support for network partitions between nodes, as well as losing or slowing down packets.

jepsen.reconnect provides automatic reconnection and retries for flaky clients. It's used internally by jepsen.control's SSH remotes, and also DB clients.

jepsen.fs-cache provides a local (e.g. on the control node) cache for arbitrary data structures and files. This is helpful when a database requires an expensive one-time setup step.

jepsen.codec encodes and decodes values to strings, for testing systems which store everything in string blobs.

jepsen.web is Jepsen's web interface, for browsing the store/ directory.

jepsen.repl is where lein repl drops you.

Tests

These namespaces are oriented towards running a specific kind of test, like a list-append test. They usually provide a package of a generator, client, and checker together.

jepsen.tests provides a no-op test map, as a stub for writing new tests.

jepsen.tests.adya provides a test for a specific kind of G2 anomaly. It's replaced by jepsen.tests.cycle.

jepsen.tests.bank models transfers between a pool of bank accounts, and checks to make sure the total balance is conserved.

jepsen.tests.causal-reverse looks for a violation of Strict Serializability where T1 < T2, but T2 is visible without T1. It's replaced by jepsen.tests.cycle.

jepsen.tests.cycle contains two workloads, jepsen.tests.cycle.append and jepsen.tests.cycle.wr, which are small wrappers around Elle, a sophisticated transactional safety checker.

jepsen.tests.kafka is for testing Kafka-style logs, in which clients publish messages to totally-ordered logs, and subscribe to messages from them.

jepsen.tests.linearizable-register is for testing a family of independent Linearizable registers. Useful for systems like etcd or Consul.

jepsen.tests.long-fork finds Long Fork anomalies. It's mostly superseded by jepsen.tests.cycle.list-append.

Extensions

These namespaces extend Jepsen to new workloads, faults, or styles of testing.

jepsen.independent lets you take a generator, client, and checker, and run several independent copies of them in the same test, either sequentially or concurrently. This is helpful when a workload needs to be "small"--say only a few keys or a few operations--but you'd like to do lots of copies of it through the course of a test.

jepsen.role supports tests where nodes have distinct roles. For instance, some might handle storage, and others transactions. It provides composable DB, Client, Nemesis, and Generator code to glue together these different subsystems.

jepsen.faketime helps run databases within an LD_PRELOAD shim that lies about the system clock.

jepsen.lazyfs runs databases within a directory that can lose un-fsynced writes on command.