README.md

Benchmark tips

env MIMALLOC_SHOW_STATS=1 MIMALLOC_LARGE_OS_PAGES=1 MIMALLOC_RESERVE_HUGE_OS_PAGES_AT=0 numactl --membind=0 --cpunodebind=0 cargo bench --features "metrics-rt" micro

Currently, the benchmark can only run on Linux.

Enable SPDK

Make sure you enable spdk in Cargo.toml by uncommenting out these lines in that file:

spdk-rs = { git = "https://github.com/openebs/spdk-rs.git", branch = "release/2.5", optional = true }
crossbeam-queue = { version = "0.3.11", optional = true }
spdk = ["dep:spdk-rs", "dep:crossbeam-queue"]

Prerequisites

All prerequisites are for Linux and are optional to run the benchmark. Those steps make sure your results are consistent with the numbers in the paper.

Setup huge pages

Use the following command to reserve huge pages (20GB).

sudo sysctl -w vm.nr_hugepages=10240

To confirm huge pages are populated

cat /proc/meminfo | grep HugePages

HugePages_Total:   10240
HugePages_Free:    10240
HugePages_Rsvd:        0
HugePages_Surp:        0

Monitor disk usage

Some useful commands:

Check the disk IO:

iostat 2 -x -h nvme0n1

Benchmark guide

cd benchmark

In-memory benchmark

env SHUMAI_FILTER="inmemory" MIMALLOC_LARGE_OS_PAGES=1 cargo run --bin bftree --release

Benchmark different storage backends

env SHUMAI_FILTER="storage" MIMALLOC_LARGE_OS_PAGES=1 cargo run --bin bftree --release

Guide to analyze the HdrHistogram

When benchmarked with metrics-rt feature, the benchmark will generate a histogram file in the target\benchmark directory. The file is in the HdrHistogram format.

To read and analyze it using python:

with open('18-52.hdr', 'rb') as f:
	de = base64.b64encode(f.read())
	histogram = HdrHistogram.decode(de)

percentiles = [0.1* x for x in range(1, 1000)]
values = [histogram.get_value_at_percentile(p) for p in percentiles]

tail_percentiles = [50, 90, 99, 99.9, 99.99, 99.999, 99.9999]
tail_values = [histogram.get_value_at_percentile(p) for p in tail_percentiles]

sns.barplot(x=tail_percentiles, y=tail_values)
sns.lineplot(x=values, y=percentiles)