Zarr Benchmarks

This repository contains benchmarks of Zarr V3 implementations.

Implementations Benchmarked

• zarrs/zarrs via zarrs/zarrs_tools
  • Read executable: zarrs_benchmark_read_sync
  • Round trip executable: zarrs_reencode
• google/tensorstore
• zarr-developers/zarr-python
  • With and without the ZarrsCodecPipeline from zarrs/zarrs-python
  • With and without dask

Implementation versions are listed in the benchmark charts.

make Targets

• pydeps: install python dependencies (recommended to activate a venv first)
• zarrs_tools: install zarrs_tools (set CARGO_HOME to override the installation dir)
• generate_data: generate benchmark data
• benchmark_read_all: run read all benchmark
• benchmark_read_chunks: run chunk-by-chunk benchmark
• benchmark_roundtrip: run roundtrip benchmark
• benchmark_all: run all benchmarks

Benchmark Data

All datasets are $1024x1024x1024$ uint16 arrays.

Name	Chunk / Shard Shape	Inner Chunk Shape	Compression	Size
Uncompressed	$2$56^{3}$$	-	None	2.00 GB
Compressed	$2$56^{3}$$	-	zstd 0	83 MB
Compressed + Sharded	$2$56^{3}$$	$6$4^{3}$$	zstd 0	439 MB

Benchmark System

• Dell 14 Pro Premium
  • CPU: Intel(R) Core(TM) Ultra 7 268V (8) @ 5.00 GHz
  • Memory: 32GB LPDDR5X 8533 MT/s
  • SSD: 2TB EG6 KIOXIA
  • OS: Arch Linux (6.18.2)

Round Trip Benchmark

This benchmark measures the minimum time and peak memory usage to "round trip" a dataset (potentially chunk-by-chunk).

• The disk cache is cleared between each measurement
• These are best of 5 measurements

Table of raw measurements (benchmarks_roundtrip.md)

Standalone

Dask

Read Chunk-By-Chunk Benchmark

This benchmark measures the minimum time and peak memory usage to read a dataset chunk-by-chunk into memory.

• The disk cache is cleared between each measurement
• These are best of 5 measurements

Table of raw measurements (benchmarks_read_chunks.md)

Standalone

Dask

Read Subchunk-By-Subchunk Benchmark

This benchmark measures the minimum time and peak memory usage to read a dataset subchunk-by-subchunk into memory. A subchunk is an inner chunk within a sharded chunk.

• The disk cache is cleared between each measurement
• These are best of 5 measurements

Table of raw measurements (benchmarks_read_subchunks.md)

Standalone

Read All Benchmark

This benchmark measures the minimum time and peak memory usage to read an entire dataset into memory.

• The disk cache is cleared between each measurement
• These are best of 5 measurements

Table of raw measurements (benchmarks_read_all.md)

Standalone

Dask