LDBC SNB DuckDB implementation

April 14, 2023 · View on GitHub

DuckDB implementation of the LDBC Social Network Benchmark's Interactive workload.

Setup

Grab DuckDB:

scripts/get.sh

The data sets need to be generated before loading it to the database. No preprocessing is required. To generate the data sets for DuckDB, use the same settings as for PostgreSQL, i.e. the Hadoop-based Datagen's CsvMergeForeign serializer classes.

Running the benchmark

Set the following environment variable based on your data source:

export DUCKDB_CSV_DIR=`pwd`/../postgres/test-data

Loading the data set

Load the data set as follows:

scripts/load.sh

Running the benchmark driver

The instructions below explain how to run the benchmark driver in one of the three modes (create validation parameters, validate, benchmark). For more details on the driver modes, check the "Driver modes" section of the main README.

Create validation parameters

Edit the driver/benchmark.properties file. Make sure that the ldbc.snb.interactive.scale_factor, ldbc.snb.interactive.updates_dir, ldbc.snb.interactive.parameters_dir properties are set correctly and are in sync.
Run the script:
```
driver/create-validation-parameters.sh
```

Validate

Edit the driver/validate.properties file. Make sure that the validate_database property points to the file you would like to validate against.
Run the script:
```
driver/validate.sh
```

Benchmark

Edit the driver/benchmark.properties file. Make sure that the ldbc.snb.interactive.scale_factor, ldbc.snb.interactive.updates_dir, and ldbc.snb.interactive.parameters_dir properties are set correctly and are in sync.
Run the script:
```
driver/benchmark.sh
```

Reload between runs

:warning: The default workload contains updates which are persisted in the database. Therefore, the database needs to be reloaded or restored from backup before each run. Use the provided scripts/backup-database.sh and scripts/restore-database.sh scripts to achieve this.