PSI V2 Benchmark

May 7, 2025 · View on GitHub

This document will introduce the PSI V2 Benchmark. It uses the PSI V2 version of the interface.

Building from source

git clone https://github.com/secretflow/psi.git
cd psi
bazel build //... -c opt

If the building is successful, you will find an executable file named main in the 'bazel-bin/psi' directory. We will use ./main file, combined with config file to run different PSI protocols. Such as:

./main --config configs/sender.config
./main --config configs/receiver.config

Generate Data

In order to measure the performance of different PSI protocols under different data scales, we need to generate dummy data through generate_psi.py

python3 generate_psi.py 10000000 10000000

python3 generate_psi.py 100000000 100000000

Prepare config file

We use the config file to specify different PSI protocols and input data.

sender.config

{
  "psi_config": {
    "protocol_config": {
      "protocol": "PROTOCOL_KKRT",
      "role": "ROLE_SENDER",
      "broadcast_result": false
    },
    "input_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/datas/psi_1_1kw.csv"
    },
    "output_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/sender/sender_output.csv"
    },
    "keys": [
      "id",
    ],
    "debug_options": {
      "trace_path": "/home/zuoxian/v2psi/sender/sender.trace"
    },
    "disable_alignment": false,
    "advanced_join_type": "ADVANCED_JOIN_TYPE_UNSPECIFIED",
    "left_side": "ROLE_RECEIVER",
    "check_hash_digest": false,
    "recovery_config": {
      "enabled": false
    }
  },
  "link_config": {
    "parties": [
      {
        "id": "receiver",
        "host": "127.0.0.1:5300"
      },
      {
        "id": "sender",
        "host": "127.0.0.1:5400"
      }
    ]
  },
  "self_link_party": "sender"
}

receiver.config

{
  "psi_config": {
    "protocol_config": {
      "protocol": "PROTOCOL_KKRT",
      "role": "ROLE_RECEIVER",
      "broadcast_result": false
    },
    "input_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/datas/psi_2_1kw.csv"
    },
    "output_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/receiver/receiver_output.csv"
    },
    "keys": [
      "id",
    ],
    "debug_options": {
      "trace_path": "/home/zuoxian/v2psi/receiver/receiver.trace"
    },
    "disable_alignment": false,
    "advanced_join_type": "ADVANCED_JOIN_TYPE_UNSPECIFIED",
    "left_side": "ROLE_RECEIVER",
    "check_hash_digest": false,
    "recovery_config": {
      "enabled": false
    }
  },
  "link_config": {
    "parties": [
      {
        "id": "receiver",
        "host": "127.0.0.1:5300"
      },
      {
        "id": "sender",
        "host": "127.0.0.1:5400"
      }
    ]
  },
  "self_link_party": "receiver"
}

Run PSI with docker

In order to measure the PSI V2 benchmark under different machine configurations and network configurations, we use two dockers to act as sender and receiver respectively.

alice

docker run -d -it --name "alice" \
         --mount type=bind,source="$(pwd)",target=/home/admin/dev/ \
         -w /home/admin/dev \
         --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
         --cap-add=NET_ADMIN \
         --cpuset-cpus 0-31 \
         --privileged=true \
         --memory="64g" \
         --entrypoint="bash" \
         secretflow/ubuntu-base-ci:latest

docker start alice
docker exec -it alice bash

bob

docker run -d -it --name "bob" \
         --mount type=bind,source="$(pwd)",target=/home/admin/dev/ \
         -w /home/admin/dev \
         --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
         --cap-add=NET_ADMIN \
         --cpuset-cpus 32-63 \
         --privileged=true \
         --memory="64g" \
         --entrypoint="bash" \
         secretflow/ubuntu-base-ci:latest

docker start bob
docker exec -it bob bash

Limit bandwidth and latency

BAND_WIDTH=100
burst=128
DELAY=10
netem_limit=8000

tc qdisc add dev eth0 root handle 1: tbf rate ${BAND_WIDTH}mbit burst ${burst}kb latency 800ms
tc qdisc add dev eth0 parent 1:1 handle 10: netem delay ${DELAY}msec limit ${netem_limit}

Benchmark

Here we show the PSI V2 Benchmark measured as above.

The default time unit is seconds, m represents minutes, and h represents hours.

Machine Configuration	Algorithm parameters	Protocol	Network Configuration	10 million~10 million	100 million to 100 million	1 billion to 1 billion
32C64G	receiver='alice', protocol='ECDH_PSI_2PC', curve_type='CURVE_FOURQ', precheck_input=False, sort=False, broadcast_result=False,	ECDH-PSI-2PC （FourQ）	LAN	42	459	1.45 h
		ECDH-PSI-2PC （FourQ）	100Mbps/10ms	63	559	1.72 h
	receiver='alice', protocol='ECDH_PSI_2PC', curve_type='CURVE_25519', precheck_input=False, sort=False, broadcast_result=False,	ECDH-PSI-2PC （CURVE_25519）	LAN	67	697	2.09 h
		ECDH-PSI-2PC （CURVE_25519）	100Mbps/10ms	69	669	2.08 h
	receiver='alice', protocol='KKRT_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	KKRT_PSI_2PC （BUCKET SIZE IS ONE MILLION）	LAN	38	357	1.13 h
		KKRT_PSI_2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	125	1258	3.53 h
	receiver='alice', protocol='RR22_FAST_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	RR22_FAST_PSI _2PC （BUCKET SIZE IS ONE MILLION）	LAN	17	225	0.62 h
		RR22_FAST_PSI _2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	44	452	1.43 h
	receiver='alice', protocol='RR22_LOWCOMM_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	RR22_LOWCOMM _PSI_2PC （BUCKET SIZE IS ONE MILLION）	LAN	21	223	0.75 h
		RR22_LOWCOMM _PSI_2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	38	388	1.21 h
16C32G	receiver='alice', protocol='ECDH_PSI_2PC', curve_type='CURVE_FOURQ', precheck_input=False, sort=False, broadcast_result=False,	ECDH-PSI-2PC （FourQ）	LAN	46	497	1.66 h
		ECDH-PSI-2PC （FourQ）	100Mbps/10ms	62	574	2.00 h
	receiver='alice', protocol='ECDH_PSI_2PC', curve_type='CURVE_25519', precheck_input=False, sort=False, broadcast_result=False,	ECDH-PSI-2PC （CURVE_25519）	LAN	69	701	2.23 h
		ECDH-PSI-2PC （CURVE_25519）	100Mbps/10ms	74	744	2.49 h
	receiver='alice', protocol='KKRT_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	KKRT_PSI_2PC （BUCKET SIZE IS ONE MILLION）	LAN	37	352	1.18 h
		KKRT_PSI_2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	124	1228	3.69 h
	receiver='alice', protocol='RR22_FAST_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	RR22_FAST_PSI _2PC （BUCKET SIZE IS ONE MILLION）	LAN	17	178	0.72 h
		RR22_FAST_PSI _2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	43	452	1.70 h
	receiver='alice', protocol='RR22_LOWCOMM_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	RR22_LOWCOMM _PSI_2PC （BUCKET SIZE IS ONE MILLION）	LAN	23	216	0.92 h
		RR22_LOWCOMM _PSI_2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	40	384	1.52 h
8C16G	receiver='alice', protocol='ECDH_PSI_2PC', curve_type='CURVE_FOURQ', precheck_input=False, sort=False, broadcast_result=False,	ECDH-PSI-2PC （FourQ）	LAN	66	669	2.6 h
		ECDH-PSI-2PC （FourQ）	100Mbps/10ms	66	666	2.6 h
	receiver='alice', protocol='ECDH_PSI_2PC', curve_type='CURVE_25519', precheck_input=False, sort=False, broadcast_result=False,	ECDH-PSI-2PC （CURVE_25519）	LAN	114	1225	3.8 h
		ECDH-PSI-2PC （CURVE_25519）	100Mbps/10ms	112	1124	3.8 h
	receiver='alice', protocol='KKRT_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	KKRT_PSI_2PC （BUCKET SIZE IS ONE MILLION）	LAN	37	351	1.6 h
		KKRT_PSI_2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	124	1211	4.0 h
	receiver='alice', protocol='RR22_FAST_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	RR22_FAST_PSI _2PC （BUCKET SIZE IS ONE MILLION）	LAN	19	166	1.08 h
		RR22_FAST_PSI _2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	44	440	1.91 h
	receiver='alice', protocol='RR22_LOWCOMM_PSI_2PC', precheck_input=False, sort=False, broadcast_result=False,	RR22_LOWCOMM _PSI_2PC （BUCKET SIZE IS ONE MILLION）	LAN	22	204	1.17 h
		RR22_LOWCOMM _PSI_2PC （BUCKET SIZE IS ONE MILLION）	100Mbps/10ms	37	374	1.72 h