We benchmark with following methods:

Method	Graph based	Spatial aware	Cross platform
SLAT	yes	yes	yes
PASTE	no	yes	no
STAGATE	yes	yes	no
Seurat	no	no	yes
Harmony	no	no	yes

Metric

Due to we can not know ground truth between real spatial datasets, we newly design CRI (Celltype and Region matching Index) metric to measure the performance of spatial alignment. CRI checks how much alignment method recover corresponding celltype and histology region simultaneously.

$$CRI= \frac{1}{M} \sum_{v_i,v_j \in M} I(i,j),\\ f(x)=\left\lbrace \begin{aligned} 1 &,\ c_1^{i}=c_2^{j} \ \mathbf{and} \ r_1^{i}=r_2^{j} \\ 0 &,\ otherwise, \\ \end{aligned} \right.$$

We also use Euclidean distance to measure the performance of spatial alignment:

$$Euclidean\ distance = \frac{1}{M} \sum_{v_i,v_j \in M} ||\mathbf{s_i} - \mathbf{s_j}||_F^2$$

Datasets

Note Dataset download links are available at here

We do benchmark on following datasets:

Index	Paper	Species	Tissue	Technology	Resolution	Cells/Spots	Genes	Download
1	Kristen et al.	Human	Brain(dorsolateral prefrontal cortex, DLPFC)	10x Visium	50μm	~3500	>20,000	website
2	Jeffrey et al.	Mouse	Brain(hypothalamic preoptic)	MERFISH	subcellular	~6,500	151	website
3	Chen et al.	Mouse	Whole embryo	Stereo-seq	0.2μm	5000-100,000	>20,000	website

Run benchmark pipeline

Note You need install extra dependencies following env/README.md.

To repeat our benchmark, just run:

snakemake --profile profiles/local -p