The repository of GM-AE is an implementation of the paper 'Parameterized Gompertz-guided Morphological AutoEncoder for Predicting Pulmonary Nodule Growth', accepted by TMI. Nodules used for GM-AE is listed in 'data/original coordinates/dataset_part_2301.csv'

Architecture of morphological autoencoder equipped with shape-aware (DS) and texture-aware (DN ) decoders.

ST-Mixer

Introduction

This repository is an implementation of the paper 'Siamese Encoder-based Spatial-Temporal Mixer for Growth Trend Prediction of Lung Nodules on CT Scans', accepted by MICCAI2022, and is built upon DeiT, thank them very much!

Schematic of our proposed method, including a siamese encoder, a spatial-temporal mixer (STM) and a two-layer H-loss.

Usage

Training:

Run

python main_ddp.py --resume --gpu 1,2 --batch-size 16 --epochs 60 --distributed

Inference:

Run

python inference.py

Datasets

The pipeline of organizing the temporal CT dataset.

The pipeline of organizing the temporal CT dataset (NLSTt), including CT scan registration, ROI pairing, and class annotation. The letter d denotes the diameter of lung nodules, and c indicates their corresponding texture types, i.e., solid, part-solid (PS), ground-glass nodule (GGN).

The paired nodules from NLST is listed in './data/registration coordinates/data.csv' (NLSTt), and please download raw CT scans from NLST

Statistics of benchmark splits of the NLSTt dataset and in-house dataset.

Types	Train set	Validation set	Test set	In-house set
GGN	2,683	564	658	129
Solid	4,121	900	892	352
PS	148	26	37	6
Total	6,952	1,490	1,587	487

Note

Unfortunately, we saved and used coordinates of nouldes after the regisration in the Paper, which is inconvenient to sharing coordinates. So we will release the original coordinates step by step in 'data/original coordinates'.

Results

AUC (in %) of different mixers and encoders on the test and in-house sets.

Encoder	Mixer	Test Set AUC@H1	Test Set AUC@H2	In-house set AUC@H1	In-house set AUC@H2
CNN	Concat	80.8	75.3	67.2	67.2
CNN	LSTM	81.8	75.0	64.0	71.0
CNN	STM (Ours)	83.0	76.3	73.5	71.6
ViT	Concat	82.6	75.2	64.2	64.1
ViT	LSTM	82.6	76.3	67.1	74.7
ViT	STM (Ours)	83.6	77.5	72.8	78.5

ACC of nodule types of the test set on different methods.

Method	ACC-GGN	ACC-Solid	ACC-PS
CNN+STM	90.9	88.2	56.8
ViT+STM	92.4	91.6	59.5

ACC of nodule types of in-house set on different methods.

Method	ACC-GGN	ACC-Solid	ACC-PS
CNN+STM	87.6	91.2	58.1
ViT+STM	93.8	90.6	60.5
Clinician A	85.3	93.2	60.5
Clinician B	86.0	94.0	62.8

Examples of predicting the growth trend by our model.

Examples of predicting the growth trend by our model and clinicians A and B. The first row is the ground-truth of the evolution classes, and the predicted results in red color are the incorrect predictions. The symbols →, ↑, and ↓ denote the classesstability, dilatation and shrinkage, respectively.

License

This repository is released under the Apache 2.0 license as found in the LICENSE file.

Citation

Please cite ST-Mixer in your publications if it helps your research.

@article{fang2022siamese,
  title={Siamese Encoder-based Spatial-Temporal Mixer for Growth Trend Prediction of Lung Nodules on CT Scans},
  author={Fang, Jiansheng and Wang, Jingwen and Li, Anwei and Yan, Yuguang and Hou, Yonghe and Song, Chao and Liu, Hongbo and Liu, Jiang},
  journal={arXiv preprint arXiv:2206.03049},
  year={2022}
}