SRMCLMERmain
April 20, 2024 · View on GitHub
Boosting Micro-expression Recognition via Self-expression Reconstruction and Memory Contrastive Learning
This project is the implementation for our paper “Boosting Micro-expression Recognition via Self-expression Reconstruction and Memory Contrastive Learning”. The codes need to run in the environment: Python 3.8.
Model framework
The proposed MER framework. It includes four components, e.g., 1) the ME preprocessing module estimates optical flow between onset and apex frame using TV-L1 for discriminative feature learning, 2) The self-expression reconstruction module includes an encoder-decoder structure that reconstructs input ME from patch-wise masked faces, 3) the prototype- based memory contrastive learning module includes a dynamically updated memory dictionary that stores class prototypes for contrastive learning, and 4) the classification head predicts the ME category.
Experimental Results
The following shows the results of performance comparison on a single dataset:
|
Method |
SMIC |
CASME II |
SAMM |
|||
|
ACC |
F1 |
ACC |
F1 |
ACC |
F1 |
|
|
LBP-TOP[10] |
0.4338 |
0.3421 |
0.3968 |
0.3589 |
0.3968 |
0.3589 |
|
LBP-SIP[6] |
0.4451 |
0.4492 |
0.4656 |
0.4480 |
- |
- |
|
MDMO[11] |
0.5897 |
0.5845 |
0.5169 |
0.4966 |
- |
- |
|
Bi-WOOF[12] |
0.6220 |
0.6200 |
0.5885 |
0.6100 |
- |
- |
|
Graph-TCN[73] |
- |
- |
0.7398 |
0.7246 |
0.7500 |
0.6985 |
|
LGCcon[18] |
- |
- |
0.6502 |
0.6400 |
0.4090 |
0.3400 |
|
AUGCN+AUFusion[19] |
- |
- |
0.7427 |
0.7047 |
0.7426 |
0.7045 |
|
DSSN[28] |
0.6341 |
0.6462 |
0.7078 |
0.7297 |
0.5735 |
0.4644 |
|
KFC[30] |
0.6585 |
0.6638 |
0.7276 |
0.7375 |
0.6324 |
0.5709 |
|
FeatRef[31] |
0.5790 |
- |
0.6285 |
- |
- |
- |
|
SLSTT[41] |
0.7371 |
0.7240 |
0.7581 |
0.7530 |
0.7239 |
0.6400 |
|
ExpMultNet[40] |
0.7999 |
0.7812 |
0.8150 |
0.8009 |
- |
- |
|
MER-Supcon[46] |
- |
- |
0.7358 |
0.7286 |
0.6765 |
0.6251 |
|
I3D+MOCO[48] |
0.7561 |
0.7492 |
0.7630 |
0.7366 |
0.6838 |
0.5436 |
|
SRMCL |
0.7898 |
0.7887 |
0.8320 |
0.8286 |
0.7463 |
0.6599 |
Data preparation:
Due to licensing restrictions, we are unable to directly disclose or use specific datasets. To ensure the compliance of our research and the accuracy of the data, we hereby provide the official link for obtaining the dataset, so that interested researchers can apply and acquire it in accordance with the official procedures.
Pretrained models:
The pre-training model for this method is provided here: '/best_model/pretrain.pt'
[Link](https://pan.baidu.com/s/101yj-1d6SloGiSShCrJXeg?pwd=omyx） Password:omyx
Testing:
Run the following codes to reproduce the recognition results provided in the paper:
（1）Validating model performance under the self-expression reconstruction task.
python Last_casmeII_SR.py
（2）Validating the performance of the full SRMCL model
python Last_casmeII_test.py