MODELS.md
October 30, 2021 ยท View on GitHub
EPIC-Kitchens-100 Test/challenge submission
Any of the models can be trained/tested on train+val/test by changing the
dataset@dataset_train and dataset@dataset_eval fields in the configs.
Here we provide the configs that were used for the challenge submission.
| Backbone | Head | Train data | Config | Model |
|---|---|---|---|---|
| TSN (RGB) | RULSTM | train | expts/05_ek100_rustm_test_testonly.txt | link |
| TSN (RGB) | AVT-h | train | expts/02_ek100_avt_tsn_test_testonly.txt | link |
| TSN (RGB) | AVT-h | train + val | expts/02_ek100_avt_tsn_test_trainval.txt | link |
| irCSN-152 (IG65M) | AVT-h | train | expts/04_ek100_avt_ig65m_test_testonly.txt | link |
| irCSN-152 (IG65M) | AVT-h | train + val | expts/04_ek100_avt_ig65m_test_trainval.txt | link |
| AVT-b (RGB) | AVT-h | train | expts/01_ek100_avt_test_testonly.txt | link |
| AVT-b (RGB) | AVT-h | train + val | expts/01_ek100_avt_test_trainval.txt | link |
| TSN (Flow) | AVT-h | train | expts/06_ek100_avt_tsnflow_test_testonly.txt | link |
| TSN (Flow) | AVT-h | train + val | expts/06_ek100_avt_tsnflow_test_trainval.txt | link |
| TSN (Obj) | AVT-h | train + val | expts/03_ek100_avt_tsn_obj_test_trainval.txt | link |
| AVT-b (RGB, longer) | AVT-h | train | expts/07_ek100_avt_longer_test_testonly.txt | link |
| AVT-b (RGB, longer) | AVT-h | train + val | expts/07_ek100_avt_longer_test_trainval.txt | link |
The predictions from all the above models were late fused and submitted for evaluation using the following script:
from notebooks.utils import *
CFG_FILES = [
# RULSTM
('expts/05_ek100_rustm_test_testonly.txt', 0),
# TSN + AVT-h (train and train+val models)
('expts/02_ek100_avt_tsn_test_testonly.txt', 0),
('expts/02_ek100_avt_tsn_test_trainval.txt', 0),
# irCSN152/IG65M + AVT-h
('expts/04_ek100_avt_ig65m_test_testonly.txt', 0),
('expts/04_ek100_avt_ig65m_test_trainval.txt', 0),
# AVT
('expts/01_ek100_avt_test_testonly.txt', 0),
('expts/01_ek100_avt_test_trainval.txt', 0),
# Flow, obj AVT
('expts/06_ek100_avt_tsnflow_test_testonly.txt', 0),
('expts/06_ek100_avt_tsnflow_test_trainval.txt', 0),
('expts/03_ek100_avt_tsn_obj_test_trainval.txt', 0),
# Longer AVT
('expts/07_ek100_avt_longer_test_testonly.txt', 0),
('expts/07_ek100_avt_longer_test_trainval.txt', 0),
]
WTS = [1.0, # RULSTM
# TSN + AVT-h
1.0, 1.0,
# irCSN152/IG65M + AVT-h
1.0, 1.0,
# AVT
0.5, 0.5,
# Flow, obj AVT
0.5, 0.5, 0.5,
# Longer AVT
1.5, 1.5]
SLS = [2, 4, 4]
package_results_for_submission_ek100(CFG_FILES, WTS, SLS)
It should obtain 16.74 on the challenge leaderboard. We also provide our final submission file here.
EPIC-Kitchens-55
| Backbone | Head | Top-1 | Top-5 | Config (for top-1/5) | Model (for top-1/5) | AR5 | Config (for AR5) | Model (for AR5) |
|---|---|---|---|---|---|---|---|---|
| TSN (RGB) | AVT-h | 13.1 | 28.1 | expts/08_ek55_avt_tsn.txt | link | 13.5 | expts/08_ek55_avt_tsn_forAR.txt | link |
| AVT-b | AVT-h | 12.5 | 30.1 | expts/09_ek55_avt.txt | link | 13.6 | expts/09_ek55_avt_forAR.txt | link |
| irCSN-152 (IG65M) | AVT-h | 14.4 | 31.7 | expts/10_ek55_avt_ig65m.txt | link | 13.2 | expts/10_ek55_avt_ig65m_forAR.txt | link |
Our final test submission was generated by late-fusing AVT model with predictions from prior work, and is available here.
EGTEA Gaze+
| Backbone | Head | Top-1 (Act) | Class-mean (Act) | Config | Model |
|---|---|---|---|---|---|
| TSN (RGB) | AVT-h | 39.8 | 28.3 | expts/11_egtea_avt_tsn.txt | link |
| AVT-b | AVT-h | 43.0 | 35.2 | expts/12_egtea_avt.txt | link |
50-Salads
| Backbone | Head | Top-1 (Act) | Config | Model |
|---|---|---|---|---|
| AVT-b | AVT-h | 48.0 | expts/13_50s_avt.txt | fold 1 fold 2 fold 3 fold 4 fold 5 |