Jittor Implementation of CamoFormer

Jittor is a high-performance deep learning framework based on JIT compiling and meta-operators. The whole framework and meta-operators are compiled just-in-time. A powerful op compiler and tuner are integrated into Jittor. It allowed us to generate high-performance code with specialized for your model. Jittor also contains a wealth of high-performance model libraries, including: image recognition, detection, segmentation, generation, differentiable rendering, geometric learning, reinforcement learning, etc. The front-end language is Python. Module Design and Dynamic Graph Execution is used in the front-end, which is the most popular design for deeplearning framework interface. The back-end is implemented by high performance language, such as CUDA, C++.

This repository contains the source code, prediction results for 'CamoFormer: Masked Separable Attention for Camouflaged Object Detection'. The technical report could be found at arXiv. The whole benchmark results can be found at One Drive, Baidu Netdisk, or Google Drive.

Contributors: 2112529, yinbow

Figure 1: Overall architecture of our CamoFormer model. First, a pretrained Transformer-based backbone is utilized to extract multi-scale features of the input image. Then, the features from the last three stages are aggregated to generate the coarse prediction. Next, the progressive refinement decoder equipped with masked separable attention (MSA) is applied to gradually polish the prediction results. All the predictions generated by our CamoFormer are supervised by the ground truth (GT).

2. Get Start

0. Install

Create environment by python3.7 -m pip install jittor on Linux. As for MacOS or Windows users, using Docker docker run --name jittor -v $PATH_TO_PROJECT:/path -it jittor/jittor /bin/bash is easier and necessary. A simple way to debug and run the script is running a new command in the container through docker exec -it jittor /bin/bash and start the experiments. (More details refer to this installation tutorial)

1. Download Datasets and Checkpoints.

• Datasets:

By default, you can put datasets into the folder 'dataset'.

Baidu Netdisk, One Drive

• Checkpoints:

By default, you can put datasets into the folder 'checkpoint'.

CamoFormer: Baidu Netdisk, One Drive Backbone: Baidu Netdisk, One Drive

2. Run the model.

bash main.sh

3. Proposed CamoFormer

COD Benchmark Results:

The prediction of our CamoFprmer can be found in One Drive, Baidu Netdisk, or Google Drive. Here are quantitative performance comparison.

Figure 3: Comparison of our CamoFormer with the recent SOTA methods. ‘-R’: ResNet, ‘-C’: ConvNext, ‘-S’: Swin Transformer, ‘-P’: PVTv2. As can be seen, our CamoFormer-P performs much better than previous methods with either CNN- or Transformer-based models. ‘↑’: the higher the better, ‘↓’: the lower the better.

Acknowlegement

Thanks mczhuge providing a friendly codebase for binary segmentation tasks and Daniel Ji providing a good Jittor codebase for COD. And our code is built based on them. Thanks to 2112529 for his significant contributions and great effort in implementing the Jittor version of the code.

Reference

You may want to cite:

@article{yin2024camoformer,
  title={Camoformer: Masked separable attention for camouflaged object detection},
  author={Yin, Bowen and Zhang, Xuying and Fan, Deng-Ping and Jiao, Shaohui and Cheng, Ming-Ming and Van Gool, Luc and Hou, Qibin},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2024},
  publisher={IEEE}
}

License

Code in this repo is for non-commercial use only.