Transferable Tactile Transfomers (T3) and the Foundation Tactile (FoTa) dataset

June 21, 2024 ยท View on GitHub

img

Paper Repo Colab Huggingface

[Project Website]

Jialiang (Alan) Zhao, Yuxiang Ma, Lirui Wang, and Edward H. Adelson

MIT CSAIL

Overview

We present T3, a heterogeneous tactile representation learning framework based on transformers, and FoTa, a large tactile dataset that contains over 3 million tactile images collected from 13 sensors and 11 tasks. T3 extracts the common representation that is sharable between different camera-based tactile sensors and downstream tasks.

Installation

git clone https://github.com/alanzjl/t3
cd t3
pip install -e .

Get started

The best way to get started with using T3 or FoTa is to checkout our Colab for step-and-step instructions on how to manipulate data and run T3. More details about file structure of FoTa can be found on Huggingface.

Citation

@article{zhao2024transferable,
      title={Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks}, 
      author={Jialiang Zhao and Yuxiang Ma and Lirui Wang and Edward H. Adelson},
      year={2024},
      eprint={2406.13640},
      archivePrefix={arXiv},
}

MIT License.