nano flash attention

April 3, 2026 · View on GitHub

a minimal implementation of Flash Attention

Running

Colab Demo

@misc{dao2022flashattentionfastmemoryefficientexact,
      title={FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness}, 
      author={Tri Dao and Daniel Y. Fu and Stefano Ermon and Atri Rudra and Christopher Ré},
      year={2022},
      eprint={2205.14135},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2205.14135}, 
}