This repository was moved to https://github.com/explosion/tokenizations, and now maintained by Explosion, spaCy's authors.

Robust and Fast tokenizations alignment library for Rust and Python

Demo: demo
Rust document: docs.rs
Blog post: How to calculate the alignment between BERT and spaCy tokens effectively and robustly