Checkpoints

We provide links for you to download the pre-trained weights of some Uni-Perceiver models.

Model	Link	Params	Hidden size	Intermediate size	Num. of heads	Enc layers
Uni Perceiverbase	Downloadtorch Downloadds	124M	768	3072	12	12
Uni Perceiver MoEbase	Downloadds	167M	768	3072	12	12
Uni Perceiverlarge	Downloadds	354M	1024	4096	16	24
Uni Perceiver MoElarge	Downloadds	505M	1024	4096	16	24

• Models ^torch are pretrained in our current codebase, while models ^ds are pretrained in our previous codebase with DeepSpeed engine.

• Params in the table is the parameters required during model deployment for image-text tasks. Note that it may be different on other tasks.

• More pre-trained weights will be released.