llama2.c-android

Port of Andrej Karpathy's llama2.c to Android. You can run it as raw binary or use it as shared library.

You can use the prebuild binaries in libs or compile on your own:

# or wherever your ndk-build script resides
cd jni && $ANDROID_HOME/ndk-bundle/ndk-build

run as binary

Get e.g. termux and install APK to run binaries.

wget https://karpathy.ai/llama2c/model.bin -P out
adb push libs/<your abi>/llama2 /storage/emulated/0/Android/data
adb push model.bin /storage/emulated/0/Android/data
adb push tokenizer.bin /storage/emulated/0/Android/data

In Termux:

cp /storage/emulated/0/Android/llama2 .
chmod +x llama2
./llama2 model.bin

run as shared lib

wip