llama.cpp/examples/speculative

February 15, 2025 ยท View on GitHub

Demonstration of speculative decoding and tree-based speculative decoding techniques

More info:

  • https://github.com/ggml-org/llama.cpp/pull/2926
  • https://github.com/ggml-org/llama.cpp/pull/3624
  • https://github.com/ggml-org/llama.cpp/pull/5625