lfq_acc.md
August 25, 2026 ยท View on GitHub
iters 200 enable_alg_ext is turned on Qwen3-8B
lfq causes large drop, maybe some bugs in our implementation
| Model | ARC-c | ARC-e | BoolQ | Hella | LAMB | MMLU | OBQA | PIQA | Truth. | Wino | GSM8K | if-eval stirct | mmlu-pro | AVG. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LFQ W2G64 | 0.3046 | 0.5396 | 0.6783 | 0.3856 | 0.2340 | 0.2822 | 0.2000 | 0.6436 | 0.3011 | 0.5564 | 0.3419 | 0.2206+0.1183 | 0.2334 | |
| ours W2G64 | 0.3942 | 0.7412 | 0.7841 | 0.4282 | 0.5242 | 0.5498 | 0.2540 | 0.7051 | 0.3097 | 0.6464 | 0.5239 | 0.2542+0.1201 | 0.2745 | |
| LFQ w3g128 | 0.4949 | 0.7605 | 0.8061 | 0.5276 | 0.4883 | 0.6558 | 0.2860 | 0.7530 | 0.3452 | 0.6717 | 0.8241 | 0.3777+0.2366 | 0.4076 | |
| oursw3g128 | 0.5213 | 0.8093 | 0.8667 | 0.5319 | 0.6445 | 0.6949 | 0.3140 | 0.7546 | 0.3623 | 0.7088 | 0.8597 | 0.3333+0.1904 | 0.4269 | |