Collecting your own measurements:

May 15, 2026 · View on GitHub

This document serves as a quick overview of the quality of various quantization methods, with a focus on INT8. We capture the latents per step, and measure how much they diverge from the BF16 baseline.

General Model Quality

The general takeaway is that in terms of tested quantization methods the ranking is:

GGUF Q8 > INT8 ConvRot > MXFP8 > FP8 >= INT8 Row > INT8 Tensorwise

Every INT8 ConvRot and INT8 Row checkpoint was created from BF16 via on the fly quantization, unless stated otherwise. INT8 ConvRot is row-wise INT8 with parameters and activations rotated before quantization via ConvRot. INT8 Row is just regular row wise INT8.

Anima

100 samples per column.

Metric	INT8 ConvRot	INT8 Row	INT8 Row Bedovyy	INT8 Tensor Silver	FP8	GGUF_Q8	INT8 QuaRot
MSE ↓	`0.00746 ±0.00103` ★	`0.01467 ±0.00167`	`0.01438 ±0.00213`	`0.04069 ±0.00331`	`0.01756 ±0.00175`	`0.01364 ±0.00155`	`0.00952 ±0.00164`
MAE ↓	`0.03456 ±0.00256` ★	`0.05743 ±0.00350`	`0.05508 ±0.00437`	`0.10956 ±0.00503`	`0.06204 ±0.00323`	`0.04882 ±0.00332`	`0.03733 ±0.00304`
Max err ↓	`1.03295 ±0.04107` ★	`1.27721 ±0.03917`	`1.24410 ±0.04644`	`1.72119 ±0.04671`	`1.33251 ±0.03424`	`1.21710 ±0.03939`	`1.09261 ±0.04083`
Rel-RMSE ↓	`0.09032 ±0.00626` ★	`0.13396 ±0.00720`	`0.13084 ±0.00920`	`0.23802 ±0.01011`	`0.14523 ±0.00679`	`0.12124 ±0.00714`	`0.09632 ±0.00664`
SNR dB ↑	`24.05 ±0.53` ★	`19.68 ±0.39`	`20.24 ±0.52`	`14.48 ±0.36`	`19.66 ±0.35`	`21.98 ±0.46`	`23.98 ±0.45`
Cos-sim ↑	`0.992165 ±0.001113` ★	`0.984617 ±0.001780`	`0.984765 ±0.002368`	`0.957751 ±0.003461`	`0.981587 ±0.001878`	`0.985553 ±0.001704`	`0.990093 ±0.001646`
Var ratio →1	`1.00173 ±0.00081`	`1.00005 ±0.00145` ★	`0.99824 ±0.00155`	`0.99655 ±0.00221`	`0.99871 ±0.00107`	`1.00965 ±0.00085`	`1.00392 ±0.00087`
Outlier% ↓	`0.00008 ±0.00002` ★	`0.00022 ±0.00004`	`0.00022 ±0.00005`	`0.00103 ±0.00015`	`0.00027 ±0.00006`	`0.00019 ±0.00003`	`0.00019 ±0.00008`
Ch-MSE max ↓	`0.01443 ±0.00204` ★	`0.02867 ±0.00359`	`0.02837 ±0.00452`	`0.08719 ±0.00820`	`0.03471 ±0.00401`	`0.02732 ±0.00333`	`0.01964 ±0.00419`
Ch-MSE std ↓	`0.00346 ±0.00048` ★	`0.00684 ±0.00083`	`0.00670 ±0.00102`	`0.02075 ±0.00198`	`0.00816 ±0.00091`	`0.00648 ±0.00077`	`0.00468 ±0.00098`
ΔMSE/step ↓	`0.000757 ±0.000111` ★	`0.001306 ±0.000160`	`0.001300 ±0.000199`	`0.003241 ±0.000283`	`0.001577 ±0.000154`	`0.001251 ±0.000142`	`0.000903 ±0.000139`
ΔCos/step ↑	`-0.0007054 ±0.0001211` ★	`-0.0010668 ±0.0001787`	`-0.0012205 ±0.0002166`	`-0.0028863 ±0.0003109`	`-0.0015127 ±0.0001643`	`-0.0012179 ±0.0001540`	`-0.0008402 ±0.0001416`

Z Image Turbo

64 samples per column

(Different prompt, seeds and resolution from the other Z Image Turbo test. This one is just here to get more samples against MXFP8 checkpoints, which are hard to find in the wild.)

Metric	GGUF_Q8	I8ConvRot	I8Row	MXFP8
MSE ↓	`0.03616 ±0.00313` ★	`0.04834 ±0.00355`	`0.14951 ±0.00857`	`0.11037 ±0.00473`
MAE ↓	`0.08745 ±0.00362` ★	`0.10531 ±0.00384`	`0.21607 ±0.00612`	`0.17953 ±0.00387`
Max err ↓	`3.34811 ±0.09199` ★	`3.61884 ±0.09058`	`4.52879 ±0.07883`	`4.33244 ±0.07534`
Rel-RMSE ↓	`0.16740 ±0.00628` ★	`0.19634 ±0.00660`	`0.35659 ±0.00968`	`0.30729 ±0.00645`
SNR dB ↑	`16.42 ±0.29` ★	`14.86 ±0.26`	`9.27 ±0.23`	`10.59 ±0.18`
Cos-sim ↑	`0.978215 ±0.001696` ★	`0.971225 ±0.001920`	`0.916394 ±0.004070`	`0.935860 ±0.002428`
Var ratio →1	`1.00006 ±0.00050` ★	`1.00338 ±0.00045`	`0.97402 ±0.00155`	`0.99629 ±0.00101`
Outlier% ↓	`0.00054 ±0.00011` ★	`0.00084 ±0.00013`	`0.00418 ±0.00049`	`0.00251 ±0.00024`
Ch-MSE max ↓	`0.06081 ±0.00660` ★	`0.08189 ±0.00736`	`0.28000 ±0.02181`	`0.19093 ±0.01077`
Ch-MSE std ↓	`0.01206 ±0.00163` ★	`0.01657 ±0.00178`	`0.06623 ±0.00638`	`0.04204 ±0.00300`
ΔMSE/step ↓	`0.006970 ±0.000753` ★	`0.008928 ±0.000936`	`0.016678 ±0.001880`	`0.015876 ±0.001259`
ΔCos/step ↑	`-0.0038860 ±0.0004616` ★	`-0.0049137 ±0.0005605`	`-0.0090652 ±0.0010369`	`-0.0086360 ±0.0007459`

Flux2 Klein 9B Base

32 samples per column

Metric	INT8 ConvRot	INT8 Row	INT8 Row ModelOpt	GGUF Q8 0	FP8_Official
MSE ↓	`0.02204 ±0.00475` ★	`0.06017 ±0.01167`	`0.04246 ±0.00481`	`0.03024 ±0.00811`	`0.04142 ±0.00540`
MAE ↓	`0.05193 ±0.00639` ★	`0.10543 ±0.01075`	`0.08763 ±0.00509`	`0.05821 ±0.01005`	`0.08112 ±0.00684`
Max err ↓	`1.57007 ±0.12075` ★	`2.13746 ±0.09461`	`1.94028 ±0.08472`	`1.62537 ±0.17338`	`1.99069 ±0.11125`
Rel-RMSE ↓	`0.11172 ±0.01310` ★	`0.21613 ±0.01848`	`0.18317 ±0.01103`	`0.12411 ±0.01924`	`0.16800 ±0.01327`
SNR dB ↑	`23.10 ±0.81`	`15.79 ±0.59`	`17.03 ±0.50`	`23.52 ±1.11` ★	`19.12 ±0.69`
Cos-sim ↑	`0.987098 ±0.002851` ★	`0.961752 ±0.007710`	`0.973305 ±0.003089`	`0.981972 ±0.005046`	`0.975436 ±0.003389`
Var ratio →1	`1.00008 ±0.00121` ★	`1.00311 ±0.00263`	`1.00190 ±0.00224`	`1.00229 ±0.00136`	`1.00226 ±0.00181`
Outlier% ↓	`0.00055 ±0.00017` ★	`0.00220 ±0.00071`	`0.00121 ±0.00027`	`0.00104 ±0.00036`	`0.00118 ±0.00021`
Ch-MSE max ↓	`0.04205 ±0.00912` ★	`0.10400 ±0.01834`	`0.07711 ±0.00852`	`0.05674 ±0.01541`	`0.07300 ±0.00954`
Ch-MSE std ↓	`0.00632 ±0.00136` ★	`0.01511 ±0.00252`	`0.01146 ±0.00125`	`0.00832 ±0.00209`	`0.01073 ±0.00139`
ΔMSE/step ↓	`0.003857 ±0.000759` ★	`0.008846 ±0.001371`	`0.006936 ±0.000717`	`0.004814 ±0.001172`	`0.006657 ±0.000822`
ΔCos/step ↑	`-0.0017513 ±0.0004183` ★	`-0.0034872 ±0.0008234`	`-0.0026765 ±0.0005044`	`-0.0022502 ±0.0006240`	`-0.0030834 ±0.0005014`

Z Image Turbo

32 samples per column

Metric	INT8 ConvRot	INT8 Row	GGUF Q8	FP8
MSE ↓	`0.04326 ±0.00622`	`0.10273 ±0.00973`	`0.02627 ±0.00370` ★	`0.09472 ±0.00712`
MAE ↓	`0.09578 ±0.00821`	`0.17967 ±0.00961`	`0.07105 ±0.00531` ★	`0.18951 ±0.00700`
Max err ↓	`2.24582 ±0.10280`	`2.99726 ±0.08630`	`2.05538 ±0.09594` ★	`2.87146 ±0.06808`
Rel-RMSE ↓	`0.16181 ±0.01205`	`0.27953 ±0.01342`	`0.12857 ±0.00877` ★	`0.28443 ±0.01002`
SNR dB ↑	`17.97 ±0.63`	`11.91 ±0.41`	`19.79 ±0.57` ★	`11.17 ±0.29`
Cos-sim ↑	`0.972530 ±0.003675`	`0.935604 ±0.005455`	`0.982917 ±0.002238` ★	`0.933439 ±0.004578`
Var ratio →1	`1.00117 ±0.00131` ★	`0.99202 ±0.00301`	`1.00208 ±0.00098`	`0.94623 ±0.00320`
Outlier% ↓	`0.00053 ±0.00012`	`0.00172 ±0.00033`	`0.00032 ±0.00010` ★	`0.00083 ±0.00014`
Ch-MSE max ↓	`0.07861 ±0.01218`	`0.19714 ±0.02157`	`0.04706 ±0.00714` ★	`0.17641 ±0.01822`
Ch-MSE std ↓	`0.01621 ±0.00276`	`0.04345 ±0.00546`	`0.00956 ±0.00172` ★	`0.03825 ±0.00444`
ΔMSE/step ↓	`0.008738 ±0.001514`	`0.015233 ±0.002232`	`0.005719 ±0.001069` ★	`0.012003 ±0.001665`
ΔCos/step ↑	`-0.0050629 ±0.0009302`	`-0.0086229 ±0.0012933`	`-0.0033847 ±0.0006569` ★	`-0.0071850 ±0.0011542`

Chroma

32 samples per column

Metric	INT8 ConvRot	INT8 Row	GGUF Q8	FP8 Mixed
MSE ↓	`0.01021 ±0.00360`	`0.02799 ±0.00564`	`0.00555 ±0.00138` ★	`0.02030 ±0.00274`
MAE ↓	`0.03999 ±0.00420`	`0.07773 ±0.00677`	`0.02807 ±0.00324` ★	`0.06772 ±0.00447`
Max err ↓	`1.46539 ±0.20187`	`2.22444 ±0.21730`	`1.35671 ±0.22226` ★	`2.04296 ±0.19891`
Rel-RMSE ↓	`0.09169 ±0.01286`	`0.16790 ±0.01750`	`0.06770 ±0.00925` ★	`0.14417 ±0.01110`
SNR dB ↑	`23.54 ±0.99`	`17.33 ±0.89`	`26.31 ±1.11` ★	`18.79 ±0.76`
Cos-sim ↑	`0.990995 ±0.002884`	`0.976308 ±0.004694`	`0.995231 ±0.001197` ★	`0.982911 ±0.002356`
Var ratio →1	`0.99150 ±0.00407`	`1.03855 ±0.00951`	`1.00278 ±0.00227` ★	`1.00961 ±0.00768`
Outlier% ↓	`0.00041 ±0.00029`	`0.00129 ±0.00049`	`0.00015 ±0.00007` ★	`0.00059 ±0.00018`
Ch-MSE max ↓	`0.02014 ±0.00781`	`0.05817 ±0.01315`	`0.01074 ±0.00273` ★	`0.03952 ±0.00520`
Ch-MSE std ↓	`0.00431 ±0.00173`	`0.01372 ±0.00325`	`0.00243 ±0.00070` ★	`0.00934 ±0.00135`
ΔMSE/step ↓	`0.000707 ±0.000291`	`0.001820 ±0.000582`	`0.000410 ±0.000138` ★	`0.001372 ±0.000381`
ΔCos/step ↑	`-0.0005174 ±0.0003418`	`-0.0010664 ±0.0007771`	`-0.0002943 ±0.0001454` ★	`-0.0009653 ±0.0004437`

Qwen Image 2512

16 samples per column.

Metric	FP8	GGUF Q4 K M	GGUF Q8	I8 Conv	I8 Row	Nunchaku BestQuality
MSE ↓	`0.01643 ±0.00334`	`0.02188 ±0.00320`	`0.01062 ±0.00377`	`0.00894 ±0.00256` ★	`0.01305 ±0.00436`	`0.02146 ±0.00354`
MAE ↓	`0.07556 ±0.00707`	`0.07740 ±0.00661`	`0.04068 ±0.00892`	`0.04043 ±0.00619` ★	`0.05007 ±0.00917`	`0.08532 ±0.00742`
Max err ↓	`0.93735 ±0.06070`	`1.05423 ±0.05437`	`0.65768 ±0.09201` ★	`0.73333 ±0.08073`	`0.75177 ±0.07628`	`0.96512 ±0.04607`
Rel-RMSE ↓	`0.22316 ±0.02186`	`0.25253 ±0.02143`	`0.13382 ±0.02853` ★	`0.13795 ±0.02225`	`0.16354 ±0.02883`	`0.24947 ±0.02144`
SNR dB ↑	`14.08 ±0.75`	`13.78 ±0.84`	`22.44 ±1.67` ★	`20.34 ±1.31`	`18.70 ±1.27`	`13.54 ±0.72`
Cos-sim ↑	`0.943337 ±0.010885`	`0.929011 ±0.010479`	`0.967114 ±0.011496`	`0.972459 ±0.007414` ★	`0.957911 ±0.013642`	`0.927933 ±0.011458`
Var ratio →1	`1.00262 ±0.00459`	`0.99685 ±0.00597`	`0.99789 ±0.00268` ★	`0.98840 ±0.00348`	`1.00248 ±0.00378`	`0.94775 ±0.00588`
Outlier% ↓	`0.00076 ±0.00029`	`0.00162 ±0.00044`	`0.00079 ±0.00040`	`0.00064 ±0.00029` ★	`0.00116 ±0.00051`	`0.00093 ±0.00046`
Ch-MSE max ↓	`0.02873 ±0.00637`	`0.04307 ±0.00754`	`0.02095 ±0.00768`	`0.01662 ±0.00475` ★	`0.02675 ±0.00918`	`0.03918 ±0.00712`
Ch-MSE std ↓	`0.00681 ±0.00167`	`0.01152 ±0.00214`	`0.00555 ±0.00216`	`0.00420 ±0.00124` ★	`0.00735 ±0.00270`	`0.01014 ±0.00205`
ΔMSE/step ↓	`0.001429 ±0.000380`	`0.002038 ±0.000518`	`0.001062 ±0.000426`	`0.000907 ±0.000322` ★	`0.001278 ±0.000448`	`0.002092 ±0.000484`
ΔCos/step ↑	`-0.0023754 ±0.0015020` ★	`-0.0055836 ±0.0019522`	`-0.0029254 ±0.0013186`	`-0.0024572 ±0.0010212`	`-0.0034970 ±0.0014186`	`-0.0057739 ±0.0019517`

HiDream O1

16 Samples per column.

FP8 Naive refers to using a BF16 checkpoint with the dtype set to FP8, which naively casts most weights to FP8.

Metric	FP8_Naive	FP8 Scaled	INT8 ConvRot	INT8 Row	MXFP8
MSE ↓	`0.02261 ±0.00697`	`0.00324 ±0.00098`	`0.00199 ±0.00058` ★	`0.05192 ±0.01084`	`0.00354 ±0.00070`
MAE ↓	`0.06901 ±0.01116`	`0.02499 ±0.00291`	`0.01877 ±0.00202` ★	`0.13052 ±0.01274`	`0.02768 ±0.00254`
Max err ↓	`0.86595 ±0.05077`	`0.53393 ±0.04962`	`0.45624 ±0.03571` ★	`1.15126 ±0.04459`	`0.56008 ±0.03832`
Rel-RMSE ↓	`0.23140 ±0.03353`	`0.08793 ±0.01196`	`0.06738 ±0.00849` ★	`0.40533 ±0.03865`	`0.09269 ±0.00912`
SNR dB ↑	`14.86 ±1.00`	`22.98 ±0.91`	`25.65 ±0.85` ★	`8.77 ±0.76`	`22.65 ±0.79`
Cos-sim ↑	`0.957479 ±0.013819`	`0.993943 ±0.001945`	`0.996338 ±0.001124` ★	`0.901425 ±0.020387`	`0.993764 ±0.001271`
Var ratio →1	`0.96638 ±0.00868`	`0.96287 ±0.00445`	`0.99691 ±0.00313`	`1.00254 ±0.02455` ★	`1.01115 ±0.00402`
Outlier% ↓	`0.00499 ±0.00257`	`0.00028 ±0.00015`	`0.00010 ±0.00008` ★	`0.01168 ±0.00462`	`0.00022 ±0.00008`
Ch-MSE max ↓	`0.02596 ±0.00826`	`0.00362 ±0.00109`	`0.00237 ±0.00068` ★	`0.05824 ±0.01247`	`0.00399 ±0.00077`
Ch-MSE std ↓	`0.00305 ±0.00116`	`0.00034 ±0.00011`	`0.00034 ±0.00012` ★	`0.00590 ±0.00154`	`0.00044 ±0.00009`
ΔMSE/step ↓	`0.002661 ±0.000879`	`0.000514 ±0.000244`	`0.000299 ±0.000185` ★	`0.005193 ±0.001433`	`0.000592 ±0.000248`
ΔCos/step ↑	`-0.0044397 ±0.0016674`	`-0.0008584 ±0.0004631`	`-0.0005061 ±0.0003415` ★	`-0.0064275 ±0.0033903`	`-0.0009670 ±0.0004783`

Anima on a 5060

16 samples per column.

Metric	INT8ConvRot	MXFP8
MSE ↓	`0.00576 ±0.00109` ★	`0.01461 ±0.00217`
MAE ↓	`0.03466 ±0.00317` ★	`0.06382 ±0.00463`
Max err ↓	`0.66684 ±0.06254` ★	`0.92180 ±0.05310`
Rel-RMSE ↓	`0.08546 ±0.00846` ★	`0.14716 ±0.01107`
SNR dB ↑	`24.22 ±0.73` ★	`18.90 ±0.58`
Cos-sim ↑	`0.991708 ±0.001573` ★	`0.979025 ±0.003469`
Var ratio →1	`1.00804 ±0.00188` ★	`1.01619 ±0.00334`
Outlier% ↓	`0.00003 ±0.00001` ★	`0.00015 ±0.00005`
Ch-MSE max ↓	`0.01101 ±0.00206` ★	`0.02518 ±0.00361`
Ch-MSE std ↓	`0.00259 ±0.00050` ★	`0.00598 ±0.00085`
ΔMSE/step ↓	`0.000954 ±0.000213` ★	`0.002096 ±0.000369`
ΔCos/step ↑	`-0.0010690 ±0.0003465` ★	`-0.0023829 ±0.0006197`

Lora

In this table, we compare the quality of our various lora approaches, against a standard bf16 lora loader baseline. The TLDR is that Pre-Lora is within marging of error of Dynamic Lora. Post-Lora is slightly worse. GGUF Q8 dequantizes to bf16 during inference to apply the lora math which is both slow and cheating. Nunchaku lora appears to be a little broken.

Interesting observation: These consistently score higher than their non-lora counterparts. I suspect it could be that there is a QAT like effect for applying loras trained with quantization to quantized models. Alternatively, maybe there is a reverse QAT like effect when using QLora on a BF16 model, lowering the quality, bringing it closer to quantized models.

Anima:

32 Samples per column.

Metric	INT8 ConvRot Pre-Lora	INT8 ConvRot Dynamic Lora	INT8 ConvRot Post-Lora	GGUF Q8_0 Lora	FP8 Lora
MSE ↓	`0.00073 ±0.00021` ★	`0.00090 ±0.00028`	`0.00186 ±0.00043`	`0.00158 ±0.00048`	`0.00641 ±0.00092`
MAE ↓	`0.01302 ±0.00091` ★	`0.01327 ±0.00113`	`0.01990 ±0.00175`	`0.01694 ±0.00196`	`0.04095 ±0.00311`
Max err ↓	`0.30456 ±0.04180`	`0.30054 ±0.04417` ★	`0.42361 ±0.04273`	`0.37600 ±0.04041`	`0.63406 ±0.04540`
Rel-RMSE ↓	`0.04963 ±0.00505` ★	`0.05100 ±0.00601`	`0.07606 ±0.00772`	`0.06606 ±0.00872`	`0.14956 ±0.01150`
SNR dB ↑	`27.39 ±0.59`	`27.52 ±0.65` ★	`24.32 ±0.66`	`26.01 ±0.83`	`18.25 ±0.64`
Cos-sim ↑	`0.997709 ±0.000687` ★	`0.997376 ±0.000778`	`0.994509 ±0.001267`	`0.995339 ±0.001409`	`0.981066 ±0.002814`
Var ratio →1	`1.00645 ±0.00144`	`0.99760 ±0.00114`	`1.00029 ±0.00176`	`1.00010 ±0.00113` ★	`0.98184 ±0.00368`
Outlier% ↓	`0.00002 ±0.00002`	`0.00001 ±0.00001` ★	`0.00005 ±0.00003`	`0.00004 ±0.00003`	`0.00023 ±0.00008`
Ch-MSE max ↓	`0.00141 ±0.00044` ★	`0.00194 ±0.00067`	`0.00400 ±0.00099`	`0.00314 ±0.00099`	`0.01286 ±0.00201`
Ch-MSE std ↓	`0.00035 ±0.00011` ★	`0.00047 ±0.00016`	`0.00098 ±0.00025`	`0.00079 ±0.00026`	`0.00326 ±0.00052`
ΔMSE/step ↓	`0.000077 ±0.000031` ★	`0.000099 ±0.000039`	`0.000194 ±0.000061`	`0.000167 ±0.000062`	`0.000599 ±0.000122`
ΔCos/step ↑	`-0.0001550 ±0.0001023` ★	`-0.0002021 ±0.0001127`	`-0.0004421 ±0.0001852`	`-0.0004007 ±0.0001874`	`-0.0014563 ±0.0004473`

Qwen Image 2512

16 Samples per column.

Metric	FP8	GGUF Q4 K M	GGUF Q8	INT8 ConvRot Post-Lora	INT8 ConvRot Pre-Lora	Nunchaku_BestQuality
MSE ↓	`0.01139 ±0.00146`	`0.00874 ±0.00147`	`0.00135 ±0.00058`	`0.00185 ±0.00050`	`0.00111 ±0.00032` ★	`0.04326 ±0.00328`
MAE ↓	`0.06940 ±0.00369`	`0.05205 ±0.00418`	`0.01490 ±0.00233` ★	`0.02129 ±0.00215`	`0.01637 ±0.00156`	`0.14596 ±0.00556`
Max err ↓	`0.83818 ±0.05885`	`0.68868 ±0.04720`	`0.37840 ±0.05948` ★	`0.45491 ±0.05199`	`0.38492 ±0.03914`	`1.08649 ±0.03813`
Rel-RMSE ↓	`0.18603 ±0.01147`	`0.14543 ±0.01242`	`0.04687 ±0.00796` ★	`0.06366 ±0.00756`	`0.05016 ±0.00546`	`0.36876 ±0.01457`
SNR dB ↑	`15.19 ±0.48`	`18.56 ±0.65`	`29.23 ±0.95` ★	`25.81 ±0.80`	`27.56 ±0.70`	`9.33 ±0.35`
Cos-sim ↑	`0.957885 ±0.005072`	`0.971353 ±0.004980`	`0.995827 ±0.001845`	`0.993908 ±0.001672`	`0.996241 ±0.001149` ★	`0.874391 ±0.010770`
Var ratio →1	`1.03367 ±0.00407`	`0.98059 ±0.00510`	`0.99394 ±0.00142`	`0.99124 ±0.00185`	`0.99651 ±0.00217` ★	`1.17955 ±0.01708`
Outlier% ↓	`0.00016 ±0.00005`	`0.00008 ±0.00003`	`0.00002 ±0.00001`	`0.00002 ±0.00001`	`0.00001 ±0.00000` ★	`0.00097 ±0.00018`
Ch-MSE max ↓	`0.02053 ±0.00283`	`0.01603 ±0.00271`	`0.00269 ±0.00108`	`0.00388 ±0.00111`	`0.00204 ±0.00053` ★	`0.08783 ±0.00643`
Ch-MSE std ↓	`0.00464 ±0.00075`	`0.00399 ±0.00073`	`0.00067 ±0.00029`	`0.00089 ±0.00025`	`0.00048 ±0.00013` ★	`0.02458 ±0.00197`
ΔMSE/step ↓	`0.000958 ±0.000252`	`0.001059 ±0.000261`	`0.000183 ±0.000087`	`0.000237 ±0.000087`	`0.000152 ±0.000071` ★	`0.003098 ±0.000772`
ΔCos/step ↑	`-0.0007560 ±0.0012897`	`-0.0025382 ±0.0009487`	`-0.0004476 ±0.0002795`	`-0.0005603 ±0.0003156`	`-0.0003486 ±0.0002816` ★	`-0.0054101 ±0.0030137`

Some loras require stochastic lora to work

Anima, 16 samples:

Metric	I8Dynamic	I8None	I8Pre	I8Stoch
MSE ↓	`0.01128 ±0.00309` ★	`0.23273 ±0.01757`	`0.01204 ±0.00277`	`0.01488 ±0.00385`
MAE ↓	`0.04268 ±0.00710` ★	`0.35066 ±0.01357`	`0.04434 ±0.00612`	`0.05195 ±0.00784`
Max err ↓	`0.77778 ±0.07449` ★	`1.83527 ±0.05833`	`0.83019 ±0.06939`	`0.90528 ±0.06125`
Rel-RMSE ↓	`0.11532 ±0.01719` ★	`0.79263 ±0.03690`	`0.12302 ±0.01520`	`0.13954 ±0.01769`
SNR dB ↑	`23.07 ±1.13` ★	`2.63 ±0.40`	`22.50 ±1.05`	`21.04 ±0.92`
Cos-sim ↑	`0.983092 ±0.004472` ★	`0.594366 ±0.035211`	`0.982433 ±0.003686`	`0.977338 ±0.005859`
Var ratio →1	`0.99949 ±0.00296` ★	`0.69981 ±0.03352`	`0.99129 ±0.00373`	`0.99126 ±0.00392`
Outlier% ↓	`0.00027 ±0.00009` ★	`0.02900 ±0.00621`	`0.00029 ±0.00009`	`0.00032 ±0.00009`
Ch-MSE max ↓	`0.01921 ±0.00513` ★	`0.50483 ±0.04664`	`0.02048 ±0.00468`	`0.02571 ±0.00644`
Ch-MSE std ↓	`0.00436 ±0.00115` ★	`0.13416 ±0.01344`	`0.00491 ±0.00116`	`0.00583 ±0.00143`
ΔMSE/step ↓	`0.001330 ±0.000360` ★	`0.015669 ±0.001669`	`0.001472 ±0.000345`	`0.001712 ±0.000422`
ΔCos/step ↑	`-0.0017781 ±0.0005309` ★	`-0.0205223 ±0.0046041`	`-0.0019693 ±0.0004810`	`-0.0023263 ±0.0006257`

Collecting your own measurements:

Use this custom node: https://github.com/BobJohnson24/ComfyUI-EvalSampler