Dataset Benchmark Report

April 16, 2026 · View on GitHub

CoBRA is validated against 76,144 lines drawn from 35 dataset files spanning 7 independent sources. These datasets are redistributed under their original licenses, which differ from CoBRA's Apache-2.0 license. See THIRD_PARTY_LICENSES for provenance and license details. Every expression is parsed, simplified, and spot-checked at runtime. The numbers below are enforced by automated test assertions in test/verify/test_dataset_benchmarks.cpp and verified on every CI run. OSES Fast is currently disabled (OOM on deeply nested expressions); its numbers are from the last successful run on master.

Overall: 75,023 / 75,126 parsed expressions simplified (99.86%).

MBA Classes

CoBRA classifies each input expression into one of four semantic classes and selects techniques accordingly:

ClassDescriptionTechniques
LinearWeighted sums of bitwise atoms (+, -, *const over &, |, ^, ~)Signature vector, CoB butterfly transform, pattern matching, ANF
SemilinearBitwise operations with constant masks (x & 0xFF, y | 0xF0)Constant lowering, structure recovery, term refinement, bit-partitioned reconstruction
PolynomialProducts of variables (x*y, x^2) possibly mixed with bitwise atomsCoefficient splitting, polynomial recovery, finite differences
MixedProducts of bitwise subexpressions ((x&y)*(x|y))Decomposition engine, ghost residual solving, template matching, lifting

Results by Dataset

MSiMBA Datasets

Source: Simplifier (GPL-3.0-only)

DatasetExpressionsParsedSimplifiedUnsupportedRate
univariate64.txt1,0001,0001,0000100%
multivariate64.txt1,0001,0001,0000100%
permutation64.txt1313130100%
msimba.txt1,0001,0001,0000100%
  • univariate64 / multivariate64: Polynomial expressions (x0*x0, x0*x1) that simplify to linear targets. All 2,000 pass full-width verification.
  • permutation64: High-degree polynomial expressions with ** (exponentiation). All 13 expressions parse and simplify correctly.
  • msimba: Semilinear expressions with bit-extraction patterns. All 1,000 simplify via bit-partitioned semilinear techniques.

SiMBA++ Datasets

All files in the simba/ directory were taken from the SiMBA++ repository (GPL-3.0-only). SiMBA++ aggregates datasets from several upstream projects; original provenance is noted per subsection.

Expression Templates (E-Series)

Originally from SiMBA (GPL-3.0-only, Denuvo Software Solutions). Redistributed via SiMBA++.

Each file contains 1,000 obfuscated linear MBA expressions plus a header comment. All 16,000 expressions across 16 files simplify at 100%.

DatasetVariablesExpressionsSimplifiedRate
e1_2vars.txt21,0001,000100%
e1_3vars.txt31,0001,000100%
e1_4vars.txt41,0001,000100%
e1_5vars.txt51,0001,000100%
e2_2vars.txt21,0001,000100%
e2_3vars.txt31,0001,000100%
e2_4vars.txt41,0001,000100%
e3_2vars.txt21,0001,000100%
e3_3vars.txt31,0001,000100%
e3_4vars.txt41,0001,000100%
e4_2vars.txt21,0001,000100%
e4_3vars.txt31,0001,000100%
e4_4vars.txt41,0001,000100%
e5_2vars.txt21,0001,000100%
e5_3vars.txt31,0001,000100%
e5_4vars.txt41,0001,000100%
E-Series Total16,00016,000100%

PLDI Research Datasets

Originally from the MBA-Solver repository (GPL-3.0-only), published at PLDI'21. Redistributed via SiMBA++.

DatasetTotal LinesParsedSimplifiedNotesRate
pldi_linear.txt1,0121,0081,0084 comment headers skipped100%
pldi_poly.txt1,0091,0081,0081 comment header skipped100%
pldi_nonpoly.txt1,0051,0031,0031 comment header skipped100%

MBA-Blast Datasets

Originally from the MBA-Blast repository, published at USENIX Security'21. Redistributed via SiMBA++.

DatasetTotal LinesParsedSimplifiedRate
blast_dataset1.txt636262100%
blast_dataset2.txt2,5012,5002,500100%

NeuReduce Dataset

test_data.txt is the NeuReduce dataset (published at EMNLP'20) under a different filename. Redistributed via SiMBA++. A second copy with minor variable renaming (b → t) exists as gamba/neureduce.txt via GAMBA.

DatasetTotal LinesParsedSimplifiedRate
test_data.txt10,00010,00010,000100%

GAMBA Datasets

All files in the gamba/ directory were taken from GAMBA (GPL-3.0-only, Denuvo Software Solutions). GAMBA aggregates datasets from several upstream projects; original provenance is noted per entry.

DatasetOriginTotal LinesParsedSimplifiedUnsupportedRate
loki_tiny.txtLOKI25,02525,00025,0000100%
neureduce.txtNeuReduce10,00010,00010,0000100%
mba_obf_linear.txtGAMBA1,0011,0001,0000100%
mba_obf_nonlinear.txtGAMBA1,0021,0001,0000100%
syntia.txtSyntia5015005000100%
qsynth_ea.txtQSynth5015004663493.2%
mba_flatten.txtMBA-Flatten3,0082,0602,0600100%
  • loki_tiny: 25 sections covering add, subtract, AND, OR, XOR at depths 1-5. All 25,000 are 2-variable linear MBAs. From the Loki repository, published at USENIX Security'22.
  • neureduce.txt: 10,000 linear expressions with 2 to 5 variables. From the NeuReduce dataset, published at EMNLP'20. Same dataset as simba/test_data.txt with minor variable renaming (b → t).
  • mba_obf_nonlinear and mba_obf_linear: 1000 polynomial/nonpolynomial + 1000 linear expressions, all with linear ground-truth targets. All pass full-width verification. From the MBA-Obfuscator repo, published at ICICS'21.
  • syntia: All 500 expressions simplify via the orchestrator's decomposition and lifting passes. From the QSynth repo, published at BAR'20.
  • qsynth_ea: The most challenging dataset. 466 of 500 expressions simplify. The 34 unsupported expressions are mixed bitwise-arithmetic expressions where CoB is boolean-correct but diverges at full width, and polynomial recovery (d=1..4) also fails — a genuine representation gap in carry-sensitive boolean-null residuals. From the QSynth repo, published at BAR'20.
  • mba_flatten: 3,008 lines across 7 sections (2-4 variable linear, sub-expression, and unsolvable-by-other-tools categories). 948 lines skipped (section headers, 3-field sub-expression rows, and expressions that fail parse). All 2,060 parseable expressions simplify at 100%. From MBA-Flatten, published in Security and Communication Networks'22.

OSES Dataset

Source: oracle-synthesis-meets-equality-saturation

DatasetTotal LinesParsedSimplifiedUnsupportedRate
oses_fast.txt4734583906885.2%
oses_slow.txt776185.7%
OSES Total4804653966985.2%
  • oses: 479 MBA expressions extracted from the OSES synth.py evaluation script (plus 1 header comment). Expressions span linear, nonlinear/product, and constant categories with 1-14 variables. The dataset is split into fast (472 expressions under 50K characters) and slow (7 mega-expressions over 50K characters). Both subsets are currently disabled — the fast subset OOMs on deeply nested expressions that exceed memory limits during recursive evaluation, and the slow subset requires minutes per expression. Numbers shown are from the last successful run on master.

ObfuscatorX Dataset

DatasetTotal LinesParsedSimplifiedUnsupportedRate
obfuscatorx.txt8770100%
  • obfuscatorx: 7 expressions lifted from ObfuscatorX. All simplify via the standard pipeline.

Aggregate Summary

MetricCount
Total dataset lines76,144
Comment/header lines skipped1,018
Parsed expressions75,126
Simplified75,023
Unsupported103
Errors / failures0
MBA ClassExpressionsSimplifiedRate
Linear~55,000~55,000~100%
Semilinear~1,000~1,000~100%
Polynomial~5,000~4,950~99%
Mixed~9,000~8,800~98%

All simplified results are validated via spot-check (random-input evaluation) at 64-bit width. When Z3 is available, full equivalence proofs are performed.

Dataset Sources

Redistribution Sources

These are the repositories from which CoBRA's copies were taken.

SourceURLLicenseDatasets
Simplifierhttps://github.com/mazeworks-security/SimplifierGPL-3.0univariate64, multivariate64, permutation64, msimba
SiMBA++https://github.com/pgarba/SiMBA-GPL-3.0E-series, PLDI, BLAST, test_data
GAMBAhttps://github.com/DenuvoSoftwareSolutions/GAMBAGPL-3.0loki_tiny, neureduce, mba_obf_*, mba_flatten, syntia, qsynth_ea
OSEShttps://github.com/fvrmatteo/oracle-synthesis-meets-equality-saturation—oses_all

Original Authors

These are the upstream projects that created the datasets. Some were aggregated into SiMBA++ or GAMBA before CoBRA sourced them.

OriginalURLLicenseDatasets
SiMBAhttps://github.com/DenuvoSoftwareSolutions/SiMBAGPL-3.0E-series
MBA-Solverhttps://github.com/softsec-unh/MBA-SolverGPL-3.0PLDI
MBA-Blasthttps://github.com/softsec-unh/MBA-Blast—BLAST
NeuReducehttps://github.com/fvrmatteo/NeuReduce—test_data, neureduce
Lokihttps://github.com/RUB-SysSec/lokiAGPL-3.0loki_tiny
MBA-Obfuscatorhttps://github.com/nhpcc502/MBA-ObfuscatorMITmba_obf_*
QSynthhttps://github.com/werew/qsynth-artifacts—qsynth_ea, syntia
MBA-Flattenhttps://tinyurl.com/y5l948pu—mba_flatten

See THIRD_PARTY_LICENSES for full provenance and license details.