Why the 93× Batch Speedup Exceeds the 53× Single-Call Speedup: A Two-Stage Explanatory Decomposition

Model provenance: GPT-5.6-Sol (via Codex CLI), 2026-07-12 (initial version); DeepSeek-V4-Pro (via Claude Code CLI), 2026-08-15 (figure correction & restructure) Correction note: The initial version mistakenly treated the "2.2× batch-interface savings" as the end-to-end speedup when building the Amdahl analysis. This version corrects it to the measured 93× end-to-end speedup (reproducible benchmark JSON, commit ca13e26) and restructures the analysis from a single-stage Amdahl model to a two-stage acceleration model. 简体中文 | 正體中文

Project scope: This repository is a Python/C++ hybrid programming exercise. The measurements below concern computational performance, not trading returns or strategy quality.

1. Introduction

Three benchmark results summarize the promise and limits of this pybind11 refactor. A dynamic time warping (DTW) distance calculation runs 34× faster in C++ than in its Python equivalent. A complete single pattern match—15-dimensional feature extraction, DTW refinement, and candidate ranking—reaches 53×. And the batch workload, covering 100 timestamps, reaches 93× end to end.

At first glance this seems counterintuitive: why would the batch speedup (93×) be higher than the single-call speedup (53×)? Intuitively, a wider measurement boundary should include more non-accelerated work and yield a smaller speedup.

The answer is that the batch interface changes the structure of the workload. The 53× comes from replacing the Python interpreter loop with compiled C++ loops; the batch interface then does a second thing—merging 100 timestamps into one C++ call and eliminating 99 Python↔C++ boundary crossings. The two stages multiply: 93× ≈ 53× × 1.75×. Here 93× and 53× are directly measured by the benchmark, while 1.75× is an explanatory estimate extrapolated from the single-call median (see §4). This is not a contradiction; it is the direct expression of the batch interface's value.

2. Experimental Setup

Benchmarks compare the Python and C++ implementations on the same machine with the same synthetic inputs, with the pattern-matching algorithm unchanged. The processor, memory system, OS state, and compiler version all affect absolute timings. Because the paired results come from the same hardware, the relative speedups are more comparable, but they still apply only to this workload and should not be treated as universal pybind11 constants.

Item	Detail
Hardware	Windows 11 consumer laptop, Intel64 Family 6 Model 183 (specific model omitted)
Python	3.12.7 + NumPy 2.4.4
C++	C++20, MSVC 19.51 (Visual Studio 2026 Community), pybind11 3.0.4
Compiler flags	/O2, Release configuration
Test data	One price series (length 1000), synthetic random walk (seed=42); single T_idx=500, batch 100 T_idx (400–499)
Measurement	time.perf_counter(), 5 warm-up runs followed by 100 timed runs, median
Reproducible commands	python verify_etf_core.py + python verify_batch.py; raw data in benchmarks/results/2026-07-16-ca13e26.json

Synthetic data is appropriate here because the goal is to measure execution cost, not to simulate a real market. The fixed random seed makes the workload reproducible, and the consistency scripts check that the C++ implementation preserves the Python algorithm output within the project's floating-point tolerances. These numbers should therefore be understood as an engineering comparison of two implementations, not a backtest, and certainly not evidence of investment performance.

3. Three-Tier Benchmark Results

Function	Python	C++	Speedup
DTW Distance (L=19)	96 µs	2.8 µs	34×
Single Pattern Match	14.0 ms	0.26 ms	53×
Batch Match (×100)	1412 ms	15.1 ms	93×

¹ The batch row compares the pure-Python batch implementation (strictly looping pattern_match_single ×100) with one C++ batch call — a 93× end-to-end speedup within the benchmark call boundary (excluding input conversion).

3.1 Tier 1 — Micro: a single compute kernel

The DTW benchmark compares one C++ function against its Python equivalent on length-19 inputs. The measurement boundary is deliberately narrow: the main cost is the nested numerical loop, with peripheral orchestration barely included in the timed region. Once the array crosses the binding boundary, C++ executes the dynamic-programming recurrence with compiled loops and contiguous numeric storage, leaving almost no Python work inside the timed region.

34× is therefore a reasonable result when an interpreter-level loop is replaced by optimized native code. It is a valid measurement, but it answers a narrow question: how much faster can this kernel's single call be at the current input dimensions? It cannot answer how much faster the whole application will ultimately be.

3.2 Tier 2 — Meso: one complete pattern match

The single-match benchmark widens the boundary to include 15-dimensional feature generation, candidate filtering, DTW scoring, and ranking. After the initial call, the stages stay chained inside C++ without intermediate values returning to Python. The native implementation also keeps the hot loops, temporary vectors, and ranking logic in the same compiled execution region.

The result is 53×, even higher than the isolated DTW speedup. This is reasonable because the C++ path accelerates more than just DTW: it eliminates Python overhead across several cooperating loops and avoids creating Python objects between internal stages. The conclusion here is not that pybind11 automatically makes functions 53× faster, but that such gains require placing a large enough, compute-dense unit of work behind a single binding call.

3.3 Tier 3 — Macro: the 100-timestamp batch workload

The batch benchmark measures a complete workflow rather than an isolated kernel. The Python-side batch implementation strictly loops 100 single matches (100 × ~14 ms ≈ 1412 ms); the C++ side merges 100 timestamps into one pattern_match_batch call (15.1 ms). The end-to-end speedup is 93×.

That 93× exceeds 53× is not a measurement error. The batch interface eliminates a class of overhead that a single call cannot: 100 Python↔C++ boundary crossings, 100 parameter validations, repeated candidate-window recomputation, and 100 output-object constructions. The batch function does each of these once.

4. Two-Stage Acceleration Analysis

4.1 Why the single-stage Amdahl model does not apply

Amdahl's law describes the overall benefit of accelerating part of a program, and it implicitly assumes the overall speedup cannot exceed the speedup of the accelerated part. If "single-call 53×" is treated as the algorithmic speedup ceiling for the batch scenario, the batch end-to-end speedup should be below 53×. But it measures 93×—above 53×—which means the batch scenario contains two independent acceleration stages that cannot be compressed into a single-stage Amdahl model.

4.2 Two-stage decomposition

Decompose the batch scenario into two orthogonal stages:

Stage one (algorithmic): replace the Python interpreter loop with compiled C++ loops.

• Python side looping 100 single matches: 1412 ms
• C++ side looping 100 single matches: ~26.5 ms (100 × 0.265 ms)
• Stage-one speedup: ~53×

Stage two (interface merging): merge 100 C++ calls into one batch call.

• C++ looping 100 calls: ~26.5 ms
• C++ batch call ×1: 15.1 ms
• Stage-two speedup: ~1.75×

Multiplying the two:

Total speedup = 53 × 1.75 ≈ 93×

This matches the measured 93×. Note that 26.5 ms is extrapolated from the single-call C++ median (264.6 µs × 100), not independently measured on the same batch input, so 1.75× is an explanatory estimate; "53× × 1.75× ≈ 93×" is algebraically identical (the intermediate 26.5 ms cancels out) — it explains the composition of 93× but does not independently verify two-stage orthogonality. The two stages are orthogonal: stage one removes the Python overhead within each timestamp, while stage two removes the repeated shared overhead between timestamps.

4.3 The source of stage two

Stage two's 1.75× comes from the batch interface reducing shared work from "per call" to "once per batch". The following four sources are reasonable hypotheses — the benchmark JSON contains only whole-function timings, with no per-stage profiling data to verify each one:

Shared work	Single call ×100	Batch call ×1
Input validation	100×	1×
Language-boundary crossing	100×	1×
Overlapping candidate-window computation	100× (repeated)	1× (reused)
Output-object construction	100×	1× (collected)

4.4 Can it go faster?

The two stages have each completed their most direct optimization. The remaining headroom lies in widening the batch boundary: the F16, F17, F20, F21 market-environment features are still computed on the Python side, and data preparation and per-item result handling also remain in Python. Merging these stages into a single end-to-end native pipeline is the next optimization direction—more valuable than further squeezing the already-optimized DTW inner loop.

5. Engineering Implications

5.1 pybind11 suits compute-dense, call-sparse workloads

A single language-boundary call has a fixed cost. Community benchmarks and the pybind11 documentation indicate roughly 0.5–2 µs per call, depending on function-signature complexity, argument conversion, platform, and build configuration (a general order-of-magnitude estimate, not a measurement from this project). For an operation that takes 15 ms and still forms a sizable native task after migration, this cost is negligible; for a 2.8 µs DTW call it is far more significant. If thousands of such tiny functions are called from Python, entering and leaving C++ dominates even when the C++ function body is excellent. The remedy is to expose coarser-grained operations.

5.2 The batch interface is the highest-leverage optimization

Changing the interface from 100 individually orchestrated calls to one batch call contributes ~1.75× on top of the 53× algorithmic speedup, pushing end-to-end acceleration to 93×. Such architectural savings are often more valuable than shaving another percentage point from an already-optimized inner loop. A good pybind11 API minimizes crossing frequency, not merely crossing cost.

5.3 Speedups must carry context

34× describes the compute kernel, 53× the native computational unit, and 93× the batch workload's end-to-end acceleration. All three are true because they answer different questions. Reporting only the single-call 53× would miss the batch interface's value; reporting only the 93× would hide its decomposition relationship to the single-call speedup. Performance reports should therefore state the benchmark boundary, input scale, call count, measurement method, and the work still left in Python. A useful report gives kernel, single-call, and batch figures together and explains the relationship among them.

6. Visualization

Use a horizontal bar chart comparing two bars: the top bar, C++ single call ×100 (~26.5 ms), is one solid bar; the bottom bar, C++ batch call ×1 (15.1 ms), is a shorter solid bar. The height difference (~11.3 ms) is the eliminated shared overhead; mark it with a dashed or transparent filler and annotate the four sources from §4.3: repeated boundary crossings, repeated validation, repeated candidate-window recomputation, and repeated output-object construction.

[Chart to be generated separately with matplotlib — data in §3 and §4]

Initial version by GPT-5.6-Sol (via Codex CLI) · 2026-07-12; corrected and restructured by DeepSeek-V4-Pro · 2026-08-15