HypercubeWTF C++ SDK

September 11, 2026 · View on GitHub

You place a fixed pattern on the hypercube — an image pack, a spectrum, or any field you built yourself. HypercubeWTF drives that field through a frozen reservoir for a short synthetic orbit, then trains a small CNN only on the state at the end. One class does the whole loop: collect episodes, train the head, predict.

You do not need to learn HypercubeESN or HypercubeCNN first. Link HypercubeWTFCore, include WTF.h, and work with WTF. Demos and packing helpers are optional recipes; they are not the product.

This guide matches the public headers for 1.0.x.

Who it is for: anyone embedding WTF in a host (collect → train → predict), and anyone learning the stack with the same API the demos use.

What you get: a C++23 static library. Headers sit at the repo root. A vendored HypercubeCNN builds the trainable readout; hosts usually never call HCNN themselves.

Section
1. Why explore HypercubeWTF	Family role, early properties, why try it
2. The big picture	Where WTF sits among ESN / CNN
3. One episode	Dynamics, spatial→temporal, mechanics
4. Product surface	Headers, rules, the loop
5. Build	CMake, binaries
6. First program	Minimal collect → train → predict
7. API	Config and methods
8–13	Boundaries, demos, pitfalls, cheat sheet

1. Why explore HypercubeWTF

HypercubeESN processes temporal streams.

HypercubeCNN processes spatial data sets.

HypercubeWTF also processes spatial data sets, however it differs from HypercubeCNN in that before reaching the convolution engine, each data set is passed through a dynamical encoder (the reservoir). The convolution engine never sees the original data set; it sees an encoded version of it generated by the reservoir.

The internal dynamics of that particular encoding process (as opposed to more traditional static function encoders) appear to have some interesting transformational properties — e.g., filtering white noise when present, while acting as an identity transformation when noise is not present, and reducing sensitivity to training data quality when noise is present. For the details, see WhiteNoiseFilter.md and TrainingDataQualitySensitivity.md.

HypercubeWTF is yet another experiment in the HypercubeAI project — our quest to map AI and ML strategies onto the hypercube as a shared computational substrate. Each product in the family is a different architecture on that same foundation; this one is the dynamical-encoder path for static fields.

Whether this dynamical encoding → CNN pipeline has real product value is still an open question. We are exploring it as a new technique on static fields — and if that kind of open question interests you, you are in the right place.

2. The big picture

Most learning systems either see a stream (one small input every step) or a static pattern (classify an image once). WTF sits in between:

You give it one full-length field on the cube (packed however you like).
It re-addresses that same field for T passes (a synthetic orbit).
It takes a single snapshot at the end and trains a CNN on those features.

The reservoir weights never train. Only the readout does. That is the whole product idea.

Library	You typically feed it…	What runs
HypercubeESN	a stream of small inputs over real time	step → state → HCNN
HypercubeCNN	a static pattern already on the cube	conv stack → labels
HypercubeWTF	a static length-N field with no real time	orbit → end state → HCNN

Your data does not have to be a power of two

dim is the size knob for the cube: set ReservoirConfig::dim and you get N = 2^dim vertices. WTF always expects a field of that length.

If your raw data is 784 pixels or 300 bins, you map it onto N floats first (pad, resize, spatial embed, custom layout — your choice). WTF does not invent that map. Demo helpers under examples/common/ are one MNIST-oriented recipe; skip them when you pack your own way.

What freezes vs what learns

Piece	Trains?
Reservoir weights and bias	No — drawn once at construct
Starting state s0 (full delay line)	No — drawn once from `ic_seed`, reloaded every episode
How you pack domain data into the field	Your problem (outside WTF)
HCNN readout	Yes

Episode start state (s0)

Every episode begins by reloading the same frozen initial condition into the full delay line, then setting the pass counter c to 0. That way two runs of the same field (same config) are deterministic, and you never inherit residual state from the previous sample.

Rule	Behavior
Size	`N × M` floats — one full delay-line worth of state
When drawn	Once, at `WTF` construction
Seed	`ic_seed` — separate from `reservoir.seed` (weights)
Distribution	i.i.d. uniform on [-0.5, 0.5] over the whole buffer
After construct	Immutable for the life of that `WTF`
Each episode	Reload into the live delay line (age-correct load, not a blind mid-rotation overwrite)
Pass counter	`c = 0` at episode start

If you change only ic_seed, weights stay the same but the end features change (different orbit start). If you change only reservoir.seed, the frozen graph weights change.

3. One episode, step by step

Think of an episode as: reset to a known start, drive for a while, read once.

That sentence is the same rhythm as classical reservoir computing (echo-state / ESN style), just aimed at a static field instead of a live stream.

Where the dynamical magic comes from

A reservoir is a fixed nonlinear dynamical system. At construction it draws random weights once and never updates them again: connections along the cube (how each vertex talks to its bit-flip neighbors), a small input map that brings the field onto those vertices (W_in), and optional per-vertex bias. The recurrent weights are then scaled so their spectral radius sits near a chosen target — strong enough to mix, not so strong that the state blows up — and left alone. You never backprop through that core. Training only fits a thin head on top of the state — in WTF, a HypercubeCNN readout.

Each step does two things that matter:

Drive — the current field pattern is injected through W_in so every vertex feels a local mix of the input (strength set by input_scaling).
Recur — each vertex updates from itself and its cube neighbors (bit-flip edges), through a nonlinearity (tanh-family), with optional leak so the state blends old and new rather than replacing fully.

Do that many times and the high-dimensional state becomes a nonlinear trajectory shaped by both the drive history and the fixed graph. The “echo” idea is that recent drive still rings in the state while older influence fades — if the spectral radius and leak are in a sensible regime, trajectories stay rich without exploding. WTF reloads the same frozen s0 every episode so two runs of the same field (same packing, same config) are deterministic.

Making time when you only have a still picture

A classical ESN expects a movie: at time 0 a small input u_0, at time 1 u_1, and so on. Each tick, fresh information arrives from the outside world. The reservoir’s job is to remember and mix that stream.

WTF usually has the opposite problem: one still — a full length-N field (image pack, spectrum, whatever) with no natural “next frame.” If you only shoved that field in once and stepped forever, most of the pattern would be a one-shot kick; the dynamics would not systematically walk the spatial structure.

So WTF invents a clock from space. Call the pass counter c (starts at 0 each episode). On pass c, every vertex v is driven by field sample

x[(v XOR c) & (N − 1)]

Read that as: the numbers in x never change; you only change which number sits on which vertex. XOR with c is a fixed, invertible shuffle of addresses on the cube. Increment c, shuffle again. Geometry (who is neighbor to whom) and all frozen weights stay put — they do not slide with c. What moves is the registration of the field onto the graph.

Do that for T passes and the reservoir experiences a synthetic time series: not new pixels from a camera, but the same global pattern seen under T successive addressings. Larger T is allowed; the mask wraps and the tour repeats. Set episode.T = 0 only if you want a full-cube orbit (T = N after construction). That orbit is what the echo digests.

You still follow RC discipline at the end: you do not train on every intermediate state. After the last pass you read once.

The reservoir keeps a short history of its state (the delay line, depth M). “Age 0” is the newest snapshot; age 1 is one step older; and so on. By default you hand the readout only the newest slice (B = 1). You can optionally pack the B newest ages into one longer feature vector (B must be a power of two and ≤ M) if you want a little more recent history in one row. Either way, that end pack is the feature the CNN sees: a nonlinear, dynamical summary of “this whole field, driven through this orbit, starting from this s0.” Different fields leave different end signatures; the head only has to separate those.

In short: spatial pattern in → synthetic time by re-addressing → reservoir dynamics → one end snapshot → trained head. The recurrent core is not learned; only the head is.

Mechanics in order

Reload the frozen initial condition s0 into the delay line.
For pass c = 0, 1, …, T−1:
- Place the field on the cube with address offset c.
- One reservoir step (field inject + W_in gather + recurrent update).
Build features from the end of the delay line only: ages 0 … B−1 concatenated → length B × N.
Hand those features to the readout (collect, train, or predict).

$\text{text} \text{x} (\text{length} \text{N}, \text{fixed} \text{for} \text{this} \text{episode}) │ ▼ \text{Load} \text{s0} → \text{drive} \text{T} \text{times} → \text{pack} \text{B} \text{end} \text{ages} → \text{features} (\text{B} \times \text{N}) │ ▼ \text{HCNN} \text{readout} → \text{class} \text{logits} \text{or} \text{regression} \text{values} $

Words you will see in the API

Word	Plain meaning
dim	Cube dimension you choose (5…16); set as `reservoir.dim`
N	Field length = 2^dim (also one reservoir state slice)
T	How many drive passes (`episode.T`; `0` expands to N at construct)
B	How many end delay-line ages go into the feature vector (`readout_slices`)
M	Delay-line depth (`history_depth`)
s0	Frozen start state, length `N × M`, U[-0.5, 0.5] from `ic_seed` (not the weight seed); reloaded every episode
FeatureSize	`B × N` — what the readout eats

B must be a power of two and no larger than M. Default is B = 1 (newest slice only).

4. What is the product (and what is not)

You care about…	Use…
Integrating the library	`WTF` + `WTFConfig`
Logging realized spectral radius, saving readout weights	`wtf.reservoir()` / `wtf.readout()`
Learning by example	`wtf_smoke`, `wtf_synth`, `wtf_mnist`
MNIST paths / packing demos	`examples/common/` (optional)
Raw HypercubeCNN	Almost never — that lives under the readout

WTF.h              front door (WTF, WTFConfig, EpisodeConfig)
Reservoir.h        ReservoirConfig (+ Reservoir for inspection)
Readout.h          ReadoutConfig, enums, Readout
… .cpp files …
third_party/HypercubeCNN/    vendored; see VENDORED.md
examples/                    demos, not the SDK definition
docs/CPP_SDK.md              this guide

Link HypercubeWTFCore (it pulls HypercubeCNNCore for you).

Rules that matter

These are product contracts, not implementation trivia.

Every field is length N. Wrong size throws.
Values are usually kept in [-1, 1]. The library trusts the host; it does not clamp.
You pack; WTF drives. No built-in image layout.
Reservoir and s0 freeze at construct. s0 is length N × M, drawn once from ic_seed (U[-0.5, 0.5]); every episode reloads that same buffer.
Only the end state goes to the readout (optional multi-age pack B).
Train noise is collect-only. Predict / RunEpisode stay clean.
Predict returns raw logits (or regression values) — no softmax.
AccuracyOnCollected / R2OnCollected are training-set scores, not test-set metrics.
One WTF per thread of control. Bulk collect parallelizes inside one call; do not call public methods concurrently on the same object.
WTF is not copyable. Prefer exclusive ownership of one instance.

The loop you will write

fill WTFConfig
construct WTF once
collect many episodes   (optional train-input noise)
TrainOnCollected
Predict / PredictClass  (always a fresh clean episode)

Optional extras: RunEpisode + LastFeatures, train-set metrics, ClearCollected, collect_threads for faster bulk collect.

5. Build and consume

You need C++23 and CMake ≥ 3.21. Prefer Release when you care about study numbers (Debug and Release float behavior can differ with this project’s fast-math flags).

In CLion: open the project, reload CMake, build. From a shell with the toolchain available:

cmake --build cmake-build-release

When this repo is the top-level project you also get:

Binary	Role
`wtf_smoke`	Fast contract + train smoke
`wtf_synth`	Multi-class synthetic fields (no data files)
`wtf_mnist`	MNIST recipe (IDX files under `C:\HypercubeWTF\data`)

If you pull HypercubeWTF in as a subdirectory, demos are skipped; you still get the library.

add_subdirectory(path/to/HypercubeWTF)
add_executable(my_app main.cpp)
target_link_libraries(my_app PRIVATE HypercubeWTFCore)
target_include_directories(my_app PRIVATE path/to/HypercubeWTF)

#include "WTF.h"

6. First program

A tiny two-class example — collect, train, predict. (Verified against the library: trains and predicts correctly on this toy task.)

#include "WTF.h"
#include <cstdio>
#include <vector>

int main() {
    WTFConfig cfg;
    cfg.reservoir.dim = 5;              // N = 32
    cfg.reservoir.history_depth = 4;
    cfg.reservoir.seed = 1;
    cfg.ic_seed = 2;
    cfg.episode.T = 100;
    cfg.episode.readout_slices = 1;     // B = 1
    cfg.readout.dim = 0;                // auto
    cfg.readout.num_outputs = 2;
    cfg.readout.task = ReadoutTask::Classification;
    cfg.readout.epochs = 80;
    cfg.readout.num_threads = 1;
    cfg.readout.restore_best_epoch = false;

    WTF wtf(cfg);
    const size_t N = wtf.N();

    auto field = [&](int label) {
        std::vector<float> x(N, 0.f);
        const float s = (label == 0) ? 1.f : -1.f;
        for (size_t i = 0; i < N / 2; ++i)
            x[i] = s * (0.2f + 0.8f * float(i) / float(N));
        return x;
    };

    for (int i = 0; i < 24; ++i) {
        wtf.CollectEpisode(field(0), 0);
        wtf.CollectEpisode(field(1), 1);
    }
    wtf.TrainOnCollected();

    // AccuracyOnCollected is the *training* set, not test data.
    std::printf("train acc=%.3f  pred0=%d pred1=%d\n",
                wtf.AccuracyOnCollected(),
                wtf.PredictClass(field(0)),
                wtf.PredictClass(field(1)));
    return 0;
}

Habits that save pain later

Build the config, construct one WTF (weights and s0 freeze here).
Collect a dataset, then train. You can call TrainOnCollected again without clearing if you want another pass on the same features.
Treat Predict / PredictClass as clean inference — they never apply collect noise.
Keep packing in your code (or a demo helper). The core only accepts length-N fields.

7. The API you actually use

Authoritative signatures and contracts live in WTF.h (and the headers it pulls). This section is the host-oriented map.

Config at a glance

Everything interesting is set before WTF is constructed.

struct EpisodeConfig {
    size_t T;                             // drive passes; 0 → use N
    size_t readout_slices = 1;            // B
    size_t collect_threads = 0;           // 0 = auto (leave OS/UI some cores)
    float  train_input_noise_sigma = 0.f; // collect only
    bool   bypass_reservoir = false;      // eval control only; needs B == 1
};

struct WTFConfig {
    ReservoirConfig reservoir{};
    ReadoutConfig   readout{};
    EpisodeConfig   episode{};
    uint64_t        ic_seed = 1;          // s0 only
};

Reservoir (frozen dynamics) — common knobs. Header defaults exist; in-tree demos tune these widely, so treat “typical” as a starting band, not a recipe.

Field	Meaning	Valid / notes	Often in demos
`dim`	Cube dimension; N = 2^dim	5…16	7…10
`seed`	Weight draws	any `uint64_t`	fixed per experiment
`spectral_radius`	Target for recurrent rescale	> 0	≈0.4…0.999
`leak_rate`	Mix each step	(0, 1]	0.5…1
`input_scaling`	How hard the field drives	≥ 0	≈0.005…0.03
`history_depth`	M (delay-line depth)	1…64	4…16
`bias_scaling`	Bias strength; 0 = off	≥ 0	0…≈0.003
`verbose`	Construction printout	bool	false

Readout (trainable head) — knobs most hosts touch:

Field	Meaning
`dim`	Feature cube dim; 0 = auto (`reservoir.dim` + log₂(B))
`num_outputs`	Classes, or regression width
`task`	`ReadoutTask::Classification` or `Regression`
`epochs`, `batch_size`	Batch training
`lr_max`, `lr_min_frac`, `lr_decay_epochs`	Cosine learning-rate schedule
`num_threads`	HCNN workers — use 1 for simple determinism
`restore_best_epoch`	Keep best-epoch weights (Readout default true; demos often false)
`seed`	Readout weight init

Deeper fields (num_layers, pooling, activation, batch-norm, optimizer, holdout fraction, epoch_tick) live on ReadoutConfig in Readout.h. Classification hosts often leave them alone. epoch_tick is an optional callback fired after each batch-training epoch with the 1-based epoch and the training-set error; the Raman example uses it to print a per-epoch RMSE.

After construct — sizes and inspection

explicit WTF(const WTFConfig& cfg);

wtf.N();  wtf.T();  wtf.B();  wtf.M();
wtf.FeatureSize();      // B * N
wtf.NumCollected();
wtf.CollectedFeatures(); // span, sample-major, NumCollected() * FeatureSize()
wtf.NumOutputs();
wtf.CollectThreads();   // configured preference (0 = auto)
wtf.BypassReservoir();

wtf.reservoir();        // const — e.g. realized spectral radius
wtf.readout();          // const — weights, save/load helpers
wtf.readout_config();   // resolved config (dim filled in if auto)

Construction checks the usual mistakes: bad T, B not a power of two, B > M, bypass with B ≠ 1, invalid noise σ, wrong readout dim, num_outputs < 1.

Run an episode (no training)

wtf.RunEpisode(x);               // x.size() == N; x is not modified
auto feats = wtf.LastFeatures(); // length FeatureSize()

LastFeatures also updates after serial CollectEpisode, Predict, and PredictClass. Bulk CollectEpisodes does not touch it — those features only go into the training set.

Collect, train, predict

Classification

wtf.CollectEpisode(x, class_label);             // one sample
wtf.CollectEpisodes(fields_flat, labels);       // bulk, sample-major
wtf.CollectEpisodes(count, labels, fill_field); // fill_field(i, span) writes N floats

Regression — same idea with target vectors (num_outputs floats per sample) via the span<const float> overloads.

wtf.ClearCollected();           // drop training rows (keeps worker pool)
wtf.TrainOnCollected();         // needs at least one sample; does not clear

auto logits = wtf.Predict(x);   // num_outputs floats; no softmax
int y = wtf.PredictClass(x);    // classification only

double acc = wtf.AccuracyOnCollected();  // train set
double r2  = wtf.R2OnCollected();        // train set, regression

Bulk layout notes:

fields_flat is sample-major: sample i starts at i * N.
fill_field may run on several samples at once — only touch index i’s buffer.
Wrong task (class API on a regression net, etc.) throws.

Train-input noise

Set episode.train_input_noise_sigma > 0 to add Gaussian noise to the field only when collecting. The draw is deterministic from ic_seed and the sample index. Inference paths ignore σ. Name is deliberate: this is not eval noise.

Faster bulk collect

episode.collect_threads:

0 — auto (leaves one or two cores free so the machine stays responsive)
1 — serial
K — up to K workers

Worker 0 reuses the primary reservoir. Extra workers clone frozen weights once. The internal thread pool grows for the life of the WTF and does not shrink.

Eval-only: skip the reservoir

episode.bypass_reservoir (requires B == 1) copies the field into the feature buffer with no orbit. Measurement control for tests and studies only — default off. Collect noise still applies when σ > 0.

8. Please do not

Temptation	Better path
Drive `Reservoir` yourself for product training	Use `WTF` episodes
Call vendored `hcnn::HCNN` from the app	Let `Readout` own it; export via `wtf.readout()` if needed
Depend on `examples/common` in production	Copy the idea; own your packing
Stream intermediate passes into the readout	Product samples end of episode only
Expect ESN-style external feedback	Not ported on purpose

Reservoir and Readout headers are public so config and inspection work. The happy path is still collect → train → predict on WTF.

9. Demos as recipes

Demos keep product knobs in MakeWTFConfig() and demo-only constants (k*) beside them:

config → WTF → collect → train → score → predict

Demo	When to open it
`tests/wtf_smoke.cpp`	Contracts: sizes, determinism, parallel collect, noise
`examples/synth/wtf_synth.cpp`	Fast multi-class without data files
`examples/mnist/wtf_mnist.cpp`	Real packing + larger train loop

More context: examples/README.md.

10. Threads, memory, and cost

Treat one WTF as exclusive for public calls.
Bulk collect is where parallelism belongs; it clones reservoirs as needed.
Orbit cost grows with T × N; features are B × N; the CNN scales with its own dim and channels.
If you already run many WTF instances in parallel, set readout.num_threads = 1 so HCNN does not oversubscribe the machine.
Prefer Release when comparing accuracies across runs.

11. Common mistakes

Symptom / assumption	Fix
Throw on collect / run	Field length must equal `wtf.N()`
“Why can’t I pass 784 floats?”	Pack to N first
Softmax inside `Predict`	You get logits; use `PredictClass` or argmax
Eval looks noisy after setting σ	σ is collect-only
`LastFeatures` empty/stale after bulk collect	Expected — re-run `RunEpisode` if you need them
Great train accuracy, bad real test	`AccuracyOnCollected` is the training set
Construct fails on B	Power of two, and `B ≤ M`
Construct fails with `bypass_reservoir`	Needs `B == 1` (eval control)
Racey results with shared `WTF`	One instance, one host thread of control
Linked HypercubeCNN only	Link `HypercubeWTFCore`, include `WTF.h`

12. Further reading

Doc	What it is
WhiteNoiseFilter.md	White-noise field study (MNIST vehicle)
TrainingDataQualitySensitivity.md	Training-set quality study (MNIST vehicle)
examples/README.md	Demo map and MNIST data notes
VENDORED.md	Which HypercubeCNN pin is in tree
HypercubeCNN’s own C++ SDK (upstream)	Deep CNN contracts if you dig into the readout

13. Cheat sheet

#include "WTF.h"

WTFConfig cfg;
cfg.reservoir.dim = 7;                 // N = 128
cfg.reservoir.history_depth = 8;
cfg.reservoir.seed = 1;
cfg.ic_seed = 2;
cfg.episode.T = 100;
cfg.episode.readout_slices = 1;        // B
cfg.episode.train_input_noise_sigma = 0.f;
cfg.readout.dim = 0;                   // auto = dim + log2(B)
cfg.readout.num_outputs = K;
cfg.readout.task = ReadoutTask::Classification;
cfg.readout.epochs = 100;
cfg.readout.num_threads = 1;

WTF wtf(cfg);

wtf.CollectEpisodes(fields_flat, labels);   // count * N floats, sample-major
wtf.TrainOnCollected();

int y = wtf.PredictClass(x);
auto logits = wtf.Predict(x);

wtf.RunEpisode(x);
auto feats = wtf.LastFeatures();            // B * N
float sr = wtf.reservoir().GetRealizedSpectralRadius();

In one line: pack a field → frozen orbit → end features → train the CNN head.