whfmt.FileFormatCatalog

May 1, 2026 · View on GitHub

Architecture
API Reference
Utility Layer
Integration Guide — Level 1: Basic Detection
Integration Guide — Level 2: Routing and Syntax
Integration Guide — Level 3: Full Pipeline
Integration Guide — Level 4: Rich Metadata
The .whfmt Format

Architecture

Assembly structure

The package ships two assemblies:

whfmt.FileFormatCatalog.nupkg
└── lib/net8.0/
    ├── WpfHexEditor.Core.Definitions.dll   — catalog engine + 675+ embedded resources
    └── WpfHexEditor.Core.Contracts.dll     — public types (interfaces, records, enums)

Consumers only need to reference WpfHexEditor.Core.Definitions. WpfHexEditor.Core.Contracts is bundled and flows transitively — no separate package reference needed.

Type ownership

Type	Assembly	Purpose
`EmbeddedFormatCatalog`	Core.Definitions	Singleton — catalog engine
`IEmbeddedFormatCatalog`	Core.Contracts	Interface — injectable abstraction
`EmbeddedFormatEntry`	Core.Contracts	Immutable record — one format definition
`FormatSignature`	Core.Contracts	Immutable record — one magic-byte signature
`FormatMatchResult`	Core.Contracts	Immutable record — scored detection result
`MatchSource`	Core.Contracts	Flags enum — Extension / MagicBytes / MimeType / Combined
`FormatCategory`	Core.Contracts	Enum — 27 known categories
`SchemaName`	Core.Contracts	Enum — 5 known embedded schemas
`FormatMatcher`	Core.Definitions	Stateless detection façade with confidence scoring
`FormatFileAnalyzer`	Core.Definitions	I/O helper — file path / FileInfo / Stream / async
`CatalogQuery`	Core.Definitions	Fluent query builder
`FormatMetadataExtensions`	Core.Definitions	Extension methods — forensic, AI hints, assertions…
`FormatSummaryBuilder`	Core.Definitions	Human-readable summaries — plain text / Markdown / dump

Initialization and caching

First call to EmbeddedFormatCatalog.Instance
    └── LazyInitializer creates the singleton (thread-safe)
        └── First call to GetAll()
            └── Scans all manifest resource names in the assembly
                └── Filters *.whfmt and *.grammar resources
                    └── LoadHeader() / LoadGrammarHeader() per resource
                        └── Extracts: name, category, extensions, MIME,
                                      signatures, preferredEditor, flags
                    └── Sorted by category then name
                    └── Stored as FrozenSet<EmbeddedFormatEntry>

GetJson(resourceKey) uses a separate Dictionary<string, string> cache — each resource is read from the embedded stream exactly once, then served from memory on all subsequent calls.

Consequence: the first call to GetAll() is the expensive one (~10–50 ms depending on hardware). All subsequent calls are O(1). Call PreWarm() on a background thread at startup to absorb this cost before the first user action.

Thread safety

Instance — safe to access from any thread (backed by LazyInitializer)
GetAll() / GetCategories() — safe; backed by immutable FrozenSet<T>
GetJson() — safe; uses lock + TryAdd pattern
DetectFromBytes() — safe; read-only enumeration over immutable entries
FormatMatcher, FormatFileAnalyzer, CatalogQuery — stateless; safe by construction
All other methods — safe; delegate to the above

Dependency graph

Your app
  └── WpfHexEditor.Core.Definitions   (net8.0)
        └── WpfHexEditor.Core.Contracts  (net8.0)
              └── [BCL only — System.*]

Zero external NuGet dependencies. No WPF, no platform-specific APIs.

API Reference

`EmbeddedFormatCatalog`

Member	Returns	Description
`Instance`	`EmbeddedFormatCatalog`	Singleton — thread-safe, lazy
`GetAll()`	`IReadOnlySet<EmbeddedFormatEntry>`	All entries, sorted by category then name
`GetCategories()`	`IReadOnlySet<string>`	Distinct category names, alphabetical
`GetByExtension(string)`	`EmbeddedFormatEntry?`	First match by extension (case-insensitive, leading dot optional)
`GetByMimeType(string)`	`EmbeddedFormatEntry?`	First match by MIME type (case-insensitive)
`GetByCategory(string)`	`IReadOnlyList<EmbeddedFormatEntry>`	All entries in a category (string overload)
`GetByCategory(FormatCategory)`	`IReadOnlyList<EmbeddedFormatEntry>`	All entries in a category (enum overload)
`DetectFromBytes(ReadOnlySpan<byte>)`	`EmbeddedFormatEntry?`	Best match by magic-byte scoring
`GetCompatibleEditorIds(string)`	`IReadOnlyList<string>`	Compatible editor IDs for a file path
`GetJson(string)`	`string`	Full .whfmt JSON for a resource key (cached)
`GetSyntaxDefinitionJson(string)`	`string?`	Raw `syntaxDefinition` block JSON
`GetSchemaJson(string)`	`string?`	Embedded schema JSON (string overload)
`GetSchemaJson(SchemaName)`	`string?`	Embedded schema JSON (enum overload)
`PreWarm()`	`void`	Pre-load all JSON into memory cache
`.Query()`	`CatalogQuery`	Begin a fluent query (extension method)

`EmbeddedFormatEntry`

Immutable positional record. All fields populated from the .whfmt header at catalog load time.

Field	Type	Notes
`ResourceKey`	`string`	Assembly manifest resource name — pass to `GetJson()` / `GetSyntaxDefinitionJson()`
`Name`	`string`	Human-readable format name, e.g. `"ZIP Archive"`
`Category`	`string`	Logical category, e.g. `"Archives"`
`Description`	`string`	Short description of the format
`Extensions`	`IReadOnlyList<string>`	File extensions with leading dot, e.g. `[".zip", ".jar"]`
`MimeTypes`	`IReadOnlyList<string>?`	MIME types, e.g. `["application/zip"]`. Null when not declared.
`Signatures`	`IReadOnlyList<FormatSignature>?`	Magic-byte signatures. Null when not declared.
`QualityScore`	`int`	0–100 completeness score from `QualityMetrics.CompletenessScore`
`Version`	`string`	Format spec version, e.g. `"1.14"`. Empty when not specified.
`Author`	`string`	Author/organization. Empty when not specified.
`Platform`	`string`	Target platform for ROM/game formats, e.g. `"Nintendo Entertainment System"`. Empty otherwise.
`PreferredEditor`	`string?`	Recommended editor ID. Null when not declared. Typical values: `"hex-editor"`, `"code-editor"`, `"structure-editor"`.
`IsTextFormat`	`bool`	True when `detection.isTextFormat` is set in the .whfmt file
`HasSyntaxDefinition`	`bool`	True when the .whfmt file contains a `syntaxDefinition` block
`DiffMode`	`string?`	Preferred diff algorithm: `"text"`, `"semantic"`, `"binary"`. Null when absent.

`FormatMatchResult`

Immutable scored detection result. Produced by FormatMatcher and FormatFileAnalyzer.

Member	Type	Description
`Entry`	`EmbeddedFormatEntry`	The matched format
`Confidence`	`double`	Normalised 0.0–1.0 confidence score
`Source`	`MatchSource`	Which strategy produced this match
`RawScore`	`double`	Accumulated signature weight before normalisation
`IsConfirmed`	`bool`	True when Source == Combined (extension + magic bytes both matched)

`MatchSource` flags enum

Value	Meaning
`Extension`	Matched by file extension
`MagicBytes`	Matched by magic-byte signature scoring
`MimeType`	Matched by MIME type string
`Combined`	Extension + MagicBytes — highest confidence

`FormatCategory` enum

Value	Category string
`Archives`	`"Archives"`
`Audio`	`"Audio"`
`CAD`	`"CAD"`
`Certificates`	`"Certificates"`
`Crypto`	`"Crypto"`
`Data`	`"Data"`
`Database`	`"Database"`
`Disk`	`"Disk"`
`Documents`	`"Documents"`
`Executables`	`"Executables"`
`Firmware`	`"Firmware"`
`Fonts`	`"Fonts"`
`GIS`	`"GIS"`
`Game`	`"Game"`
`Images`	`"Images"`
`MachineLearning`	`"MachineLearning"`
`Medical`	`"Medical"`
`Network`	`"Network"`
`Programming`	`"Programming"`
`RomHacking`	`"RomHacking"`
`Science`	`"Science"`
`Subtitles`	`"Subtitles"`
`Synalysis`	`"Synalysis"`
`System`	`"System"`
`Text`	`"Text"`
`Video`	`"Video"`
`_3D`	`"3D"`
`Other`	`"Other"`

FormatCategory._3D maps to the string "3D" — the enum overload handles this automatically.

`SchemaName` enum

Value	Schema file	Use case
`Whfmt`	`whfmt.schema.json`	Validate `.whfmt` format definitions
`Whcd`	`whcd.schema.json`	Class diagram visual state
`Whdbg`	`whdbg.schema.json`	Debug launch configuration
`Whidews`	`whidews.schema.json`	Workspace archive manifest
`Whscd`	`whscd.schema.json`	Solution-wide class diagram

Utility Layer

The utility layer sits on top of IEmbeddedFormatCatalog and covers four concerns: matching with confidence scores, I/O-free file analysis, fluent filtering, and rich metadata extraction. All utilities are cross-platform (net8.0, zero external dependencies).

Namespaces

using WpfHexEditor.Core.Definitions.Matching;   // FormatMatcher, FormatFileAnalyzer
using WpfHexEditor.Core.Definitions.Query;      // CatalogQuery, CatalogQueryExtensions
using WpfHexEditor.Core.Definitions.Metadata;   // FormatMetadataExtensions, FormatSummaryBuilder

`FormatMatcher` — Scored detection façade

FormatMatcher is a stateless class whose methods combine extension + magic-byte + MIME-type detection into a single FormatMatchResult with a normalised confidence score.

Confidence scale

Confidence	Source	Meaning
`1.0`	`Combined`	Extension + magic bytes both agree
`0.51–0.99`	`MagicBytes`	Byte scoring only (no extension, or extension disagreed)
`0.5`	`Extension`	Extension matched, no signatures to confirm
`0.4`	`MimeType`	MIME type matched only

Single best match

using WpfHexEditor.Core.Definitions.Matching;

var catalog = EmbeddedFormatCatalog.Instance;
byte[] header = File.ReadAllBytes("myfile.zip")[..512];

var result = FormatMatcher.Match(catalog, ".zip", header);
// result.Entry.Name    → "ZIP Archive"
// result.Confidence    → 1.0
// result.Source        → MatchSource.Combined
// result.IsConfirmed   → true

Match from a full file path (extension extracted automatically):

var result = FormatMatcher.Match(catalog, @"C:\uploads\archive.zip", header);

Top-N ranked candidates

Useful when debugging ambiguous files or building a "pick format" UI:

var candidates = FormatMatcher.GetTopMatches(catalog, header, maxResults: 5);

foreach (var match in candidates)
    Console.WriteLine(match); // "ZIP Archive [MagicBytes] 99% (raw=1.00)"

All entries for an extension

Some extensions map to multiple formats (e.g. .bin). Get all candidates ranked by quality:

var all = FormatMatcher.GetMatchesByExtension(catalog, ".bin");
// Returns multiple FormatMatchResult sorted by QualityScore descending

MIME-type match

var result = FormatMatcher.MatchMime(catalog, "application/pdf");
// result.Entry.Name   → "PDF Document"
// result.Confidence   → 0.4   (MIME types are often ambiguous)
// result.Source       → MatchSource.MimeType

`FormatFileAnalyzer` — I/O without boilerplate

FormatFileAnalyzer reads the first 512 bytes from a file (or stream) and delegates to FormatMatcher. It eliminates the byte-reading boilerplate from every call site.

From a file path

using WpfHexEditor.Core.Definitions.Matching;

var catalog = EmbeddedFormatCatalog.Instance;

var result = FormatFileAnalyzer.Analyze(catalog, @"C:\files\document.pdf");
Console.WriteLine(result?.Entry.Name);      // "PDF Document"
Console.WriteLine(result?.Confidence);      // 1.0
Console.WriteLine(result?.IsConfirmed);     // true

From a `FileInfo`

var fi = new FileInfo(@"C:\files\archive.7z");
var result = FormatFileAnalyzer.Analyze(catalog, fi);

From a `Stream` (extension hint optional)

using var stream = assembly.GetManifestResourceStream("MyApp.Resources.data.bin")!;
var result = FormatFileAnalyzer.Analyze(catalog, stream, extension: ".bin");

Async — from a file path

var result = await FormatFileAnalyzer.AnalyzeAsync(catalog, @"C:\uploads\file.unknown");
if (result is null)
    Console.WriteLine("Format not recognised.");
else
    Console.WriteLine($"{result.Entry.Name} ({result.Source}, {result.Confidence:P0})");

Async — from a stream

await using var fs = File.OpenRead(uploadedFile);
var result = await FormatFileAnalyzer.AnalyzeAsync(catalog, fs,
    extension: Path.GetExtension(uploadedFile),
    cancellationToken: ct);

Batch — scan a directory

var catalog = EmbeddedFormatCatalog.Instance;

foreach (var (path, match) in FormatFileAnalyzer.AnalyzeDirectory(
    catalog,
    directory: @"C:\Data",
    searchPattern: "*.*",
    recursive: true))
{
    var name = match?.Entry.Name ?? "Unknown";
    var conf = match?.Confidence.ToString("P0") ?? "—";
    Console.WriteLine($"{Path.GetFileName(path),-30}  {name,-25}  {conf}");
}

`CatalogQuery` — Fluent filtering

CatalogQuery is a composable query builder. Obtain one via the .Query() extension method on any IEmbeddedFormatCatalog instance.

Basic example

using WpfHexEditor.Core.Definitions.Query;

var catalog = EmbeddedFormatCatalog.Instance;

var highQualityDiskFormats = catalog
    .Query()
    .InCategory(FormatCategory.Disk)
    .WithMinQuality(80)
    .HasMagicBytes()
    .OrderByQuality()
    .Execute();

foreach (var fmt in highQualityDiskFormats)
    Console.WriteLine($"{fmt.Name,30}  {fmt.QualityScore}%");

Filter methods

Method	Description
`.InCategory(FormatCategory)`	Restrict to a category (enum — compile-time safe)
`.InCategory(string)`	Restrict to a category (string overload)
`.WithMinQuality(int)`	Keep entries with QualityScore ≥ threshold
`.PriorityOnly()`	Shortcut: QualityScore ≥ 85
`.HasMagicBytes()`	Only entries with at least one signature
`.WithExtension(string)`	Only entries that handle a given extension (leading dot optional)
`.TextFormatsOnly()`	Only text-based formats
`.BinaryFormatsOnly()`	Only binary formats
`.HasSyntaxDefinition()`	Only entries with a grammar block
`.WithPreferredEditor(string)`	Exact editor ID match, e.g. `"code-editor"`
`.HasPreferredEditor()`	Any non-null, non-empty preferred editor (v1.1.1)
`.HasMimeType()`	Only entries that declare a MIME type
`.HasPlatform()`	Only entries with a non-empty platform field
`.ForPlatform(string)`	Platform substring match, e.g. `"Nintendo"`
`.WithDiffMode(string)`	`"text"`, `"binary"`, or `"semantic"`
`.WithName(string)`	Exact name match (case-insensitive) (v1.1.1)
`.Containing(string)`	Full-text search in name + description
`.Where(predicate)`	Custom predicate

Ordering methods

Method	Description
`.OrderByQuality()`	Highest quality score first
`.OrderByName()`	Alphabetical by name
`.OrderByCategoryThenName()`	Category, then name within category

Terminal methods

Method	Returns	Description
`.Execute()`	`IReadOnlyList<EmbeddedFormatEntry>`	All matching entries
`.First()`	`EmbeddedFormatEntry?`	First match or null
`.Count()`	`int`	Count without materialising the list
`.Any()`	`bool`	True when at least one entry matches (v1.1.1)
`.Select<T>(selector)`	`IReadOnlyList<T>`	Project each entry (v1.1.1)
`.ToDictionary<K,V>(keySelector, valueSelector)`	`Dictionary<K,V>`	Build a dictionary; auto `OrdinalIgnoreCase` for string keys (v1.1.1)
`.ToExtensionDictionary()`	`Dictionary<string, EmbeddedFormatEntry>`	Flat extension→entry map (v1.1.1)
`.ToExtensionDictionary<V>(valueSelector)`	`Dictionary<string, V>`	Extension→value routing map (v1.1.1)
`.GroupByCategory()`	`IReadOnlyDictionary<string, IReadOnlyList<EmbeddedFormatEntry>>`	Alphabetically sorted category groups (v1.1.1)

Real-world examples

All code-editor grammars with a syntax definition:

var grammars = catalog
    .Query()
    .InCategory(FormatCategory.Programming)
    .HasSyntaxDefinition()
    .OrderByName()
    .Execute();

Console.WriteLine($"{grammars.Count} language grammars available.");

ROM formats for Nintendo platforms:

var nintendoRoms = catalog
    .Query()
    .InCategory(FormatCategory.RomHacking)
    .ForPlatform("Nintendo")
    .OrderByQuality()
    .Execute();

Extension→editor routing map (one line):

// Build a lookup: file extension → preferred editor ID
var editorMap = catalog.Query()
    .HasPreferredEditor()
    .ToExtensionDictionary(e => e.PreferredEditor!);

var editor = editorMap.GetValueOrDefault(".cs"); // "code-editor"

Group by category for a tree view:

foreach (var (category, entries) in catalog.Query().OrderByName().GroupByCategory())
{
    Console.WriteLine($"[{category}]");
    foreach (var e in entries)
        Console.WriteLine($"  {e.Name}");
}

Check if a name exists in the catalog:

bool exists = catalog.Query().WithName("ZIP Archive").Any();

High-risk crypto formats:

var cryptoFormats = catalog.Query()
    .InCategory(FormatCategory.Crypto)
    .WithMinQuality(50)
    .Execute();

var highRisk = cryptoFormats
    .Where(e => e.IsHighRisk(catalog))
    .ToList();

Structure-editor formats with export templates:

var structured = catalog
    .Query()
    .WithPreferredEditor("structure-editor")
    .WithMinQuality(70)
    .Execute()
    .Where(e => e.GetExportTemplates(catalog).Count > 0)
    .ToList();

`FormatMetadataExtensions` — Rich whfmt metadata

These extension methods on EmbeddedFormatEntry surface the deep metadata blocks from the full .whfmt JSON — forensic, AI hints, navigation, assertions, inspector groups, export templates, and technical details.

All methods take the catalog as a second parameter to load the JSON on demand. The JSON is cached by the catalog after the first call.

Bulk parse — `GetAllMetadata()` (v1.1.1)

When you need two or more metadata blocks for the same entry, use GetAllMetadata() — it parses the .whfmt JSON exactly once and returns a FormatMetadata record with all blocks.

var meta = entry.GetAllMetadata(catalog);
// meta.Forensic          → ForensicSummary?
// meta.AiHints           → AiHints?
// meta.Bookmarks         → IReadOnlyList<NavigationBookmark>
// meta.Assertions        → IReadOnlyList<AssertionRule>
// meta.InspectorGroups   → IReadOnlyList<InspectorGroup>
// meta.ExportTemplates   → IReadOnlyList<ExportTemplate>
// meta.TechnicalDetails  → TechnicalDetails?
// meta.IsHighRisk        → bool (shortcut)
// meta.SupportsEncryption→ bool (shortcut)

// Much more efficient than chaining individual calls:
// ❌ entry.GetForensicSummary(catalog) + entry.GetAiHints(catalog)  → 2 JSON parses
// ✅ entry.GetAllMetadata(catalog)                                    → 1 JSON parse

Forensic data

using WpfHexEditor.Core.Definitions.Metadata;

var catalog = EmbeddedFormatCatalog.Instance;
var entry = catalog.GetByExtension(".jks")!;  // Java KeyStore

var forensic = entry.GetForensicSummary(catalog);
// forensic.Category           → "crypto"
// forensic.RiskLevel          → "medium"
// forensic.IsHighRisk         → false
// forensic.SuspiciousPatterns → list of SuspiciousPattern records

foreach (var p in forensic!.SuspiciousPatterns)
    Console.WriteLine($"⚠ {p.Name}: {p.Description}  (condition: {p.Condition})");

// Quick boolean check:
bool dangerous = entry.IsHighRisk(catalog);

AI analysis hints

var ai = entry.GetAiHints(catalog);
// ai.AnalysisContext       → "Java KeyStore format..."
// ai.SuggestedInspections  → ["Check version field", "Inspect alias count", ...]
// ai.KnownVulnerabilities  → []

Console.WriteLine(ai?.AnalysisContext);
foreach (var hint in ai?.SuggestedInspections ?? [])
    Console.WriteLine($"  → {hint}");

var apfs = catalog.GetByExtension("")
    ?? catalog.Query().InCategory(FormatCategory.Disk).Containing("APFS").First()!;

var bookmarks = apfs.GetNavigationBookmarks(catalog);
// [
//   NavigationBookmark("Object Header",        Offset=null, OffsetVar="currentOffset", Icon="header"),
//   NavigationBookmark("NXSB Magic",           Offset=32,   OffsetVar=null,            Icon="signature"),
//   NavigationBookmark("Container UUID",       Offset=72,   OffsetVar=null,            Icon="id"),
// ]

foreach (var b in bookmarks)
    Console.WriteLine($"  [{b.Icon}] {b.Name} @ {b.Offset?.ToString("X4") ?? b.OffsetVar}");

Validation assertions

var assertions = apfs.GetAssertions(catalog);
// [
//   AssertionRule("NXSB magic valid", "nxMagic == 'NXSB'", "error", "Invalid APFS magic"),
//   AssertionRule("block size is power of 2", "nxBlockSize >= 512 && ...", "warning", null),
// ]

foreach (var a in assertions)
    Console.WriteLine($"  [{a.Severity.ToUpperInvariant()}] {a.Name}: {a.Expression}");

Inspector groups

var groups = apfs.GetInspectorGroups(catalog);
// [
//   InspectorGroup("Object Header",      "header", ["objChecksum","objOid","objXid",...]),
//   InspectorGroup("Container Superblock","disk",  ["nxMagic","nxBlockSize",...]),
//   InspectorGroup("Feature Flags",      "flag",  ["nxFeatures",...]),
// ]

foreach (var g in groups)
    Console.WriteLine($"  [{g.Icon}] {g.Title}: {string.Join(", ", g.Fields)}");

Export templates

var templates = apfs.GetExportTemplates(catalog);
// [
//   ExportTemplate("APFS Container Header (JSON)", "json", ["nxMagic","nxBlockSize",...]),
//   ExportTemplate("APFS Summary (CSV)",           "csv",  ["nxMagic","nxBlockSize",...]),
// ]

foreach (var t in templates)
    Console.WriteLine($"  Export as {t.Format.ToUpperInvariant()}: {t.Name}  ({t.Fields.Count} fields)");

Technical details

var td = apfs.GetTechnicalDetails(catalog);
// td.Endianness         → "little"
// td.DataStructure      → "Copy-on-write B-tree filesystem..."
// td.SupportsEncryption → true
// td.Encryption         → "Per-file or per-volume encryption..."

Console.WriteLine($"Endianness: {td?.Endianness}");
Console.WriteLine($"Encrypted : {td?.SupportsEncryption}");

// Quick boolean check:
bool encrypted = apfs.SupportsEncryption(catalog);

`FormatSummaryBuilder` — Human-readable summaries

Generates formatted summaries without any WPF or MVVM dependency.

using WpfHexEditor.Core.Definitions.Metadata;

var entry = EmbeddedFormatCatalog.Instance.GetByExtension(".zip")!;

string label = FormatSummaryBuilder.BuildOneLiner(entry);
// "ZIP Archive (Archives) — .zip .jar .apk — Quality: 92%"

Plain text (multi-line)

string text = FormatSummaryBuilder.BuildPlainText(entry);
// Name        : ZIP Archive
// Category    : Archives
// Description : ZIP archive format (PKZIP)...
// Extensions  : .zip  .jar  .apk
// MIME        : application/zip
// Quality     : 92/100
// Signatures  :
//    @   0  50 4B 03 04  weight=1.00
//    @   0  50 4B 05 06  weight=0.80

Pass the catalog to include rich metadata in the output:

string richText = FormatSummaryBuilder.BuildPlainText(entry, catalog);
// … includes Forensic, AI context, TechnicalDetails

Markdown card

string md = FormatSummaryBuilder.BuildMarkdown(entry, catalog);
// ## ZIP Archive
// > ZIP archive format (PKZIP)...
// | Field | Value |
// |---|---|
// | Category | `Archives` |
// | Extensions | `.zip` `.jar` `.apk` |
// | MIME | `application/zip` |
// | Quality | 92/100 |
// ...
// ### Magic Bytes
// | Offset | Signature | Weight |
// |---|---|---|
// | `0x0000` | `50 4B 03 04` | 1.00 |
// ...
// ### Forensic (low risk)
// ### Navigation Bookmarks
// ### Validation Assertions

Diagnostic dump (debugging)

string dump = FormatSummaryBuilder.BuildDiagnosticDump(entry, catalog);
// === FORMAT DIAGNOSTIC DUMP ===
// ResourceKey         : WpfHexEditor.Core.Definitions.FormatDefinitions.Archives.ZIP.whfmt
// Name                : ZIP Archive
// QualityScore        : 92
// Forensic.RiskLevel  : low  (high=False)
// Assertions (3):
//   [ERROR] NXSB magic valid: nxMagic == 'NXSB'
// Bookmarks (3):
//   Object Header  offset=currentOffset  icon=header
// ==============================

Signature formatting utility

string hex = FormatSummaryBuilder.FormatHex("504B0304");
// "50 4B 03 04"

Level 1: Basic Detection

This level covers the most common scenario: identifying a file and deciding what to do with it.

Extension lookup

The fastest lookup — no I/O, pure in-memory scan.

using WpfHexEditor.Core.Definitions;
using WpfHexEditor.Core.Contracts;

var catalog = EmbeddedFormatCatalog.Instance;

var entry = catalog.GetByExtension(".zip");
// entry.Name          → "ZIP Archive"
// entry.Category      → "Archives"
// entry.Description   → "ZIP archive format (PKZIP)..."
// entry.PreferredEditor → "hex-editor"
// entry.MimeTypes[0]  → "application/zip"

Extensions are case-insensitive and the leading dot is optional:

catalog.GetByExtension(".ZIP");  // same result
catalog.GetByExtension("zip");   // same result

Magic-byte detection

Use when the extension is missing, untrusted, or ambiguous. Pass as many bytes as you can — at least 16, ideally 512.

byte[] header = new byte[512];
using (var fs = File.OpenRead(filePath))
    fs.Read(header, 0, header.Length);

var entry = catalog.DetectFromBytes(header);
if (entry is not null)
    Console.WriteLine($"Detected: {entry.Name}");

How scoring works

Each entry's Signatures list is evaluated. For every signature where the bytes match at the declared offset, its Weight (0.0–1.0) is added to a running score. The entry with the highest total score wins. Entries with no signatures are skipped entirely.

ZIP has 3 signatures:
  "504B0304" offset 0  weight 1.0  ← Local File Header
  "504B0506" offset 0  weight 0.8  ← End of Central Directory
  "504B0708" offset 0  weight 0.5  ← Data Descriptor

A normal .zip file matches the first → score 1.0 → wins if no other format scores higher.

MIME type lookup

var entry = catalog.GetByMimeType("image/png");
// entry.Name       → "PNG Image"
// entry.Extensions → [".png"]

Category browsing

// Enum overload — recommended
var archives = catalog.GetByCategory(FormatCategory.Archives);

// Iterate
foreach (var fmt in archives)
    Console.WriteLine($"{fmt.Name}: {string.Join(", ", fmt.Extensions)}");

Level 2: Routing and Syntax

This level covers editor routing and syntax highlighting integration.

Editor routing

GetCompatibleEditorIds returns all editor IDs that can open a given file. "hex-editor" is always included as the universal fallback.

var editors = catalog.GetCompatibleEditorIds("report.pdf");
// ["hex-editor", "structure-editor"]

var editors2 = catalog.GetCompatibleEditorIds("main.cs");
// ["hex-editor", "code-editor", "text-editor"]

Routing logic applied internally:

Condition	Editor added
Always	`"hex-editor"`
`PreferredEditor` is set	that value
`IsTextFormat == true`	`"code-editor"`, `"text-editor"`
`Category == "Images"`	`"image-viewer"`
`Category == "Audio"`	`"audio-viewer"`
`DiffMode == "text"`	`"diff-viewer"`

Registering grammars for a syntax engine

GetSyntaxDefinitionJson returns the raw syntaxDefinition JSON block from the .whfmt file. Feed it into your tokenizer or highlighter.

var catalog = EmbeddedFormatCatalog.Instance;

// Load all programming language grammars
foreach (var entry in catalog.GetByCategory(FormatCategory.Programming)
                              .Where(e => e.HasSyntaxDefinition))
{
    string? grammar = catalog.GetSyntaxDefinitionJson(entry.ResourceKey);
    if (grammar is null) continue;

    // Example: register with a hypothetical tokenizer registry
    // MyTokenizer.Register(entry.Name, grammar);
    Console.WriteLine($"  {entry.Name} → {entry.Extensions.FirstOrDefault()}");
}

Available language grammars (35): Assembly, Batch, C, C++, C#, C# Script, CSS, Dart, F#, Go, HTML, Java, JavaScript, JSON, Kotlin, Lua, Markdown, PHP, Perl, PowerShell, Python, Ruby, Rust, Shell, SQL, Swift, TOML, TypeScript, VB.NET, WHFMT, XAML, XML, XMLMarkup, YAML.

Detecting extension spoofing

bool IsExtensionSpoofed(string filePath)
{
    var byExtension = catalog.GetByExtension(Path.GetExtension(filePath));
    if (byExtension is null) return false;

    using var fs = File.OpenRead(filePath);
    var header = new byte[512];
    int read = fs.Read(header, 0, header.Length);
    var byBytes = catalog.DetectFromBytes(header.AsSpan(0, read));

    return byBytes is not null && byBytes.ResourceKey != byExtension.ResourceKey;
}

if (IsExtensionSpoofed(@"uploads\document.pdf"))
    throw new SecurityException("File content does not match declared extension.");

Or use the confidence-aware version via FormatMatcher:

using WpfHexEditor.Core.Definitions.Matching;

byte[] header = File.ReadAllBytes(filePath)[..512];
var result = FormatMatcher.Match(catalog, filePath, header);

bool spoofed = result is not null
    && result.Source == MatchSource.MagicBytes
    && !result.IsConfirmed;

MIME negotiation for HTTP

// Extension → MIME (Content-Type header)
string? GetContentType(string extension)
    => catalog.GetByExtension(extension)?.MimeTypes?.FirstOrDefault()
       ?? "application/octet-stream";

// MIME → extension (download filename)
string GetExtensionForMime(string mimeType)
    => catalog.GetByMimeType(mimeType)?.Extensions.FirstOrDefault()
       ?? ".bin";

// In an ASP.NET controller:
Response.ContentType = GetContentType(Path.GetExtension(fileName));

Accessing embedded JSON schemas

Use SchemaName enum for compile-time safety:

// Get the whfmt schema to validate user-provided format definitions
string? schema = catalog.GetSchemaJson(SchemaName.Whfmt);

// Pass to a JSON schema validator (e.g. JsonSchema.Net)
// var jsonSchema = JsonSchema.FromText(schema);
// var result = jsonSchema.Evaluate(JsonNode.Parse(userWhfmt));

Level 3: Full Pipeline

This level covers production-grade integration patterns.

File identification with `FormatFileAnalyzer` (recommended)

using WpfHexEditor.Core.Definitions.Matching;

var catalog = EmbeddedFormatCatalog.Instance;

// One line — no byte-reading boilerplate
var result = FormatFileAnalyzer.Analyze(catalog, filePath);

if (result is null)
{
    Console.WriteLine("Format not recognised.");
}
else
{
    Console.WriteLine($"Format    : {result.Entry.Name}");
    Console.WriteLine($"Category  : {result.Entry.Category}");
    Console.WriteLine($"Confidence: {result.Confidence:P0}");
    Console.WriteLine($"Source    : {result.Source}");
    Console.WriteLine($"Confirmed : {result.IsConfirmed}");
}

File identification pipeline with fallback chain (low-level)

public sealed class FileIdentifier
{
    private readonly IEmbeddedFormatCatalog _catalog;

    public FileIdentifier(IEmbeddedFormatCatalog catalog) => _catalog = catalog;

    public EmbeddedFormatEntry? Identify(string filePath)
    {
        // 1 — Fast path: extension lookup
        var byExt = _catalog.GetByExtension(Path.GetExtension(filePath));
        if (byExt?.Signatures is { Count: > 0 })
        {
            // Confirm with magic bytes when signatures are available
            using var fs = File.OpenRead(filePath);
            var header = new byte[512];
            int read = fs.Read(header, 0, header.Length);
            var byBytes = _catalog.DetectFromBytes(header.AsSpan(0, read));
            if (byBytes is not null) return byBytes;
        }

        // 2 — Extension match without signatures (text formats, config files)
        if (byExt is not null) return byExt;

        // 3 — Pure magic-byte scan (no extension, or unknown extension)
        {
            using var fs = File.OpenRead(filePath);
            var header = new byte[512];
            int read = fs.Read(header, 0, header.Length);
            return _catalog.DetectFromBytes(header.AsSpan(0, read));
        }
    }
}

Dependency injection setup

IEmbeddedFormatCatalog is the injectable interface. Register the singleton in your DI container:

// Microsoft.Extensions.DependencyInjection
services.AddSingleton<IEmbeddedFormatCatalog>(EmbeddedFormatCatalog.Instance);

// Then inject normally
public class MyService(IEmbeddedFormatCatalog catalog) { ... }

Background pre-warming

public static class AppStartup
{
    public static Task PreWarmCatalogAsync()
        => Task.Run(() =>
        {
            // Forces singleton creation + full entry scan + JSON cache fill
            EmbeddedFormatCatalog.Instance.PreWarm();
        });
}

// In Program.cs or App.xaml.cs — fire and forget, don't await
_ = AppStartup.PreWarmCatalogAsync();

Batch folder scanner with parallel processing

using WpfHexEditor.Core.Definitions.Matching;

var catalog = EmbeddedFormatCatalog.Instance;

var results = Directory
    .EnumerateFiles(@"C:\Data", "*.*", SearchOption.AllDirectories)
    .AsParallel()
    .WithDegreeOfParallelism(Environment.ProcessorCount)
    .Select(path =>
    {
        FormatMatchResult? match = null;
        try { match = FormatFileAnalyzer.Analyze(catalog, path); }
        catch { /* skip locked / inaccessible files */ }

        return new
        {
            Path      = path,
            Name      = match?.Entry.Name ?? "Unknown",
            Category  = match?.Entry.Category ?? "Unknown",
            Confidence= match?.Confidence ?? 0,
            IsSpoofed = match?.Source == MatchSource.MagicBytes && !match.IsConfirmed,
        };
    })
    .ToList();

// Summary by category
foreach (var g in results.GroupBy(r => r.Category).OrderByDescending(g => g.Count()))
    Console.WriteLine($"{g.Key,-20} {g.Count(),5} file(s)  spoofed: {g.Count(f => f.IsSpoofed)}");

Full syntax engine bootstrap

using WpfHexEditor.Core.Definitions.Query;

public sealed class SyntaxEngineBootstrapper
{
    private readonly IEmbeddedFormatCatalog _catalog;
    private readonly Dictionary<string, string> _grammarsByExtension = new(StringComparer.OrdinalIgnoreCase);

    public SyntaxEngineBootstrapper(IEmbeddedFormatCatalog catalog)
    {
        _catalog = catalog;
        LoadAll();
    }

    private void LoadAll()
    {
        // Fluent query — all programming language grammars
        var entries = _catalog
            .Query()
            .InCategory(FormatCategory.Programming)
            .HasSyntaxDefinition()
            .Execute();

        foreach (var entry in entries)
        {
            var grammar = _catalog.GetSyntaxDefinitionJson(entry.ResourceKey);
            if (grammar is null) continue;
            foreach (var ext in entry.Extensions)
                _grammarsByExtension.TryAdd(ext, grammar);
        }
    }

    public string? GetGrammar(string fileExtension)
        => _grammarsByExtension.GetValueOrDefault(fileExtension);

    public string GetWhfmtSchema()
        => _catalog.GetSchemaJson(SchemaName.Whfmt) ?? throw new InvalidOperationException("Schema not found.");
}

Level 4: Rich Metadata

This level covers access to the deep metadata blocks embedded in each .whfmt file — forensic intelligence, AI hints, navigation, assertions, inspector groups, and export templates.

Security scanner — detect high-risk formats

using WpfHexEditor.Core.Definitions.Metadata;
using WpfHexEditor.Core.Definitions.Matching;
using WpfHexEditor.Core.Definitions.Query;

var catalog = EmbeddedFormatCatalog.Instance;

// Identify file
var result = FormatFileAnalyzer.Analyze(catalog, filePath);
if (result is null) return;

var entry = result.Entry;

// Forensic check
var forensic = entry.GetForensicSummary(catalog);
if (forensic?.IsHighRisk == true)
{
    Console.WriteLine($"⛔ HIGH RISK: {entry.Name} ({forensic.RiskLevel})");
    foreach (var p in forensic.SuspiciousPatterns)
        Console.WriteLine($"   ⚠ {p.Name}: {p.Description}");
}

// AI-assisted inspection hints
var ai = entry.GetAiHints(catalog);
if (ai?.SuggestedInspections.Count > 0)
{
    Console.WriteLine("Suggested inspections:");
    foreach (var hint in ai.SuggestedInspections)
        Console.WriteLine($"   → {hint}");
}

Binary validator — run format assertions

// Get assertion rules for the detected format
var assertions = entry.GetAssertions(catalog);

Console.WriteLine($"Validation rules for {entry.Name} ({assertions.Count} assertions):");
foreach (var a in assertions)
    Console.WriteLine($"  [{a.Severity,-7}] {a.Name}: {a.Expression}");

// assertions can be passed to a rule-engine / expression evaluator
// e.g. MyRuleEngine.Evaluate(parsedFields, assertions);

Structure viewer — build inspector groups

// Inspector groups define how parsed fields are visually organised
var groups = entry.GetInspectorGroups(catalog);

foreach (var group in groups)
{
    Console.WriteLine($"[{group.Icon}] {group.Title}");
    foreach (var field in group.Fields)
        Console.WriteLine($"    {field}");
}

Export pipeline — enumerate available templates

var templates = entry.GetExportTemplates(catalog);

foreach (var tmpl in templates)
{
    Console.WriteLine($"Export: {tmpl.Name} [{tmpl.Format.ToUpperInvariant()}]");
    Console.WriteLine($"  Fields: {string.Join(", ", tmpl.Fields)}");
}

var bookmarks = entry.GetNavigationBookmarks(catalog);

foreach (var b in bookmarks)
{
    var position = b.Offset.HasValue
        ? $"0x{b.Offset:X4}"
        : $"${b.OffsetVar}";
    Console.WriteLine($"  [{b.Icon,-12}] {b.Name,-25} @ {position}");
}

Diagnostic report — full entry dump

using WpfHexEditor.Core.Definitions.Metadata;

var entry = catalog.GetByExtension(".jks")!;
string dump = FormatSummaryBuilder.BuildDiagnosticDump(entry, catalog);
Console.WriteLine(dump);

Format report — Markdown card for documentation

var entry = catalog.Query()
    .InCategory(FormatCategory.Disk)
    .Containing("APFS")
    .First()!;

string markdown = FormatSummaryBuilder.BuildMarkdown(entry, catalog);
File.WriteAllText("apfs-format-card.md", markdown);

All-in-one: catalogue of high-quality formats with rich metadata

using WpfHexEditor.Core.Definitions.Query;
using WpfHexEditor.Core.Definitions.Metadata;

var catalog = EmbeddedFormatCatalog.Instance;

var formats = catalog
    .Query()
    .WithMinQuality(80)
    .HasMagicBytes()
    .OrderByQuality()
    .Execute();

Console.WriteLine($"{"Format",-30} {"Category",-15} {"Q",3}  {"Risk",-8}  {"Assertions",10}  {"Exports",7}");
Console.WriteLine(new string('-', 85));

foreach (var fmt in formats)
{
    var forensic   = fmt.GetForensicSummary(catalog);
    var assertions = fmt.GetAssertions(catalog);
    var exports    = fmt.GetExportTemplates(catalog);

    Console.WriteLine(
        $"{fmt.Name,-30} {fmt.Category,-15} {fmt.QualityScore,3}  " +
        $"{(forensic?.RiskLevel ?? "—"),-8}  {assertions.Count,10}  {exports.Count,7}");
}

The .whfmt Format

A .whfmt file is a JSONC document (JSON with // comments and trailing commas allowed) that describes a binary or text file format. Here is an annotated example:

{
  // Root identification fields
  "formatName": "ZIP Archive",           // Human-readable name
  "version": "1.14",                     // Format spec version
  "category": "Archives",               // Must match a FormatCategory value
  "description": "...",                 // Short description
  "author": "WPFHexaEditor Team",
  "preferredEditor": "hex-editor",      // "hex-editor" | "code-editor" | "structure-editor" | etc.
  "diffMode": "binary",                 // "text" | "semantic" | "binary"

  // File association
  "extensions": [ ".zip", ".jar", ".apk" ],
  "MimeTypes":  [ "application/zip" ],

  // Detection rules
  "detection": {
    "signatures": [
      { "value": "504B0304", "offset": 0, "weight": 1.0 },  // hex bytes, byte offset, confidence
      { "value": "504B0506", "offset": 0, "weight": 0.8 }
    ],
    "matchMode": "any",                  // "any" | "all"
    "required": true,                   // whether a signature match is mandatory
    "EntropyHint": { "min": 5.0, "max": 8.0 },
    "isTextFormat": false               // true for plain-text formats (CSV, INI, etc.)
  },

  // Quality metadata
  "QualityMetrics": {
    "CompletenessScore": 90             // 0–100
  },

  // Forensic intelligence
  "forensic": {
    "category": "archive",
    "riskLevel": "low",                 // "low" | "medium" | "high" | "critical"
    "suspiciousPatterns": [
      {
        "name": "Zip bomb",
        "description": "Deeply nested archives with extreme compression ratio",
        "condition": "compressionRatio > 1000"
      }
    ]
  },

  // AI-assisted analysis hints
  "aiHints": {
    "analysisContext": "ZIP is a widely-used container format...",
    "suggestedInspections": [
      "Check local file header counts",
      "Verify central directory offset"
    ]
  },

  // Navigation bookmarks
  "navigation": {
    "bookmarks": [
      { "name": "Local File Header", "offset": 0,  "icon": "signature" },
      { "name": "Central Directory", "offsetVar": "cdOffset", "icon": "directory" }
    ]
  },

  // Validation assertions
  "assertions": [
    {
      "name": "Magic bytes valid",
      "expression": "signature == 'PK'",
      "severity": "error",
      "message": "Missing ZIP magic bytes"
    }
  ],

  // Inspector layout (field grouping for UI)
  "inspector": {
    "groups": [
      {
        "title": "File Header",
        "icon": "header",
        "fields": ["signature", "version", "flags", "compression"]
      }
    ]
  },

  // Export templates
  "exportTemplates": [
    {
      "name": "ZIP Summary (JSON)",
      "format": "json",
      "fields": ["signature", "version", "compression", "crc32"]
    }
  ],

  // Technical details
  "TechnicalDetails": {
    "endianness": "little",
    "compressionMethod": "DEFLATE / Store / BZip2 / LZMA",
    "supportsEncryption": true,
    "encryption": "Traditional PKWARE, AES-256"
  },

  // Optional: syntax grammar block (only for text/code formats)
  "syntaxDefinition": {
    "name": "...",
    "scopeName": "source.example",
    "patterns": [ ]                     // tokenizer rules
  }
}

Key fields used by the catalog API

Field	API surface
`formatName`	`EmbeddedFormatEntry.Name`
`category`	`EmbeddedFormatEntry.Category` / `GetByCategory()` / `.Query().InCategory()`
`extensions`	`EmbeddedFormatEntry.Extensions` / `GetByExtension()` / `.Query().WithExtension()`
`MimeTypes`	`EmbeddedFormatEntry.MimeTypes` / `GetByMimeType()` / `FormatMatcher.MatchMime()`
`detection.signatures`	`EmbeddedFormatEntry.Signatures` / `DetectFromBytes()` / `FormatMatcher.Match()`
`preferredEditor`	`EmbeddedFormatEntry.PreferredEditor` / `GetCompatibleEditorIds()`
`detection.isTextFormat`	`EmbeddedFormatEntry.IsTextFormat` / `.Query().TextFormatsOnly()`
`syntaxDefinition`	`EmbeddedFormatEntry.HasSyntaxDefinition` / `GetSyntaxDefinitionJson()` / `.Query().HasSyntaxDefinition()`
`diffMode`	`EmbeddedFormatEntry.DiffMode` / `.Query().WithDiffMode()`
`QualityMetrics.CompletenessScore`	`EmbeddedFormatEntry.QualityScore` / `.Query().WithMinQuality()`
`forensic`	`entry.GetForensicSummary(catalog)` / `entry.IsHighRisk(catalog)`
`aiHints`	`entry.GetAiHints(catalog)`
`navigation.bookmarks`	`entry.GetNavigationBookmarks(catalog)`
`assertions`	`entry.GetAssertions(catalog)`
`inspector.groups`	`entry.GetInspectorGroups(catalog)`
`exportTemplates`	`entry.GetExportTemplates(catalog)`
`TechnicalDetails`	`entry.GetTechnicalDetails(catalog)` / `entry.SupportsEncryption(catalog)`

Validating your own .whfmt files

string? schema = EmbeddedFormatCatalog.Instance.GetSchemaJson(SchemaName.Whfmt);
// Pass to JsonSchema.Net, Newtonsoft.Json.Schema, or any JSON Schema validator

The embedded schema (whfmt.schema.json) is the authoritative v2.4 schema used internally to validate all 675+ bundled definitions at build time.