Plugin system (third-party backends + global functions)
September 13, 2026 · View on GitHub
Status: default-context SPI + scan report + INSTALLER all BUILT and verified; ALC isolation and UNINSTALL DEFERRED. A plugin (a folder of managed assemblies) under a plugin root —
FABRICATOR_PLUGIN_DIRif set, else~/.duckdb/fabricator/plugins, searched RECURSIVELY — is discovered at load, itsIBackend(s) registered, and its global functions surfaced as a barefn(...)with NO ATTACH — verified end-to-end (Fabricator.SamplePlugin'splug_greet,test/verify_plugin.test).fabricator_plugins()reports what the scan looked at and why it refused anything;fabricator_install_plugin()unpacks an archive into a root and makes its PROVIDER usable in the SAME session (test/verify_plugin_install.test). Loaded into the default (non-isolated) context for now; per-pluginAssemblyLoadContextisolation (for conflicting transitive deps) is the deferred upgrade — a loader-internal swap, no contract change. The contract assemblyFabricator.Abstractionsis extracted (a plugin references it + Apache.Arrow only — see recommendation #2). Builds onBackendRegistry+ docs/global-functions.md + docs/provider-extensibility.md. The load-bearing constraint regardless of ALC: Apache.Arrow must be shared, never isolated.As-built (no-ALC SPI) — the one non-obvious real-world finding: hostfxr loads the bridge into a non-default ALC, so the loader must load plugins into the bridge's own context (
AssemblyLoadContext.GetLoadContext(typeof(BackendRegistry).Assembly)), NOTAssemblyLoadContext.Default. Loading into Default made the plugin bind to a separate copy ofFabricator.Bridge(from the plugin dir) → itsIBackendwas a different, non-assignable type → 0 backends registered. The loader (BackendRegistry): splitsFABRICATOR_PLUGIN_DIR(comma list of dirs), installs aResolvinghook on the host context (probes the plugin dirs for a plugin's private transitive deps), skips assemblies already loaded in the host context (the shared set —Fabricator.Bridge,Apache.Arrow, built-in providers — so a plugin-dir copy of the bridge isn't reflected + itsStubBackendre-registered; ⚠ MINUS what the scan itself loaded from a plugin directory, or a RE-scan would report every plugin assharedand drop it — see the three re-scan hazards under the installer), andLoadFromAssemblyPaths the rest into the host context, reflecting forIBackend. The scan runs insideDiscover()(firstBackendRegistry.All(), at load — before thelist_global_functionsunion), so a plugin's global functions register with no ABI/C++ change. No-op Sample:dotnet/Fabricator.SamplePlugin(a catalog-lessIBackendwhose only job is to contribute the global scalarplug_greet), built to a folder and pointed at via the env var.
The 2026-08-18 pass: a DEFAULT root, a RECURSIVE search, and a scan that says what it did
Three changes, all C#-only, no ABI. Together they are the prerequisite for a plugin INSTALLER (see the installer section below) rather than features in their own right.
- A default root:
~/.duckdb/fabricator/plugins. Before this the scan RETURNED IMMEDIATELY whenFABRICATOR_PLUGIN_DIRwas unset, so there was nowhere to install to.FABRICATOR_PLUGIN_DIRstill wins, and it REPLACES rather than extends the default — a rig that narrows the search must actually get a narrow search, or it is not testing what it claims.- ⚠ NOT under the managed directory, and that is a MEASURED hazard rather than taste. Several
projects publish into the managed dir and
dotnet publishDELETES files its own previous publish wrote whose closure no longer contains them — that is what silently removed fiveMicrosoft.Data.SqlClientDLLs from a populated payload on 2026-08-18. A plugin installed there would be wiped by an ordinarypublish-managed.ps1run, with no error.~/.duckdbis also DuckDB's own per-user directory (whereINSTALLputs extensions), is writable without admin, and is STABLE while the managed dir is not: that one moves between a build tree,~/.duckdb/extensions/<version>/<platform>/, and the single-file distribution's cache.
- ⚠ NOT under the managed directory, and that is a MEASURED hazard rather than taste. Several
projects publish into the managed dir and
- The search is RECURSIVE. It was
Directory.GetFiles(dir, "*.dll")— top level only — so a plugin laid out the way an installer writes one (<root>/<name>/<version>/<platform>/) was never seen. Candidates are ordered by path, which is not cosmetic: the FIRST provider registered under a name wins, andDirectoryenumeration order is filesystem-dependent, so an unordered scan makes which plugin wins a property of the disk rather than of the configuration. The dependency-probingResolvinghook now gets every directory holding a candidate, not just the roots — a plugin's private deps sit next to it, several levels down. SELECT * FROM fabricator_plugins()— one row per root plus one per candidate, with a status and a reason:root/root_missing/loaded/no_backend/shared/rejected.
⚠ WHY THE DIAGNOSTIC IS THE LOAD-BEARING PART. The scan ends every candidate in a catch, so a plugin
built against a different Apache.Arrow major, or missing a private dependency, was skipped with no signal
at all — and a failing verify_plugin is indistinguishable from "the plugin loaded and chose to register
nothing". Four states used to be one silence, and each now names itself:
| status | means |
|---|---|
root_missing | a configured root does not exist — the most common real cause, previously invisible |
rejected | load or reflection threw; detail carries the exception (e.g. BadImageFormatException) |
no_backend | loaded fine, declares no IBackend — the ordinary state of a plugin's private dependency, and NOT a failure |
shared | skipped because the host already has an assembly of that name — deliberate, so it must be visible |
Gates: verify_plugin 10 -> 17 (service tier), mutation-tested — a scan that records nothing dies at
assertion 11, i.e. after all ten pre-existing plugin assertions pass, which is the right kill because the
plugin still WORKS and only the report is silent. Plus 10 tier-0 cases (PluginPathsTests, floor 196 ->
206) for the two properties SQL structurally cannot reach: the default root is under the real user's home, so
no hermetic suite may create it, and the override precedence is only observable with the variable UNSET —
which verify_plugin must set.
- ⚠ A PLUGIN CAN STILL SHADOW A BUILT-IN PROVIDER SILENTLY — found while building the report, NOT fixed,
and deliberately out of this pass's scope.
BackendRegistry.Addismap[backend.Name] = backend, an OVERWRITE, so a plugin whoseIBackend.Nameissqlserverreplaces the first-party provider and the scan reports it as an ordinaryloadedrow. Pre-existing behaviour (nothing about this pass changed it) and nobody has hit it, but it is exactly the class of silencefabricator_plugins()exists to remove, and the report is the natural place to surface it: thedetailof aloadedrow could name any provider name it DISPLACED. Doing it needs a decision this pass did not want to take — whether shadowing should be reported, refused, or allowed as an override mechanism. - ⚠ No cap on the candidate count, deliberately. A self-contained plugin can carry hundreds of DLLs and
most will be
rejected; a silent truncation would read as "covered everything". Every one gets a row. - ⚠
Environment.GetFolderPath(SpecialFolder.UserProfile)does NOT read%USERPROFILE%on Windows — it calls the Win32 shell API. So the empty-profile trick this repo uses to simulate a bare runner does not redirect the default plugin root.
Installing a plugin — BUILT 2026-08-18 (fabricator_install_plugin)
Steps 2 and 3 of the installer, on top of the scan work above. C#-only, no ABI change. What follows is the as-built record; the sketch it replaced is kept below it because every prediction in it held.
What shipped
BackendRegistry.Invalidate()— drops the memoized provider map so the next resolve re-discovers. Ten lines. Everything hard about it is in the three re-scan hazards below.fabricator_install_plugin(archive [, root := …] [, replace := …])— a table function returning ONE row:name,version,platform,destination,files,providers,activated,detail.fabricator_allow_plugin_install— a BOOLEAN setting, default false, gating the above.- The archive contract: a
fabricator-plugin.jsonmanifest at the root plusany/and/or<duckdb platform>/, merged with the platform overlayingany/.
THE THREE RE-SCAN HAZARDS — this is the part a naive Invalidate() gets wrong
Map() was _byName ??= Discover() and nothing ever cleared it, so the scan had never run twice in one
process. Three things silently depended on that, and none of them fails loudly:
- The "shared" skip would have DROPPED every plugin on the second scan. An assembly cannot be unloaded,
so on re-scan the plugins loaded by the first are in
host.Assemblies— they match the already-loaded-by-the-host skip set, get reportedshared, and are never registered into the FRESH map. Fixed by subtracting what we ourselves loaded from a plugin directory (BackendRegistry.PluginLoaded);LoadFromAssemblyPaththen returns the already-loaded instance and the provider goes back in. A plugin whose FILES were deleted simply stops being a candidate and drops out — the right answer for an uninstall. - The dependency resolver CAPTURED its probe directories. Correct while the scan ran once; wrong the
moment a plugin can be installed mid-session, because the new plugin's directory is not in the captured
array and its private dependencies would not resolve — surfacing as
rejectedwith aFileNotFoundExceptionnaming a dependency sitting right next to it. The hook is now installed once and reads a field replaced on every scan. _defaultProvideris deliberately NOT cleared. It is set from the first provider discovered, so clearing it would let an install re-derive which provider is the default and silently re-point every call site that carries no provider name. An install adds a provider; it must not move the existing ones.
Existing ATTACHed catalogs are unaffected either way — they hold an already-resolved IBackend and its
catalog object, neither reached through the map.
⚠⚠ THE BUG THIS FOUND IN MY OWN FUNCTION, WHICH IS THE MOST TRANSFERABLE THING HERE
fabricator_install_plugin read the session-scoped opt-in setting inside its async iterator body. That
body runs at the first BATCH PULL — a different ABI crossing from the one that set the ambient, on whatever
thread DuckDB pulls from. AmbientOpener / ProviderSettingsStore.CurrentSession are AsyncLocal per
crossing, so the iterator can legitimately see session 0, which falls back to the GLOBAL settings layer,
where the registration default (false) sits. An enabled function then reports itself disabled.
- It is NON-DETERMINISTIC, and it passed the first time it was run. The same suite refused an install at the THIRD call in one build and the FOURTH in another — the two differing only in an unrelated mutant and a stderr probe. A "works on my run" check would have shipped it.
- It was found by mutation testing, not by review — and not by the mutant it was aimed at. The mutant died at the right place for the WRONG REASON ("disabled" rather than a provider mismatch), and chasing that discrepancy instead of banking the kill is what exposed it. A kill by an unexplained mechanism is not a kill you have understood.
- The fix is the pattern already in the tree: capture the ambients in
Execute()— the plain method, which runs inside the crossing that set them — and re-establish them at the top of the iterator.DeltaGlobalTableFunctiondoes exactly this, with a comment saying why, andBulkSessiondoes it for its background thread. - Standing rule it generalises to: a global table function must read every ambient in
Execute(). By execution time the opener, the transaction and the settings session are all gone or arbitrary.
Decisions worth keeping
- The layout is FIXED, never inferred. A flat archive (assemblies at the root) is REFUSED. The
alternative needs a rule that recognises a platform directory by NAME, under which an archive shipping only
linux_amd64/looks flat on Windows and its Linux binaries get installed — a wrong answer, not a missing feature. - The write is STAGE-THEN-MOVE. Extraction goes to
<root>/.staging/<guid>and the finished directory isDirectory.Moved onto<root>/<name>/<version>: atomic on one volume, so two processes installing one version race on a put-if-absent instead of interleaving their writes. ⚠ Stated precisely because atomicity claims are where this codebase has been burned before: the refusal is EXACT on Windows (MoveFileExwithoutMOVEFILE_REPLACE_EXISTING) and CONDITIONAL on Unix (POSIXrenamefails withENOTEMPTYonly for a NON-EMPTY destination and silently replaces an empty one). It holds here only because a destination is never created any other way than by this same fully-populated rename. Both staging and.trashlive INSIDE the root to keep the move on one volume, andEnumerateCandidatesnow skips any path segment beginning with.so a concurrent scan cannot load a half-extracted plugin. (The ROOT itself may be dotted — the default one is~/.duckdb/....) replace := trueMOVES the old directory aside rather than deleting it. A loaded assembly is locked on Windows: it can be renamed, not removed. Measured —Directory.Moveof a directory containing a loaded assembly succeeds on Windows.- The entry assembly's presence is checked BEFORE the move. An archive that installs cleanly and contains no plugin is the exact "install succeeded, nothing happened" failure the scan report exists to remove.
abstractionsVersionis recorded and NOT gated on. Nothing versionsFabricator.Abstractions— every assembly is 1.0.0.0 — so a comparison would pass always or fail always, i.e. an untestable flag. The real incompatibility already has an honest report: the scan records it asrejectedwith the exception.activatedis read back out of the FRESH scan, so the row distinguishes "installed" from "installed and usable" rather than assuming them equal — and an install into a root nothing scans says SO, in different words from a plugin that was scanned and declared nothing. The two look identical in the report and mean completely different things.- The platform string is asked of DuckDB (
pragma_platform()), never derived fromRuntimeInformation. The spelling is DuckDB's, so deriving it would be a second implementation free to drift from the one the archive was built against. - The gate is OUR setting, not
allow_unsigned_extensions. The latter is nearly always true by the time this extension is loaded at all, so gating on it would gate nothing. Remote URLs are refused outright. - The zip-slip guard is REUSED, not re-written.
Fabricator.Bridgeproject-referencesFabricator.Installer.CoreforArchivePathalone. A security guard is the last thing that should exist twice in one codebase with two chances to drift.
Gates
- Tier 0,
PluginPackageTests+34 (floor 206 → 240): the manifest and the merge. These are the rules an end-to-end suite structurally cannot reach — it installs ONE archive, built on the machine running it, for the platform running it, so "another platform's directory is never taken", "an archive carrying nothing for this platform is refused" and "a manifest naming../..is refused" have no fixture there. Plus twoPluginPathsTestsfor the hidden-segment rule, including that a DOTTED ROOT is still searched (without which the default root would disable discovery out of the box). verify_plugin_install.test(31, service tier), run against its OWN empty plugin root — every assertion in it is of the form "this changed", so with the plugin already loaded the before-state assertions fail and the after-state ones would pass with the install doing nothing at all.- The load-bearing pair: after the install the ATTACH error CHANGES from "unknown provider" to the
plugin's own "global functions only" (nothing but a re-discovery produces that), while
plug_greetis STILL absent — the documented half of the split, pinned so a future "improvement" has to reckon with why it cannot be added. - Mutation-tested, each mutant killed at its own section: removing
Invalidate()dies at the install row (providersempty,activatedfalse) after 9 assertions pass — the files landed, the session did not see them; removing the plugin-loaded subtraction dies at the ATTACH assertion with "unknown provider" after 13 pass, INCLUDING the first install's success, which is the right discrimination since that subtraction only becomes load-bearing on the second scan. - ⚠ Hazard 2 (the resolver's directories) is REASONED, NOT GATED — the sample plugin has no private dependencies, so no mutant of it dies. Say so rather than implying the three are equally covered.
- The load-bearing pair: after the install the ATTACH error CHANGES from "unknown provider" to the
plugin's own "global functions only" (nothing but a re-discovery produces that), while
- ⚠ The archive fixture is emitted by the plugin's OWN build (
PackPluginArchive, MSBuild'sZipDirectory), becausezipis not present in Git Bash on Windows and a fixture that exists on one platform is a gate that runs on one platform. It stages underobj/, notbin/: staging in the output directory put a second copy of the plugin under the now-RECURSIVE scan andverify_pluginduly reported TWO loaded plugins. Measured, not theorised — that is how the line came to be written.
Uninstall, and the shadowing refusal (BUILT 2026-08-18)
fabricator_uninstall_plugin(name [, version := ...] [, root := ...]) MOVES, it does not delete, and that
is the mechanism rather than a limitation. A loaded assembly is locked on Windows and the load context is not
collectible, so a plugin that has been USED cannot be deleted -- but it can be RENAMED. Moving the version
directory into <root>/.trash/<guid> takes it out of the scan immediately (the trash is hidden by its leading
dot), which is the mark-for-deletion the sketch called for, arrived at without inventing a marker file. The
bytes go when they can: a best-effort delete now, plus a SWEEP of the whole trash at the start of the next
install or uninstall -- by which time a restart has usually released the lock.
removedandpurgedare separate columns because they are separate questions. Out of the scan (removed) is the only real failure when false; bytes reclaimed (purged) is ORDINARILY false. One boolean would make the normal outcome look like a failure.- ⚠ The provider stops resolving; the code does not go away. Nothing can unload the assembly. The guarantee is that the re-scan finds no candidate and does not register it -- worth stating rather than implying the plugin is gone.
- The sweep runs at install/uninstall only, never on a read path: sweeping during a scan would put filesystem writes on the ATTACH path.
A plugin may not take a registered provider's name -- REFUSED. BackendRegistry.Add is a plain dictionary
assignment, so a plugin declaring IBackend.Name = "sqlserver" used to REPLACE the first-party provider and be
reported as an ordinary loaded row: every later ATTACH going somewhere the user never chose. Of the three
options, refusing is the only one that cannot end in a wrong ANSWER; an override mechanism, if ever wanted,
should be something the USER asks for by name rather than something a file appearing in a directory can do.
- ⚠ Checked BEFORE anything is added, so an assembly whose SECOND backend collides does not leave its first
half-registered. The throw rides the scan's existing per-candidate handler, so the plugin is
rejectedwith a message naming both sides and every other plugin still loads. Aliases are checked too. - The fixture had to be a second assembly (
dotnet/Fabricator.CollidingPlugin): a name collision needs two assemblies claiming one name, and nothing in a manifest, a root or an install argument can manufacture that. It could not live insideFabricator.SamplePlugineither, since the refusal is all-or-nothing per assembly.
What is still NOT built
- Signature or checksum verification of an archive.
Hashingis there; nothing consumes it here. - Per-plugin ALC isolation -- see the sequenced recommendation below. Still correctly deferred.
The FLUID plugin — the first plugin with a PRIVATE PACKAGE CLOSURE (2026-09-01)
fabricator_render(template, params), the Fluid/Liquid template engine, was a first-party global scalar
registered by Fabricator.SqlServer. It is now dotnet/Fabricator.FluidPlugin, a plugin. The move is
user-directed and the mechanism needed no change to the bridge, the ABI or the plugin system — a plugin
contributes global scalars through the same IBackend.GlobalScalarFunctions a backend does.
Why it was worth moving. A template engine inside the SQL Server backend has nothing to do with SQL
Server, and its Fluid.Core closure rode into every shipped payload whether or not anyone rendered a
template. MEASURED: Fluid.dll, Parlot.dll and TimeZoneConverter.dll are gone from
build/release/extension/fabricator/fabricator after the move. It is also the one dependency
the AOT SKU had to reason about — Parlot's compiled mode uses System.Linq.Expressions —
so that conditional dissolves rather than being solved.
⚠ CONSEQUENCE: fabricator_render is registered by a plugin now. A global function can only be
registered during Extension::Load(), so the plugin must be in a plugin root at load time — installing it
mid-session surfaces it only at the next start.
⚠ SINCE 2026-09-01 IT ALSO SHIPS A GLOBAL SQLGEN FUNCTION — fluid_replacement_query(template [, params := …]) —
and that is a SECOND registration path, not a second function. fabricator_render arrives through
IBackend.GlobalScalarFunctions and becomes a DuckDB scalar; fluid_replacement_query arrives through
IBackend.GlobalSqlTableFunctions and becomes a bind_replace TABLE function whose call disappears at bind.
This is the first plugin in the tree to use the latter at all, so it is the first evidence that the
plugin scan carries a provider's sqlgen declarations as well as its scalars — which is why the distribution
smoke gained a check for it rather than trusting the render one to cover both. Full record:
fluid-templating.md §7.
It IS in the distribution artifact, via the bundled root below — so a user on a released binary keeps
fabricator_render with no configuration. ⚠ But a user who sets FABRICATOR_PLUGIN_DIR loses it, because
that variable REPLACES every default root rather than extending them.
The BUNDLED root — how a plugin ships at all
The single-file payload is exactly the core loadable plus the managed directory, so a plugin ships only by
living inside the managed directory. PluginPaths.BundledRelativeRoot (plugins) makes
<managed>/plugins/ a default search root, and pack-distribution.ps1 step 2b builds each bundled
plugin and copies its assemblies to <managed>/plugins/<name>/.
- ⚠ Step 2b MUST run after
publish-managed.ps1and before the pack.dotnet publishdeletes files under the managed directory that a previous publish wrote and the current closure no longer contains — the mechanism that silently removed five SqlClient DLLs on 2026-08-18 — so a plugin copied in earlier would simply not be in the artifact, with no error anywhere. - ⚠ This does NOT contradict the rule that the default root is not under the managed directory. That rule protects a plugin the USER installed, which a publish would destroy. A bundled plugin is part of the artifact and is rewritten by every pack, so being wiped by a publish is correct rather than data loss. The distinction is ownership, not location.
- ⚠⚠ The bundled root is searched LAST, and the first version of this shipped with the ordering — and the
justification — BACKWARDS. The scan is FIRST-ROOT-WINS: it registers with
refuseCollisions: true, so a duplicate provider name met in a later root is reportedrejectedwith a collision message naming both, never overwritten. The original comment justified bundled-FIRST byBackendRegistry.Addbeingmap[name] = backend(last-wins) — true of the BUILT-IN registration path, false of the plugin path. Under that ordering the shipped copy won and a user's install was rejected, the opposite of the intent. MEASURED with two roots holding one plugin: the first loads, the second is rejected. Gated offline (PluginPathsTests.Bundled_root_sits_under_the_managed_directory_and_is_searched_LAST), mutation-tested: reversing the two kills that test and nothing else. ⚠ The consequence to expect in normal use: a user who installs their own copy of a bundled plugin sees arejectedrow for the shipped one. That is the honest report, not a fault. - ⚠
FABRICATOR_PLUGIN_DIRreplaces it too, deliberately: the hermetic tier points that variable at an empty directory precisely so its plugin set is provably independent of machine state, and a bundled root that survived the override would make a tier's result depend on whether anyone had run a pack into that build tree. The cost is the footgun above, which is why every root is REPORTED byfabricator_plugins().- An EXTEND spelling (
FABRICATOR_PLUGIN_DIR_EXTEND=1) was proposed and DECLINED (2026-09-01, user-decided) — recorded so it is not re-proposed as an obvious ergonomic win. The motivation was dev/test: point the variable at a plugin under development without losing the bundled ones. The root-ordering fix above removes that pain:~/.duckdb/fabricator/pluginsis searched BEFORE the bundled root, so dropping a build there wins over a shipped copy of the same plugin while every other bundled plugin stays present — no variable needed. What extend would have cost is the hermetic tier's exclusivity guarantee, and a,-separated list already expresses "mine plus the bundled root" explicitly for anyone who wants it.
- An EXTEND spelling (
- The managed directory is derived from the bridge assembly's own location, not from
FABRICATOR_MANAGED_DIR— that variable is an INPUT toclr_hostand is absent whenever the host used its default, which is the case the distribution takes. - Gate:
test/distribution/smoke_distribution.py(12 → 14 checks). ⚠ It attributes by root, not by the function merely working: that session sets no environment variables, so the per-user root is searched too and a developer with Fluid installed there would make a bare "does render work" check pass on an artifact shipping nothing.
⚠⚠ What this plugin tests that Fabricator.SamplePlugin cannot
Fabricator.SamplePlugin is pure IL with no third-party package. So until this existed, nothing exercised
BackendRegistry.InstallPluginResolver actually loading a plugin's own NuGet closure out of the plugin
folder — the resolver had a test for the case where there is nothing to resolve. Fluid pulls six
assemblies (Fluid, Parlot, TimeZoneConverter and two Microsoft.Extensions.*), all resolved by that
hook, and since the move none of them is in the bridge payload — so any successful render IS the closure
resolving. (The exact list moved with the pin: Fluid, Parlot, TimeZoneConverter, System.Linq.Async
and two Microsoft.Extensions.* on 3.0.0-beta.7.)
⚠ The Fluid pin is a PRERELEASE — Fluid.Core 3.0.0-beta.7, because v3.0 has no stable package. That
makes a bump here a CODE-COMPATIBILITY question rather than a routine version bump, and
verify_plugin_fluid.test is what answers it. Moving 2.31.0 → 3.0.0-beta.7 already changed an annotation
under us: v3 declares TemplateContext.SetValue(string, object) NON-nullable while its body still maps null
to NilValue.Instance. Nothing broke — but the plugin now routes null to the FluidValue overload itself
rather than depending on that internal branch, precisely because an internal null-handling branch is the kind
of thing that moves between betas and moves SILENTLY. ⚠ The compiler warning is what found it, and the
suite had no assertion either way — a NULL is ordinary here (a STRUCT field can be NULL, JSON has null),
so three assertions were added for it. Go to the stable 3.0.0 as soon as it ships.
⚠ It stays a PackageReference rather than a source reference to a local clone: a sibling-path reference
pins nothing, no clone but the author's can build it — the shape this repo converted engineered-wood and
DuckDB.ExtensionKit away from — and it would leave CI silently gating a DIFFERENT Fluid than the developer
runs.
⚠ The trap the build walked into, which every future plugin with a dependency will meet
A library does not copy its NuGet closure to the output directory. CopyLocalLockFileAssemblies defaults
to false for libraries; package assemblies are materialised only by dotnet publish. Fabricator.SqlServer
got Fluid that way, through publish-managed.ps1. A plugin has no publish step — its build OUTPUT is
what a plugin root points at — so the first build produced Fabricator.FluidPlugin.dll and nothing else, and
the plugin would have loaded and then failed at first render with a FileNotFoundException naming an
assembly nobody had copied. Set CopyLocalLockFileAssemblies=true.
And then the opposite hazard arrives. With it set, Apache.Arrow — which reaches the plugin transitively
through Fabricator.Abstractions — gets copied too, which is exactly the "aligned dependency closure" hazard
this document warns about elsewhere. ExcludeAssets="runtime" on an explicit Apache.Arrow /
Apache.Arrow.Scalars reference keeps the compile reference and drops the copy. ⚠ ExcludeAssets does NOT
flow to a transitive dependency, so Apache.Arrow.Scalars has to be named separately.
⚠ It references Abstractions + Common, and NEVER Bridge
⚠ CORRECTED 2026-09-02. This section used to read "it references Abstractions ONLY, and paid ~20 lines
for it": ArrowValueReader.ReadScalar lived in Fabricator.Bridge, IScalarFunction's doc said it was
available "if a provider references the bridge", and the plugin therefore carried a local copy
(ArrowScalar.Read). That is the gap Fabricator.Common closed — the reader is in Common now, which a
plugin may reference without taking on the host's Azure/Fabric/unsafe closure
(docs/plugin-services.md §9).
What did NOT change is the rule the old wording was defending: the plugin still references NO Bridge, and
that is the acceptance test for the split rather than a stylistic preference. What became of the ~20 lines is
more interesting than "they were deleted": the local reader had GROWN into FluidValueModel.ReadCell, a
deliberate superset with Fluid-specific float/blob/date handling, so it stays. The copy that actually went is
ReadTimestamp, which was character-for-character identical to the bridge's including the two (object)
casts that fix a four-month defect (§9.4). The duplicate worth removing was the small invisible one, not
the big obvious one.
Gates
test/verify_plugin_fluid.test (20, service tier). The runner points FABRICATOR_PLUGIN_DIR at
build/plugins/fluid and nothing else for that one suite, because two of its assertions — exactly one
loaded provider, exactly one fabricator_render registration — say nothing with the tier's normal root
(holding Fabricator.SamplePlugin) also in scope. The single-registration one is what pins the MOVE rather
than the behaviour: a build that had merely ADDED the plugin while leaving the first-party copy in place
would pass every render assertion.
⚠ Two things moved rather than being deleted, and verify_global_functions says so at both sites: the nine
render assertions, and the untyped-NULL-in-an-ANY-declared-position regression — fabricator_render's
params was the only ANY-declared non-varargs global scalar parameter in the tree, so that crossing was
re-homed onto the VARARGS tail (fabricator_va_concat('-', 1, NULL, 'x')), MEASURED to cross identically
before the substitution was made. verify_plugin.test also had to stop using fabricator_render for its
"a built-in global coexists with a plugin's" assertion — which stopped meaning anything the day render
itself became a plugin.
A plugin can declare a CORRELATED LATERAL function (2026-08-22, ABI v79)
ILateralFunction lives in Fabricator.Abstractions, so declaring one costs a plugin no more than
declaring a scalar — and it is the function kind a plugin most often wants, because its callee is typically a
REST or model call and the whole point is that the WHOLE INPUT CHUNK crosses in one call:
SELECT t.id, r.* FROM t, my_plugin_fn(t.a, t.b); -- one call per ~2048 outer rows, not per row
The sample plugin ships plug_lat_slow(n, millis), which sleeps once per call. That is not a demo, it is
the INSTRUMENT: a per-call cost is the only thing batching can amortise, so verify_plugin can assert the win
as a ratio between two legs of one statement rather than quoting a remembered number — measured 0.870 s
row-by-row vs 0.154 s batched for 8 distinct outer rows at 100 ms, with max(batch_rows) (1 vs 8) as the
mechanism assertion. Full record: lateral_unnest_analysis.md §8.
⚠ Two constraints a plugin author needs before designing one. A named argument cannot be used in the
CORRELATED shape (a DuckDB limitation — duckdb-upstream-issues.md §5), so
declare a per-call constant POSITIONALLY. And the correlated values are DE-DUPLICATED before the call:
DuckDB puts a DISTINCT under it and re-expands by joining above, so cost scales with distinct argument
tuples rather than with outer rows — which is usually good news for a network callee, and is the reason a
performance estimate based on row count will be wrong.
HTTP from a plugin — use the host's transport, once you can reach it
A plugin whose backend is a REST API should not carry its own TLS trust, proxy configuration or retry
policy: DuckDbHttpHandler (ABI v76) is an ordinary .NET HttpMessageHandler whose transport is DuckDB's
own HTTP stack, so a call inherits the TYPE http secret whose SCOPE covers the URL, ca_cert_file,
http_proxy*, http_timeout and the retry knobs. Full record: http-transport.md.
⚠ Two things to know before planning around it. The class lives in Fabricator.Bridge, and a plugin
normally references only Fabricator.Abstractions — so it is not on a plugin's compile-time surface yet;
how the transport is HANDED to a plugin is the open question in that doc's §6. And DuckDB's TYPE http
secret carries a static credential only (BEARER_TOKEN / EXTRA_HTTP_HEADERS); an OAuth2
client-credentials API still needs its own secret type and its own token exchange, so it gains the
transport half and not the credential half.
The original sketch — kept because every prediction in it held
The shape agreed 2026-08-18: a zip carrying any/ (platform-independent) and <platform>/ folders named with
DuckDB's own platform strings (windows_amd64, linux_amd64, osx_arm64 — the extension already knows its
own), plus a manifest declaring name, version, entry assembly and the Fabricator.Abstractions version it was
built against; fabricator_install_plugin(<zip>) extracts it under the default root.
Most of the machinery exists: Fabricator.Installer.Core is the same problem solved for the extension
itself — PayloadExtractor already does zip extraction with a working zip-slip guard, PayloadManifest
already carries the platform string, and there is a CrossProcessLock and Hashing. It is BCL-only and
tier-0 tested.
⚠ THE RELOAD QUESTION SPLITS, and only one half is a problem — this is what should drive the design:
| a plugin contributes | resolved when | addable mid-session |
|---|---|---|
IBackend (ATTACH ... PROVIDER 'x') | at ATTACH, via BackendRegistry.Resolve | yes — needs only an invalidation of the memoized map |
| catalog-bound functions | at ATTACH, via that catalog | yes — rides on the above |
| global functions | loader.RegisterFunction during Extension::Load() | no, by no trick |
DuckDB permits global registration only during extension load, has no unload API at all, and re-LOAD of
a loaded extension is a no-op. GlobalFunctions' maps are Lazy<> besides — evaluated once per PROCESS — so
even a second database instance would miss a newly installed plugin's globals.
⚠ Upgrade and uninstall are the hard part. LoadFromAssemblyPath maps the file, which LOCKS it on
Windows, and the bridge's ALC (created by hostfxr) is not collectible — so a loaded assembly can never be
replaced in-process. That forces the UX: install into a version-stamped folder and activate at next start;
uninstall must mark for deletion rather than delete.
Security, stated once: this is arbitrary in-process .NET execution from a SQL-reachable path, unsandboxed.
DuckDB gates its own unsigned extensions behind allow_unsigned_extensions; an installer should do at least
the same, refuse remote URLs initially, and never auto-install.
Why / when
Today the bridge loads providers (Fabricator.SqlServer, Fabricator.AnalysisServices) by reflection into the
default ALC (ProviderRegistry, a Fabricator*.dll glob of the managed directory since 2026-09-02, or
the FABRICATOR_BACKEND_ASSEMBLY override). They're all version-aligned with the
bridge, so they share one of everything — fine. A plugin system adds value only when plugins have genuinely
conflicting managed deps (e.g. two providers needing different Azure SDK / Newtonsoft versions) or are
third-party (you don't control their dependency graph). ALC isolation gives each plugin its own private
dependency closure while still speaking the shared contract.
It works on our host: the bridge runs on CoreCLR (hostfxr, self-contained .NET 10), where AssemblyLoadContext
AssemblyDependencyResolverare the standard mechanism. The two classic limitations don't bite us — we are not Native AOT (ALC exists) and not loading .NET Framework 4.x assemblies (CoreCLR only).
The crux — Apache.Arrow MUST be shared (the whole boundary hinges on this)
Every cross-boundary call traffics Apache.Arrow types: IScalarFunction.Parameters → Schema,
Invoke(RecordBatch) → IArrowArray, ITableFunctionBinding.Execute → IArrowArrayStream, and the bridge's
C-ABI marshaling (CArrowArrayStreamExporter/Importer, CArrowSchemaExporter) all operate on Apache.Arrow
types. Types from different ALCs are not assignable. So if a plugin loaded its own Apache.Arrow, its
RecordBatch would be a different type than the bridge's and every Invoke/Bind/export would throw
InvalidCastException (or hand the exporter a foreign object).
Therefore:
Apache.Arrow+Apache.Arrow.Care contract surface — loaded once, in the default context, shared by the bridge and every plugin.- Hard constraint: every plugin pins the same Apache.Arrow version as the bridge (today 23.0.0). Isolation buys plugins freedom for their other managed deps (SqlClient, ADOMD, Fluid, Azure.Identity, engineered-wood, JSON, …) but never for Arrow. (engineered-wood works in a plugin ALC precisely because it is already Arrow-23-aligned; a plugin needing a different Arrow for some private lib simply cannot.)
The shared boundary for fabricator
Extract a thin Fabricator.Abstractions assembly = the interfaces + the Arrow-typed contract POCOs
(IBackend, IBackendCatalog, IScalarFunction/ICatalog*, ITableFunction/ITableFunctionBinding,
IInOutFunction, ICollectorFunction, IAggregateFunction/IAggregateState/IAggregateSession,
ProviderSetting, SecretField, TableFunctionScan, ScanSpec, FilterNode, ITableFunctionSession, …). Shared
(default context). Fabricator.Bridge references it and keeps the ABI/marshaling/Bootstrap/GlobalFunctions/
BackendRegistry (also default context — it's the hostfxr entry assembly). A plugin references only
Fabricator.Abstractions + Apache.Arrow (both host-provided, NOT copied into the plugin dir) + its own private
deps.
Why a separate Abstractions rather than "bridge = contracts": plugins should bind to a minimal, stable
contract surface, not the ABI internals (Bootstrap's [UnmanagedCallersOnly] exports, the marshaling). It
also guarantees every type in a contract signature is shared (Abstractions / Apache.Arrow / BCL) — no contract
method exposes a plugin-private type, which would otherwise force that dependency to be shared too.
The complete shared set (returned as null from a plugin's Load, i.e. resolved from the default context):
Fabricator.Abstractions, Apache.Arrow, Apache.Arrow.C, and the BCL/System.* (the runtime shares framework
assemblies automatically).
PluginLoadContext — the one correction over the textbook sketch
The standard sketch returns null from Load only when the resolver misses. That is insufficient here:
AssemblyDependencyResolver will succeed for Apache.Arrow (it's in the plugin's deps.json), so it would
load an isolated Arrow copy and break everything. You must short-circuit the shared set to null before
consulting the resolver:
private static readonly HashSet<string> Shared = new(StringComparer.OrdinalIgnoreCase)
{
"Fabricator.Abstractions", "Apache.Arrow", "Apache.Arrow.C",
// BCL/System.* are shared by the runtime automatically.
};
public sealed class PluginLoadContext : AssemblyLoadContext
{
private readonly AssemblyDependencyResolver _resolver;
public PluginLoadContext(string pluginPath) : base(isCollectible: false) // we never unload (see Lifetime)
=> _resolver = new AssemblyDependencyResolver(pluginPath);
protected override Assembly? Load(AssemblyName name)
{
if (Shared.Contains(name.Name!)) return null; // force fall-through to AssemblyLoadContext.Default
var path = _resolver.ResolveAssemblyToPath(name);
return path != null ? LoadFromAssemblyPath(path) : null; // else plugin-private
}
protected override IntPtr LoadUnmanagedDll(string name)
{
var p = _resolver.ResolveUnmanagedDllToPath(name);
return p != null ? LoadUnmanagedDllFromPath(p) : IntPtr.Zero;
}
}
null → the runtime falls back to AssemblyLoadContext.Default, where the bridge already loaded the shared set
— so typeof(IBackend).IsAssignableFrom(pluginType) resolves to the same IBackend and the cast works (the
reason the contracts must be shared). Plugins build the shared refs with <Private>false</Private> /
ExcludeAssets so they aren't copied into the plugin folder (the host provides them).
Lifetime — non-collectible (no unload machinery)
Global functions register at Extension::Load and live for the process; BackendRegistry / GlobalFunctions
(static, default context) hold the plugin objects, which pins the plugin ALCs alive regardless. So use
isCollectible: false — simpler and faster, and it skips the WeakReference<AssemblyLoadContext> + unload
discipline (and the restrictions collectible ALCs impose). We never unload a plugin.
Integration (additive to BackendRegistry)
- First-party providers (SqlServer, DAX) stay in the default context — version-aligned with the bridge, so isolation buys them nothing.
- Add a plugin-dir scan: for each plugin folder → a
PluginLoadContext→ load its entry assembly → findIBackendtypes (theirIBackendresolves to the default-context one, soIsAssignableFromworks) → instantiate →BackendRegistry.Register. TheirGlobalScalarFunctions/GlobalInOutFunctions/ … get unioned byGlobalFunctionsexactly like first-party ones — no change to the global-function machinery, since it only ever touches shared (Arrow / Abstractions) types. - Timing: the scan must run at bridge init, before the first
list_global_functions(which lazily unionsBackendRegistry.All()). Slot it intoBootstrap.Initialize/ first registry access. - Static state stays in the default context (
BackendRegistry,GlobalFunctions,ProviderSettingsStore,Handles,AmbientTransaction) — one process-wide instance the plugin objects register into. Correct (one registry, one handle table); it also means these are contract surface (already in the bridge/Abstractions).
Gotchas
- Native deps aren't ALC-isolated.
ResolveUnmanagedDllToPathresolves a plugin's native libs per ALC, but the OS loads a native DLL once per process — two plugins needing different native versions of the same library still collide (e.g. SqlClient's native SNI on Windows). ALC cleanly isolates managed conflicts only; flag native conflicts as out of scope. - One Arrow version, forever-pinned (restated because it's the whole ballgame): bumping the bridge's Apache.Arrow is a coordinated change across all plugins.
- Reflection across ALCs works only through the shared contracts — a plugin type is matched via the
default-context
IBackend; never reflect over a plugin's private types from the host.
Recommendation (sequenced)
-
Default-context plugin-dir loader — DONE (this build):
FABRICATOR_PLUGIN_DIRscan inBackendRegistry, plugins loaded into the bridge's ALC (not Default — see As-built), additive beside the env-assembly discovery. Plugins referenceFabricator.Bridgedirectly (noAbstractionsneeded without ALC — everything is one context). Sample plugin +verify_plugin.test. Plugins must align their full dependency closure with the host (Apache.Arrow always; every other shared dep too — there is no version isolation without ALC). -
Extract
Fabricator.Abstractions— DONE — the contract surface (theI*Function/IBackend/ITableFunctionSession/IAggregateSessioninterfaces +ProviderSetting/SecretField/TableFunctionScan/ScanSpec/FilterNode) is now a separate assembly, kept in theFabricator.Bridgenamespace (assembly split only — zero source churn).Fabricator.Bridgereferences it (the ABI/marshaling/Bootstrap/BackendRegistry/Static-bases/ adapters stay in Bridge); theBackendRegistry,InOutExchangeStream/InOutExchange, andCollectorInOutBindingimpls split back out of their old interface files into Bridge.Fabricator.SamplePluginnow referencesFabricator.AbstractionsONLY (+ Apache.Arrow, host-provided) — a lean, Bridge-independent plugin surface (its plugin folder is justFabricator.Abstractions.dll+ the plugin dll). Behavior-preserving; fullverify_*suite +verify_plugingreen. -
ALC isolation (deferred) — a loader-internal swap (
host.LoadFromAssemblyPath→ a per-pluginPluginLoadContext) with the shared-name allowlistLoadabove (non-collectible). Adopt only when a real dependency conflict / a third-party plugin with conflicting managed deps lands — version-aligned plugins gain nothing and pay the cost (per-plugindeps.json, the allowlist, the "don't copy the shared set" build config, the native-dep caveat). The contract + the plugin packaging do NOT change when isolation is turned on. -
The scan report, a default root and a recursive search — DONE (2026-08-18):
fabricator_plugins(),~/.duckdb/fabricator/plugins, and a search that reaches the nested layout an installer writes. See the 2026-08-18 section at the top. -
The installer — DONE (2026-08-18):
BackendRegistry.Invalidate()+fabricator_install_plugin()+ thefabricator_allow_plugin_installgate. See the Installing-a-plugin section. -
Uninstall (NOT built) — needs mark-for-deletion semantics, because a loaded assembly cannot be removed while the process lives, plus a decision on what a half-removed plugin looks like to the scan.
Net: the SPI is built and works on our CoreCLR host without ALC — third-party plugins contribute backends +
global functions today, provided they align their dependency closure with the host (Apache.Arrow always). ALC
isolation is a non-breaking later upgrade to the loader, worth turning on only when a genuine dep conflict
appears; the must-fix for that day is the explicit shared-name allowlist in Load (the resolver would otherwise
isolate Apache.Arrow and break every Arrow-typed call).
Appendix — the CLAUDE.md entry, moved verbatim (2026-08-23)
The working record of the scan/report/installer/uninstall work, in the order it happened, with the reasoning and the traps.
CLAUDE.mdkeeps the standing rules (the ambient-in-iterator bug, the three re-scan hazards, the default root's location, candidate ordering).
- THE PLUGIN SCAN STOPPED BEING SILENT, GAINED A DEFAULT ROOT, AND SEARCHES RECURSIVELY — BUILT
2026-08-18 (C#-only, no ABI). Step 1 of a plugin INSTALLER the user is designing; the installer itself is
NOT built. Full record + the installer sketch: docs/plugin-system.md.
- Three changes. (a) A DEFAULT root
~/.duckdb/fabricator/plugins— the scan used to RETURN IMMEDIATELY whenFABRICATOR_PLUGIN_DIRwas unset, so there was nowhere to install to;FABRICATOR_PLUGIN_DIRstill wins and REPLACES rather than extends it. (b) The search is RECURSIVE — it wasDirectory.GetFiles(dir, "*.dll"), TOP LEVEL ONLY, so a plugin laid out the way an installer writes one (<root>/<name>/<version>/<platform>/) was never seen. (c)SELECT * FROM fabricator_plugins()— one row per root plus one per candidate, with a status and a reason. - ⚠⚠ THE DIAGNOSTIC IS THE LOAD-BEARING PART, and the reason is a property this file already
records:
ScanPluginDirectoriesends every candidate in acatch, so a plugin built against a differentApache.Arrowmajor or missing a private dep was skipped with NO SIGNAL — and a failingverify_pluginis indistinguishable from "it loaded and registered nothing". FOUR distinct states were one silence, each now named:root_missing(a configured root does not exist — the commonest real cause, and previously the scan FILTERED such roots away),rejected(+ the exception message),no_backend(loaded, declares noIBackend— the ordinary state of a plugin's private DEPENDENCY, so it must not read as failure),shared(skipped because the host has that assembly — deliberate, so it must be visible). Verified live: a garbage*.dllbeside a working plugin now reportsBadImageFormatExceptionand the plugin still loads. - ⚠ THE DEFAULT ROOT IS DELIBERATELY NOT UNDER THE MANAGED DIR — a MEASURED hazard, not taste.
dotnet publishdeletes files its own previous publish wrote whose closure no longer contains them, which is what silently removed five SqlClient DLLs hours earlier the same day; a plugin installed there would be wiped by an ordinarypublish-managed.ps1run with no error.~/.duckdbis DuckDB's own per-user dir, is writable without admin, and is STABLE while the managed dir is not (build tree vs~/.duckdb/extensions/<v>/<platform>/vs the single-file cache vsFABRICATOR_MANAGED_DIR). - ⚠ CANDIDATE ORDER IS LOAD-BEARING: the FIRST provider registered under a name wins and
Directoryenumeration order is filesystem-dependent, so the candidates are sorted by path — otherwise which plugin wins is a property of the disk rather than of the configuration. And no cap on the count, deliberately: a self-contained plugin can carry hundreds of DLLs, mostrejected, and a silent truncation would read as "covered everything". - ⚠
Environment.GetFolderPath(SpecialFolder.UserProfile)DOES NOT READ%USERPROFILE%on Windows — it calls the Win32 shell API. So the empty-profile trick this file records for simulating a bare runner does NOT redirect the default plugin root; a test that relies on it is measuring the developer's real home. - ⚠ GLOBAL FUNCTIONS ARE BACKEND-CONTRIBUTED, so a diagnostic ABOUT the provider machinery had nowhere
to live.
GlobalFunctions.BuildwalksBackendRegistry.All(); a question like "which providers were found, and why not this one" must still be answerable when the answer is "none loaded". NewHostGlobalFunctionsis merged FIRST into the map, so a provider declaring a colliding name trips the existing duplicate-name error instead of silently shadowing a diagnostic. - THE RELOAD QUESTION SPLITS, and this is what should drive the installer design (established from
source, not assumed): an
IBackendprovider and its catalog-bound functions resolve at ATTACH viaBackendRegistry.Resolve-> the memoized_byName, so install-and-use in ONE session needs only an invalidation (there is none today). Global functions cannot be added mid-session by any trick —loader.RegisterFunctionis permitted only duringExtension::Load(), DuckDB has NO unload API at all (grepped), re-LOADis a no-op, andGlobalFunctions' maps areLazy<>per PROCESS so even a second database instance would miss them. - ⚠ UPGRADE/UNINSTALL IS THE HARD PART:
LoadFromAssemblyPathmaps the file, which LOCKS it on Windows, and the bridge's ALC (hostfxr-created) is not collectible — so a loaded assembly can never be replaced in-process. Install must write a version-stamped folder and activate at next start; uninstall must mark-for-deletion. ReuseFabricator.Installer.Core:PayloadExtractoralready does zip extraction with a working zip-slip guard,PayloadManifestalready carries DuckDB's platform string, and there is aCrossProcessLock+Hashing— BCL-only and tier-0 tested. - ⚠⚠ THE DEFAULT ROOT BROKE THE HERMETIC TIER'S DEFINING PROPERTY, AND I CAUGHT IT ONLY BY ASKING
WHAT AN UNSET VARIABLE NOW MEANS.
run-suites.shUNSETFABRICATOR_PLUGIN_DIRfor the hermetic tier — which, the moment a default root exists, means "scan the developer's home", i.e. MACHINE STATE, in a tier whose own comment says the clearing exists so the set is "PROVABLY hermetic rather than hermetic by assumption". A plugin contributing a global function changes whatduckdb_functions()returns, which several suites count. It now points at an EMPTYmktemp -d(trap-cleaned) instead — which is exactly what the replace-not-extend precedence buys.- MEASURED, with a true A/B rather than reasoned: a plugin placed in the REAL
~/.duckdb/fabricator/plugins, same binary run twice, only the variable differing — unset ⇒ 2 plugin functions visible induckdb_functions(); empty dir ⇒ 0. (Home directory cleaned up afterwards.) The tier passing at 7646 alone would have proved nothing: it is equally true of a machine with no plugins.
- MEASURED, with a true A/B rather than reasoned: a plugin placed in the REAL
- ⚠ A PLUGIN CAN STILL SHADOW A BUILT-IN PROVIDER SILENTLY — found while building the report, NOT
fixed.
BackendRegistry.Addismap[backend.Name] = backend, an OVERWRITE, so a plugin whoseIBackend.NameissqlserverREPLACES the first-party provider and the scan reports it as an ordinaryloadedrow. Pre-existing (nothing in this pass changed it) and nobody has hit it, but it is the same class of silence the report exists to remove, and the report is the natural place: aloadedrow'sdetailcould name any provider it DISPLACED. Left alone because it needs a decision this pass did not want to take — report, refuse, or allow as a deliberate override mechanism. - Gates:
verify_plugin10 -> 17 (service tier, floor 2028 -> 2035), mutation-tested — a scan that records nothing dies at assertion 11, i.e. AFTER all ten pre-existing plugin assertions pass, which is the right kill (the plugin still works; only the report is silent). Plus tier-0PluginPathsTests+10 (floor 196 -> 206) for the two properties SQL structurally cannot reach: the default root is under the REAL user's home so no hermetic suite may create it, and the override precedence is only observable with the variable UNSET — whichverify_pluginmust set. Tiers: hermetic 70/70 — 7646 (unchanged, correct:verify_pluginis service-tier) and service 50/50 — 2035. - STEPS 2 AND 3 ARE BUILT — 2026-08-18, C#-only, no ABI.
BackendRegistry.Invalidate()+fabricator_install_plugin(archive [, root := …] [, replace := …])+ thefabricator_allow_plugin_installsetting. Full as-built: docs/plugin-system.md §Installing a plugin. A plugin installed mid-session becomes usable as a PROVIDER immediately; its GLOBAL functions still appear only at next start, and the suite pins BOTH halves.- ⚠⚠ THE MOST VALUABLE THING HERE IS A BUG IT FOUND IN THE NEW FUNCTION ITSELF, and the way it was
found: a mutant died at the RIGHT PLACE for the WRONG REASON, and chasing that instead of banking the
kill is what exposed it.
fabricator_install_pluginread the session-scoped opt-in INSIDE its async iterator body — which runs at the first BATCH PULL, a different ABI crossing from the one that set the ambient, on whatever thread DuckDB pulls from.AmbientOpener/ProviderSettingsStore.CurrentSessionareAsyncLocalPER CROSSING, so the iterator legitimately saw session 0, which falls back to the GLOBAL settings layer where the registration defaultfalsesits ⇒ an enabled function reported itself disabled. NON-DETERMINISTIC (the same suite refused at the THIRD call in one build and the FOURTH in another) and it passed the first time it was run.- Fixed with the pattern already in the tree: capture the ambients in
Execute()— the plain method, which runs inside the crossing that set them — and re-establish them at the top of the iterator.DeltaGlobalTableFunctiondoes exactly this with a comment saying why;BulkSessiondoes it for its background thread. - STANDING RULE: a global table function must read EVERY ambient in
Execute(), never in the iterator. By execution time the opener, the transaction and the settings session are all gone or arbitrary.
- Fixed with the pattern already in the tree: capture the ambients in
- ⚠⚠ A NAIVE
Invalidate()IS WRONG IN THREE WAYS, because the scan had NEVER run twice in one process and three things silently depended on that. (1) The "shared" skip would have dropped every plugin on the second scan — an assembly cannot be unloaded, so the plugins loaded by the first scan are inhost.Assemblies, match the skip set, get reportedshared, and never re-register into the FRESH map; fixed by subtracting what we ourselves loaded (PluginLoaded). A plugin whose FILES were deleted then simply stops being a candidate, which is the right answer for an uninstall. (2) The dependency resolver CAPTURED its probe directories, so a plugin installed mid-session would fail to resolve its private deps and be reportedrejectedwith aFileNotFoundExceptionnaming a file sitting next to it; the hook is now installed once and reads a field replaced per scan. (3)_defaultProvideris deliberately NOT cleared — it is set from the first provider discovered, so clearing it would let an install silently re-point every call site that carries no provider name. - The gate is OUR OWN setting, not
allow_unsigned_extensions— that one is nearly always true by the time this extension is loaded at all, so gating on it would gate nothing. NewHostSettingsmirrorsHostGlobalFunctions: settings the HOST declares, crossing under the pseudo-provider namefabricator, which needs no C++ change because the host round-trips the provider column opaquely. - THE LAYOUT IS FIXED, NEVER INFERRED:
fabricator-plugin.jsonat the archive root plusany/and/or<duckdb platform>/, merged with the platform overlayingany/. A FLAT archive is REFUSED — accepting it needs a rule that recognises a platform directory by NAME, under which an archive shipping onlylinux_amd64/looks flat on Windows and its Linux binaries get installed. A wrong answer, not a missing feature. - The write is STAGE-THEN-MOVE: extract to
<root>/.staging/<guid>, thenDirectory.Moveonto<root>/<name>/<version>— atomic on one volume, so concurrent installers race on a put-if-absent instead of interleaving. ⚠ Say that precisely rather than claiming atomicity outright, since atomicity claims are where this codebase has been burned before: the refusal is EXACT on Windows (MoveFileExwithoutMOVEFILE_REPLACE_EXISTING) and CONDITIONAL on Unix (POSIXrenamefails withENOTEMPTYonly for a NON-EMPTY destination and silently replaces an empty one) — it holds only because a destination is never created any way other than by this same fully-populated rename. Both.stagingand.trashlive INSIDE the root to keep the move on one volume, andEnumerateCandidatesnow skips any path segment starting with.(the ROOT itself may be dotted — the default one is~/.duckdb/…).replace := trueMOVES the old directory aside rather than deleting it: a loaded assembly is LOCKED on Windows and can be renamed but not removed — measured,Directory.Moveof a directory holding a loaded assembly succeeds there. activatedis read back out of the FRESH scan, so one row separates "installed" from "installed and usable"; an install into a root nothing scans says so in DIFFERENT WORDS from a plugin that was scanned and declared nothing (the report is empty either way and they mean opposite things). The platform string is asked of DuckDB (pragma_platform()), never derived — the spelling is DuckDB's.abstractionsVersionis recorded and NOT gated on: nothing versionsFabricator.Abstractions, so a comparison would pass always or fail always — an untestable flag. The real incompatibility already has an honest report (rejected+ the exception).- The zip-slip guard is REUSED: Bridge now project-references
Fabricator.Installer.CoreforArchivePathalone (InternalsVisibleTo, theFabricator.Deltaprecedent). A security guard is the last thing that should exist twice with two chances to drift. - Gates: tier-0
PluginPackageTests+34 (floor 206 → 240) for the rules an end-to-end suite structurally cannot reach (it installs ONE archive, for the ONE platform running it), plus twoPluginPathsTestsfor the hidden-segment rule incl. a dotted ROOT still being searched — without which the default root would disable discovery out of the box. Newverify_plugin_install.test(31, service tier) run against its OWN empty plugin root, since every assertion in it is of the form "this changed". Mutation-tested, each mutant killed at its own section: noInvalidate()dies at the install row (providersempty,activatedfalse) after 9 pass — the files landed, the session did not see them; noPluginLoadedsubtraction dies at the ATTACH assertion with "unknown provider" after 13 pass INCLUDING the first install's success, which is the right discrimination. ⚠ Hazard (2) is REASONED, NOT GATED — the sample plugin has no private dependencies, so no mutant of it dies. Tiers: hermetic 70/70 — 7646 (identical to baseline) and service 51/51 — 2066 (2035 + exactly the 31 new assertions). **ALL FOUR CI WORKFLOWS GREEN ondb59dc2$** — \text{tier} 0 (\text{the} 240 \text{floor}, \text{both} \text{TFMs} \times \text{Windows} \text{and} \text{Linux}), \text{tier} 1 \text{on} **\text{all} \text{three} \text{platforms}**, \text{tier} 2 ($verify_plugin_install+ the archive fixture on Linux) and docs. That closes the one thing the local run could not settle: a brand-newProjectReferenceplus an MSBuildZipDirectorytarget is exactly the shape that trips off Windows. - ⚠ THE FIXTURE BROKE
verify_plugin, and the cause is the recursion working. The archive is emitted by the plugin's OWN build (PackPluginArchive, MSBuildZipDirectory—zipis absent from Git Bash on Windows and a fixture that exists on one platform is a gate that runs on one platform). Staged inbin/, it put a SECOND copy of the plugin under the now-recursive scan and the suite reported TWO loaded plugins. It stages underobj/now.
- ⚠⚠ THE MOST VALUABLE THING HERE IS A BUG IT FOUND IN THE NEW FUNCTION ITSELF, and the way it was
found: a mutant died at the RIGHT PLACE for the WRONG REASON, and chasing that instead of banking the
kill is what exposed it.
- UNINSTALL + THE SHADOWING REFUSAL — BUILT 2026-08-18 (C#-only, no ABI), user-directed.
fabricator_uninstall_plugin(name [, version := …] [, root := …])IS A MOVE, NOT A DELETE, and that is the whole mechanism. An assembly loaded from a file is LOCKED on Windows and the bridge's load context is not collectible, so a plugin that has been USED cannot be deleted — but it CAN be renamed. Moving the version directory into<root>/.trash/<guid>takes it out of the scan at once, which is the mark-for-deletion the design called for without inventing a marker file. The bytes go when they can: a best-effort delete now, plus a SWEEP of the whole trash at the start of the next install or uninstall.- The row reports TWO things and they are different questions:
removed(out of the scan — the only real failure when false) andpurged(bytes gone — ordinarily FALSE, because this process still holds the assembly). Collapsing them into one boolean would make the normal outcome look like a failure.purgedis deliberately NOT asserted in the suite: it would pin an OS behaviour, not ours. - ⚠ The PROVIDER stops resolving; the CODE does not go away. Nothing can unload the assembly, so the guarantee is that the re-scan finds no candidate and does not register it. Say that rather than implying the plugin is gone.
- The sweep runs at install/uninstall only, never on a read path — sweeping during a scan would put filesystem writes on the ATTACH path.
- The row reports TWO things and they are different questions:
- A PLUGIN MAY NOT TAKE A REGISTERED PROVIDER'S NAME — REFUSED (user decision, 2026-08-18).
Addis a plain dictionary assignment, so a plugin declaringIBackend.Name = "sqlserver"used to REPLACE the first-party provider and be reported as an ordinaryloadedrow. Of report / refuse / allow-as-override, refusing is the only one that cannot end in a wrong ANSWER; an override, if ever wanted, should be something the USER asks for by name rather than something a file appearing in a directory can do.- ⚠ Checked BEFORE anything is added — the backends are materialised first, so an assembly whose
SECOND backend collides does not leave its first half-registered. The throw rides the scan's existing
per-candidate handler, so the plugin is
rejectedwith a message naming BOTH sides and every other plugin still loads. - Aliases are checked too, not just
Name.
- ⚠ Checked BEFORE anything is added — the backends are materialised first, so an assembly whose
SECOND backend collides does not leave its first half-registered. The throw rides the scan's existing
per-candidate handler, so the plugin is
- The fixture is a SECOND ASSEMBLY and had to be (
dotnet/Fabricator.CollidingPlugin): a name collision needs two assemblies claiming one name, which no manifest, root or install argument can manufacture. It cannot live insideFabricator.SamplePlugineither — the refusal is all-or-nothing per assembly, so that plugin would stop registering andverify_pluginwould fail. ⚠ Its csproj FIXES the output path (build/test-plugins/collide, no TFM, no RID) so the runner and CI can name it, and the runner passes it as a RELATIVE root:$PWDunder Git Bash is an MSYS path that .NET turns intoD:\d\repos\..., which reports asroot_missingrather than failing — it looks like the fixture is simply absent. - Gates:
verify_plugin_install31 -> 45, mutation-tested, each mutant killed at its own section: allowing the collision dies at the before-state count after 5 pass (the plugin shows asloaded); dropping the uninstall re-scan dies at the post-uninstall "unknown provider" after 36 pass. ⚠ The collision section's LOAD-BEARING assertion is the positive control —mssql_marsstill registered, i.e.Fabricator.SqlServerstill holds the name; "the plugin was rejected" alone would pass on a build where BOTH were broken. - ⚠ The setting's NAME is now narrower than its meaning:
fabricator_allow_plugin_installalso gates uninstall. One opt-in for "SQL may manage the plugin root" is the right granularity — a caller who may add executable code has no reason to be denied removing it — but the name says install. Left deliberately rather than renamed in the same breath as adding the second consumer. - STILL NOT BUILT: archive signature/checksum verification (
Hashingexists inFabricator.Installer.Coreand nothing consumes it here).
- Three changes. (a) A DEFAULT root