Upgrading to EQL 3.0
July 27, 2026 · View on GitHub
3.0.0 is a breaking release. This guide covers the ten breaking changes in
the eql_v3 surface (U-001 through U-010 below).
Coming from EQL v2? 3.0 also removes the
eql_v2schema surface from this repository. There is no in-place conversion foreql_v2_encryptedcolumns — the v3 wire format and domains are different — so migrating an existing v2 deployment means re-encrypting each column into a v3 domain via the CipherStash client tooling, then dropping the v2 schema. See the CipherStash Stack rollout tooling for the column-by-column migration path.
TL;DR
- The
eql_v3JSON envelope version is nowv: 3(U-001). Everyeql_v3domainCHECKpinsVALUE->>'v' = '3', and the published payload bindings (Rust / TypeScript / JSON Schema) accept exactly3. Payloads carrying the legacyv: 2are rejected on insert or cast. - Query-operand domains are
eql_v3.query_<name>(U-002). Renamed from the_querysuffix to aquery_prefix AND moved frompublicinto theeql_v3schema:public.integer_eq_query→eql_v3.query_integer_eq, and the encrypted-JSONB containment needlepublic.jsonb_query→eql_v3.query_json. Only affects 3.0.0 pre-release adopters — the old names never shipped in a final release. - Non-superuser installs disable the ORE-backed domains (U-003). When the installer role cannot create the ORE operator class (cloud-hosted Supabase, Cloud SQL, Azure Flexible Server — AWS RDS and Aurora can; see Install privileges), the
_ord_ore/text_search_oredomains and their query twins now raisefeature_not_supportedon first use instead of silently degrading to seq scans. Use_ord(indexed OPE ordering),_eq, andtext_matchon those platforms. Superuser installs are unchanged. - SteVec (encrypted JSONB) ordering is CLLW-OPE: path entries may carry
op; entries never carryhmoroc(U-004). Theeql_v3_internal.ore_cllwcomposite type, its custom comparator, operators, and superuser-only btree operator class are removed; entry ordering extractseql_v3.ord_term(entry)— a bytea domain that orders under the DEFAULT btree opclass. Exact equality uses value-selector presence, not an entry term. Existing documents must be re-encrypted. - The
_orddomains are backed by CLLW-OPE, not block-ORE (U-005).public.<T>_ordandpublic.eql_v3_text_ordnow require theopterm. The extractor keeps its name —eql_v3.ord_term(col)— but now returnseql_v3_internal.ope_cllw, so ordered functional indexes no longer need a superuser-created operator class. The block-ORE surface moves behind the qualifiedeql_v3.ord_term_ore(col)onpublic.<T>_ord_ore. Two behaviour changes ride along:text_ordnow accepts the empty string, andreal/double_ordcolumns now distinguish-0.0from+0.0. public.eql_v3_text_searchis backed by CLLW-OPE too, andpublic.eql_v3_text_search_oreis new (U-006). The combined search domain now requireshm+op+bfand orders viaeql_v3.ord_term(col); equality (hm) and bloom fuzzy match (bf) are unchanged. The block-ORE shape (hm+ob+bf) lives on the newpublic.eql_v3_text_search_ore.text_searchnow accepts the empty string.- The encrypted-JSON domains are renamed, and
public.eql_v3_jsonis now storage-only (U-007). The searchable SteVec documentpublic.eql_v3_json→public.eql_v3_json_search,public.eql_v3_jsonb_entry→public.eql_v3_json_entry,eql_v3.query_jsonb→eql_v3.query_json; the barepublic.eql_v3_jsonis now a new ciphertext-only storage-only domain. Only affects 3.0.0 pre-release adopters. - Text bloom fuzzy match is
@@/eql_v3.matches, not@>/eql_v3.contains(U-008). Onpublic.eql_v3_text_match/text_search/text_search_orethe fuzzy match is now the single directional operator@@(backed byeql_v3.matches);@>/<@(andeql_v3.contains/contained_by) now raise on these domains. It is n-gram token matching, not containment. The GIN index recipe is unchanged. Genuine JSON containment keeps@>/<@. - Uninstalling can exceed the stock lock budget (U-009). The single-transaction uninstaller drops ~6,400+ objects, each taking one lock — right at the capacity a default
max_locks_per_transaction = 64cluster affords. A lone uninstall on a quiet cluster normally succeeds; under concurrent load it can fail without of shared memory. Raisemax_locks_per_transaction(needs a restart) or uninstall during a quiet window. Installation is unaffected. - SteVec exact match is value-selector presence, and the document wire format is restructured (U-010). Encrypted-JSON field equality now works exactly for every type (including
text/bigint/numeric) via document containment on a value-inclusive selector — spell itcol @> \$1::eql_v3.query_json; the per-valuehmterm is retired, and extract-surface equality (-> 'sel' = …) is blocked (ranges stay). The document now carries a once-per-document key headerh(no rootc); each entry'scis raw AEAD output with a selector-derived nonce;eql_v3.jsonb_array_elements_textis removed. Re-encryption required — there is no mechanical conversion. - An empty-bloom fuzzy-match term no longer matches every row (U-011).
col @@ \$1(eql_v3.matches) on the text match/search domains matched the entire table when the query term carried an empty bloom filter (a search string with no n-gram tokens — e.g. a value below the tokeniser's trigram floor), because empty-array containment is vacuously true. It now usesLIKE ''semantics: an empty needle matches only a value whose own bloom is also empty. Normal (non-empty needle) queries are unchanged and still index-accelerated. No re-encryption, no schema change.
Compatibility
| Component | Status |
|---|---|
eql_v3 schema name | Unchanged. A major bump does not rename the schema. |
Scalar column-domain names (public.<T>, _eq, _ord, _ord_ore, …) | Unchanged. The encrypted-JSON domains are renamed — see U-007. |
| Operator names | Unchanged, except text bloom fuzzy match — see U-008. |
Query-operand domains (public.<name>_query, public.jsonb_query — 3.0.0 pre-releases only) | Changed. Now eql_v3.query_<name> / eql_v3.query_json — see U-002. |
eql_v3 payload envelope version (v) | Changed. 2 → 3. Re-encryption / re-emission with a v3-envelope client required — see U-001. |
ORE-carrying domains (_ord_ore / text_search_ore + query twins) on non-superuser installs | Changed. Poisoned at install time — raise feature_not_supported on use instead of silently degrading — see U-003. Superuser installs unchanged. |
public.<T>_ord, public.eql_v3_text_ord ordering SEM | Changed. Block-ORE (ob) → CLLW-OPE (op). The extractor keeps the name eql_v3.ord_term, but its return type changes eql_v3_internal.ore_block_256 → eql_v3_internal.ope_cllw — see U-005. |
public.<T>_ord_ore, public.eql_v3_text_ord_ore ordering SEM | Unchanged. Still block-ORE (ob). |
public.eql_v3_text_search ordering SEM | Changed. Block-ORE (ob) → CLLW-OPE (op) — see U-006. |
public.eql_v3_text_search_ore | New. The block-ORE (ob) sibling of text_search — see U-006. |
eql_v3.ord_term(public.<T>_ord_ore) extractor | Renamed to eql_v3.ord_term_ore — the qualified name now belongs to block-ORE — see U-005. |
eql_v3.ord_ope_term (3.0.0 pre-releases only) | Removed. Folded into eql_v3.ord_term — the scalar _ord / _ord_ope overloads and the SteVec public.eql_v3_json_entry overload alike — see U-005. |
public.<T>_ord_ope | Unchanged. Now a byte-identical twin of _ord; extracted with eql_v3.ord_term. |
real / double _ord signed-zero semantics | Changed. -0.0 and +0.0 are now distinct under = and < (block-ORE canonicalized them; CLLW-OPE does not) — see U-005. |
text_ord and the empty string | Changed. Now accepted and sorted first; previously a check_violation — see U-005. |
eql_v3 payload term keys (hm / ob / bf / op) | Unchanged as keys; which key an _ord column requires changed — see U-005. |
| SteVec sv-entry term keys | Changed. An optional op (CLLW-OPE) and nothing else — hm is retired (exact match is value-selector presence) and oc (CLLW-ORE) is rejected — see U-004 and U-010. |
| SteVec document wire format | Changed. Per-document key header h (no root c); entry c is raw AEAD output with a selector-derived nonce; value entries encrypt a sentinel. Re-encryption required — see U-010. |
| Encrypted-JSON field equality | Changed. Exact for every type via col @> \$1::eql_v3.query_json (value-inclusive selectors); extract-surface = / <> is blocked — see U-010. |
eql_v3.jsonb_array_elements_text | Removed. A bare-ciphertext stream is no longer independently decryptable; use eql_v3.jsonb_array_elements — see U-010. |
Encrypted-JSON domains (public.eql_v3_json searchable, public.eql_v3_jsonb_entry — 3.0.0 pre-releases only) | Renamed. Searchable document → public.eql_v3_json_search, entry → public.eql_v3_json_entry; the bare public.eql_v3_json is now a new storage-only domain — see U-007. |
eql_v3.ore_cllw(entry) / eql_v3.has_ore_cllw(entry) / eql_v3_internal.ore_cllw (+ comparator, operators, opclass) | Removed. Entry ordering is eql_v3.ord_term(entry) — see U-004. |
Text bloom fuzzy match (public.eql_v3_text_match / text_search / text_search_ore) | Changed. @> / <@ (eql_v3.contains / contained_by) → the single directional @@ (eql_v3.matches); @> / <@ now raise on these domains — see U-008. |
Empty-bloom fuzzy-match needle (@@ / eql_v3.matches) | Changed. An empty needle bloom matched every row (vacuous containment); now LIKE ''-shaped — matches only empty-bloom values. Normal queries and the GIN index recipe unchanged — see U-011. |
| Uninstaller lock footprint | Grown. The single-transaction uninstall takes ~6,400+ locks — at/over the default max_locks_per_transaction = 64 cluster budget under concurrent load — see U-009. Installation unaffected. |
JSON containment (public.eql_v3_json_search @> / <@, eql_v3.ste_vec_contains) | Unchanged. Genuine containment keeps the containment vocabulary — see U-008. |
Legacy eql_v2 wire documentation (docs/reference/schema/eql-payload-v2.*.schema.json) | Unchanged. Stays v: 2. |
Upgrade notes
U-001: eql_v3 payloads carry v: 3
What changed. The eql_v3 tier previously reused the v2 wire's envelope
version (v: 2) for continuity. With 3.0.0 the tier carries the envelope
version matching its schema generation: every generated scalar domain CHECK
and the hand-written public.eql_v3_json_search SteVec document CHECK pin
VALUE->>'v' = '3', and eql-bindings' SchemaVersion (with the emitted
TypeScript alias and the JSON Schema const) accepts exactly 3.
Who is affected. Any producer writing into eql_v3 domain columns. A
cipherstash-client that emits v: 2 cannot insert into eql_v3 columns after
this upgrade — the domain CHECK rejects the payload at insert/cast time with
a check_violation.
What to do. Upgrade the encryption client (CipherStash Stack / protect-ffi /
CipherStash Proxy via cipherstash-client) to a version that emits the v3
envelope before installing EQL 3.0 into a database that receives writes,
and re-encrypt any stored eql_v3 payloads that carry v: 2 (values written
under the earlier pre-release eql_v3 surface). Reads of already-stored
payloads are unaffected until a value is re-written or re-cast against the
domain CHECK.
Verification.
-- Must fail with a check_violation:
SELECT '{"v":2,"i":{},"c":"x","hm":"aa"}'::jsonb::public.eql_v3_text_eq;
-- Must succeed:
SELECT '{"v":3,"i":{},"c":"x","hm":"aa"}'::jsonb::public.eql_v3_text_eq;
U-002: Query-operand domains are eql_v3.query_<name>
What changed. The query-operand domains — the index-terms-only twins a client casts query parameters to — changed identity in two coordinated ways relative to the 3.0.0 pre-releases:
- Named with a
query_PREFIX instead of the_querysuffix:query_integer_eq,query_text_ord, …, andquery_jsonfor the encrypted-JSONB containment needle (wasjsonb_query). - Moved from
publicinto theeql_v3schema:eql_v3.query_integer_eq,eql_v3.query_json, ….
The old names never shipped in a final release. The column-type domains
(public.eql_v3_integer_eq, public.eql_v3_json_search, public.eql_v3_json_entry, …) are unchanged —
they stay in public so dropping EQL-owned schemas can never drop an
application column.
Why. A query operand is never a valid column type, so it doesn't belong in
the column-type namespace: in public the operands interleaved with the
actual column types in alphabetical type listings (most visibly Supabase
Studio's Table Builder type picker), and the survive-schema-drop rationale for
public placement doesn't apply to a type no application column should use.
In eql_v3 the operands are versioned and uninstalled with the rest of the
public API surface, and the query_ prefix keeps them sorted together
wherever they are listed.
Who is affected. Only adopters of the 3.0.0 pre-releases. Any SQL casting
to a pre-release query domain (WHERE col = \$1::public.integer_eq_query,
WHERE doc @> \$1::public.jsonb_query), and any client resolving the
eql-bindings query domains by name (QueryPayload::parse("integer_eq_query", …),
the schema/v3/*_query.json JSON Schema files).
What to do.
- Rename the casts:
$1::eql_v3.query_integer_eq,$1::eql_v3.query_json. - Update
eql-bindingsto the matching release (theDomainType::sql_domainstrings,QueryPayload::parsenames, and JSON Schema file names follow the new convention). - Grants: casting to an
eql_v3domain requiresUSAGE ON SCHEMA eql_v3— the same grant a querying role already needs for the extractors and comparison wrappers (seedocs/reference/permissions.md), so no new grant is expected in practice. - Audit for misuse: if any table column was declared with a pre-release
query-operand type, move it to the matching storage domain — from 3.0.0 the
uninstaller's
DROP SCHEMA eql_v3 CASCADEdrops query-operand-typed columns with the schema. - GIN containment indexes built over
eql_v3.to_ste_vec_query(col)are unaffected — the function name is unchanged; only the domain type moved.
Verification.
-- Must succeed (the relocated operand domains exist):
SELECT '{"v":3,"i":{},"hm":"aa"}'::jsonb::eql_v3.query_text_eq;
SELECT '{"sv":[{"s":"aa"}]}'::jsonb::eql_v3.query_json; -- entries carry s (+ optional op) only; hm is rejected (U-004/U-010)
-- Must fail with "type does not exist" (the pre-release names are gone):
SELECT '{"v":3,"i":{},"hm":"aa"}'::jsonb::public.text_eq_query;
SELECT '{"v":3,"i":{},"hm":"aa"}'::jsonb::public.query_text_eq;
U-003: Non-superuser installs disable the ORE-backed domains
What changed. CREATE OPERATOR CLASS requires superuser, so on managed
platforms whose installer role is not superuser (cloud-hosted Supabase, most
hosted Postgres) the installer has always attempted the ORE operator class and
skipped it with a NOTICE on insufficient_privilege. Previously the
ORE-carrying domains still installed and silently degraded: < / >
comparisons ran as unindexable seq scans, while
CREATE INDEX ... (eql_v3.ord_term(col)) failed with
data type eql_v3_internal.ore_block_256 has no default operator class and
bare ORDER BY could not find an ordering operator. From 3.0.0 the installer
capability-detects the skip and poisons every ORE-carrying domain —
_ord_ore on each ordered scalar family, text_search_ore, and their
eql_v3.query_* twins — with an always-raising CHECK
constraint. The set is derived from the catalog, so it shrinks automatically as
families move off block-ORE: _ord (see U-005) and text_search (see U-006) are
CLLW-OPE-backed and are not poisoned. The first value cast or inserted into one (including NULL)
raises feature_not_supported (SQLSTATE 0A000) naming the domain, with a
HINT listing the same family's platform-supported alternatives. The
constraint is added NOT VALID, so rows stored before the capability was
lost — data written under an earlier superuser install, before the installing
role was demoted — stay readable, and re-running the installer over them
succeeds; only new casts and inserts raise.
Why. Failing loudly at first use beats a seq-scan performance trap and
opaque index-time errors discovered in production. The supported alternatives
are strictly better on these platforms: the _ord / _ord_ope domains order via
CLLW-OPE, whose extractor (eql_v3.ord_term) returns a type with a
native btree operator class — indexed ordering with no superuser required.
Who is affected. Installations run by a non-superuser role that use an ORE-carrying domain. Superuser installs (self-managed Postgres, Docker) are unchanged: the operator class is created and nothing is poisoned. The detection is install-time — an installer that ran as superuser keeps full ORE support regardless of which role queries later.
What to do. On managed platforms, type ordered columns as _ord (or the
_ord_ope twin) instead of _ord_ore, and index with
CREATE INDEX ... USING btree (eql_v3.ord_term(col)). The combined
public.eql_v3_text_search domain is CLLW-OPE-backed (see U-006) and is not
poisoned, so equality, ordering, and bloom fuzzy match all work on it here —
only its block-ORE sibling public.eql_v3_text_search_ore is disabled. Nothing needs
to be split into text_eq + text_match + text_ord columns.
Scope caveat — resolved. The encrypted-JSONB (SteVec) ordered-comparison
path needs no such gate as of this release: SteVec ordering switched from the
ore_cllw operator class to CLLW-OPE terms that order under the native bytea
opclass (U-004), so there is
no superuser-only object left to skip and nothing to poison.
Verification.
-- On a non-superuser install: must raise SQLSTATE 0A000 with the alternatives HINT.
SELECT '{"v":3,"i":{"t":"t","c":"c"},"c":"ct","ob":["aa"]}'::jsonb::public.eql_v3_integer_ord_ore;
-- And the OPE-backed default must work (it carries no ORE term, so it is not poisoned):
SELECT '{"v":3,"i":{"t":"t","c":"c"},"c":"ct","op":"aa"}'::jsonb::public.eql_v3_integer_ord;
-- On a superuser install: no domain is poisoned.
SELECT count(*) FROM pg_constraint WHERE conname = 'eql_ore_unavailable'; -- 0
U-004: SteVec ordering terms are CLLW-OPE (op)
What changed. Encrypted-JSONB (SteVec) entries previously carried hm
XOR oc, where oc was a CLLW-ORE ciphertext compared by a hand-written
per-byte protocol on the eql_v3_internal.ore_cllw composite type — with its
own comparator, six operators, and a DEFAULT FOR TYPE btree operator class
that only a superuser install could create. An entry now carries an optional
op and no hm at all (the per-value equality term was retired in 3.0.0
— see U-010),
where op is a CLLW-OPE (Order-Preserving Encryption) ciphertext whose
hex-decoded bytes order under native bytea comparison — the same op
term and eql_v3_internal.ope_cllw bytea domain the scalar _ord /
_ord_ope domains use. The domain CHECKs on public.eql_v3_json_search /
public.eql_v3_json_entry / eql_v3.query_json reject both oc and hm. The
eql_v3.ore_cllw(entry) and eql_v3.has_ore_cllw(entry) functions and the
entire eql_v3_internal.ore_cllw surface are removed; the entry ordering
extractor is the public.eql_v3_json_entry overload of eql_v3.ord_term
(SQL NULL when the entry carries no op).
Why. CREATE OPERATOR CLASS requires superuser, so SteVec entry
ordering was the last EQL surface that could not index on cloud-hosted
Supabase / managed Postgres. The OPE term needs no operator class at all —
eql_v3.ord_term returns a bytea domain with the default btree opclass,
the whole comparison chain is inlinable SQL, and a plain functional btree
index engages structurally on any install.
Who is affected. Every producer and consumer of public.eql_v3_json_search /
public.eql_v3_json_entry payloads, and any SQL using the removed functions.
What to do.
- Upgrade the encryption client (CipherStash Stack / protect-ffi /
CipherStash Proxy via cipherstash-client) to a version that emits CLLW-OPE
(
op) SteVec terms, and re-encrypt stored SteVec documents that carryocentries — there is no mechanical conversion (ORE ciphertext bytes do not order under byte comparison;eql-bindings'from_v2fails closed withUnconvertibleSteVecDocumentfor every legacy SteVec document). - Rebuild ordered-access indexes:
CREATE INDEX ... ((eql_v3.ord_term(doc -> '<selector>'::text)))replaces any index overeql_v3.ore_cllw(doc -> '<selector>'::text), andORDER BYexpressions change the same way. - Replace
eql_v3.has_ore_cllw(entry)checks witheql_v3.ord_term(entry) IS NOT NULL. - Containment (
@>/<@), equality on extracted entries, GIN recipes overeql_v3.jsonb_array(col)/eql_v3.to_ste_vec_query(col), andmin/maxaggregates keep their spelling — they pick up the new term key internally.
Verification.
-- Must succeed (op-bearing entry; document carries the key header h):
SELECT '{"i":{},"v":3,"k":"sv","h":"kh","sv":[{"s":"aa","c":"ct","op":"01ff"}]}'::jsonb::public.eql_v3_json_search;
-- Must fail with a check_violation (oc is no longer a valid entry term):
SELECT '{"i":{},"v":3,"k":"sv","h":"kh","sv":[{"s":"aa","c":"ct","oc":"01ff"}]}'::jsonb::public.eql_v3_json_search;
-- Must fail with a check_violation (hm is retired):
SELECT '{"i":{},"v":3,"k":"sv","h":"kh","sv":[{"s":"aa","c":"ct","hm":"01ff"}]}'::jsonb::public.eql_v3_json_search;
-- Must fail with "function does not exist":
SELECT eql_v3.ore_cllw(NULL::public.eql_v3_json_entry);
U-005: _ord domains are backed by CLLW-OPE
What changed. The default ordering domain switched its searchable-encrypted-metadata (SEM) scheme from block-ORE to CLLW-OPE:
| before | after | |
|---|---|---|
public.<T>_ord CHECK | requires ob (non-empty array) | requires op |
public.eql_v3_text_ord CHECK | requires hm + ob (non-empty array) | requires hm + op |
| ordering extractor | eql_v3.ord_term(col) → eql_v3_internal.ore_block_256 | eql_v3.ord_term(col) → eql_v3_internal.ope_cllw |
| comparison | custom N-block plpgsql protocol (calls pgcrypto) | native bytea comparison |
| btree operator class | hand-written for a composite type, created by a superuser-only DO block; if skipped, an ordered index silently binds record_ops and never engages | bytea_ops, the base type's default |
The extractor name follows the domain, not the cipher. _ord is the default
ordering domain, so it keeps the unqualified eql_v3.ord_term. What changed under
that name is the return type (ore_block_256 → ope_cllw) and the payload term
it reads (ob → op). Block-ORE — now the by-name escape hatch — moves to the
qualified eql_v3.ord_term_ore.
No ordering capability was removed. public.<T>_ord_ore and
public.eql_v3_text_ord_ore still carry the block-ORE term, now extracted with
eql_v3.ord_term_ore; eql_v3_internal.ore_block_256 and its comparator,
operators, and operator class all still ship. _ord and _ord_ope are now the
byte-identical twins (previously it was _ord and _ord_ore).
public.eql_v3_text_search made the same move — see U-006.
Pre-release adopters only: eql_v3.ord_ope_term (which served _ord_ope in
3.0.0-alpha.3) no longer exists. Its overloads folded into eql_v3.ord_term.
Why. eql_v3_internal.ope_cllw is a DOMAIN ... AS bytea whose ciphertext is
order-preserving under plain byte comparison, so it inherits bytea's comparison
operators and its default operator class, bytea_ops, outright. A functional
btree on eql_v3.ord_term(col) therefore installs with no special privilege,
and the opfamily already contains the < <= > >= the planner must match.
The block-ORE path does not have that property. eql_v3_internal.ore_block_256 is a
composite type whose btree operator class is hand-written and created by a DO
block that silently skips on insufficient_privilege — the ordinary case on
Supabase and other managed PostgreSQL. The failure mode is quiet rather than loud:
with the opclass absent, CREATE INDEX … btree (eql_v3.ord_term_ore(col)) still
succeeds, because PostgreSQL falls back to record_ops on the composite. That
index then never engages, since the ORE comparison operators are not members of
record_ops, and the byte order it stores is not the ORE order. You get an index
that costs writes and buys nothing. _ord removes the failure mode rather than
reporting it.
Who is affected. Anyone with a column typed public.<T>_ord or
public.eql_v3_text_ord, or a client emitting ob into one. Separately, anyone with an
index or extractor-form ORDER BY written against eql_v3.ord_term(col) for an
_ord_ore column, since that extractor is now named eql_v3.ord_term_ore. Pre-release
adopters using eql_v3.ord_ope_term on _ord_ope are affected too.
What to do.
-
Rebuild ordered functional indexes on
_ordcolumns. The index DDL text is unchanged —eql_v3.ord_term(col)still resolves — but the expression now returns a different type over a different payload term, so the existing index is invalid and must be rebuilt after the data migration in step 2:REINDEX INDEX events_at_ord; -- or DROP + CREATE, identical DDLBare-form predicates (
WHERE col < \$1) and bareORDER BY colneed no change — the operators inline to the new extractor under the same name. -
Rename extractor-form sites on
_ord_orecolumns.eql_v3.ord_term(public.<T>_ord_ore)no longer exists; the index DDL and any extractor-formORDER BYmust useeql_v3.ord_term_ore. This is a pure rename — same term (ob), same type, same behaviour — so no re-encryption and no reindex is required, only the DDL text:DROP INDEX events_at_ord_ore; CREATE INDEX events_at_ord_ore ON events USING btree (eql_v3.ord_term_ore(encrypted_at));Pre-release adopters on
_ord_ope: replaceeql_v3.ord_ope_term(col)witheql_v3.ord_term(col). Also a pure rename —_ord_opestill carriesop. -
Re-encrypt
_ordcolumns, or retype them. A stored payload carrying onlyobfails the new_ordCHECK. Either re-encrypt through a client that emitsop(cipherstash-client 0.38.1+ already emits it for an ordered column configured with theopeindex), or keep block-ORE by retyping the column:ALTER TABLE events ALTER COLUMN encrypted_at TYPE public.eql_v3_timestamp_ord_ore;Retyping is the zero-re-encryption path and is also the rollback.
-
Note the empty-string change on
text_ord.public.eql_v3_text_ordnow accepts the empty string and sorts it first. Previously its non-empty-obCHECK rejected it with acheck_violation(issue #262), because encrypting""yields an empty, unorderable ORE term (ob: []). OPE has no such failure mode:""encrypts to a well-formed one-byteopterm that is a proper prefix of every non-empty term, sobyteacomparison sorts it below them — exactly where the plaintext belongs. If your application relied on the database rejecting empty strings in an ordered text column, enforce that with your ownCHECK, or usepublic.eql_v3_text_ord_ore, which still rejects it. -
Note the signed-zero change on
real/double_ordcolumns. Block-ORE encodes floats throughorderable-bytes, which canonicalizes-0.0 -> +0.0; CLLW-OPE does not. So on an OPE-backed_ordfloat column:_ord_ore(block-ORE)_ord(CLLW-OPE)-0.0 = +0.0true false -0.0 < +0.0false true IEEE 754 says
-0.0 == 0.0, so_ord_oreagrees with it and_orddoes not. Note thatpublic.<T>_eqalready disagreed with IEEE here — thehmterm hashes the rawf64bytes, sign bit included (#387). So this change makes_ordconsistent with_eq(both distinguish signed zero) where block-ORE was inconsistent with it. It is a behaviour change either way.-0.0arises from ordinary arithmetic (0.0 * -1.0, underflow, a JSON-0literal). If a float column must treat±0.0as one value for=orORDER BY, type it_ord_ore, or normalize the sign of zero before encrypting. This does not affect the integer,numeric,date,timestamp, ortextfamilies.
Verification.
-- `ord_term` still serves _ord, but now returns the OPE type.
SELECT to_regprocedure('eql_v3.ord_term(public.eql_v3_integer_ord)') IS NOT NULL; -- t
SELECT pg_get_function_result(to_regprocedure('eql_v3.ord_term(public.eql_v3_integer_ord)'));
-- eql_v3_internal.ope_cllw
-- Block-ORE moved to the qualified name; the old overload is gone.
SELECT to_regprocedure('eql_v3.ord_term_ore(public.eql_v3_integer_ord_ore)') IS NOT NULL; -- t
SELECT to_regprocedure('eql_v3.ord_term(public.eql_v3_integer_ord_ore)') IS NULL; -- t
-- The pre-release OPE extractor name no longer exists.
SELECT to_regprocedure('eql_v3.ord_ope_term(public.eql_v3_integer_ord)') IS NULL; -- t
-- An ordered functional index needs no superuser, and binds the base type's
-- default opclass (bytea_ops) rather than a hand-installed one.
CREATE TABLE t (c public.eql_v3_integer_ord);
CREATE INDEX t_ord ON t USING btree (eql_v3.ord_term(c)); -- succeeds as a plain role
SELECT oc.opcname -- expect: bytea_ops
FROM pg_index x
JOIN pg_class i ON i.oid = x.indexrelid
JOIN pg_opclass oc ON oc.oid = x.indclass[0]
WHERE i.relname = 't_ord';
-- Audit an existing _ord_ore index: `record_ops` means the ORE opclass was
-- skipped at install time and the index is inert.
SELECT oc.opcname
FROM pg_index x
JOIN pg_class i ON i.oid = x.indexrelid
JOIN pg_opclass oc ON oc.oid = x.indclass[0]
WHERE i.relname = '<your _ord_ore index>'; -- want ore_block_256_operator_class
-- An _ord payload must carry `op`, not `ob`.
SELECT '{"v":3,"i":{},"c":"x","op":"00ff"}'::jsonb::public.eql_v3_integer_ord; -- ok
SELECT '{"v":3,"i":{},"c":"x","ob":["aa"]}'::jsonb::public.eql_v3_integer_ord; -- check_violation
SELECT '{"v":3,"i":{},"c":"x","ob":["aa"]}'::jsonb::public.eql_v3_integer_ord_ore; -- ok
Rollback. Retype the affected columns back to public.<T>_ord_ore /
public.eql_v3_text_ord_ore and rebuild their indexes against eql_v3.ord_term_ore(col).
The block-ORE surface is still installed, so no EQL downgrade is required.
U-006: text_search is backed by CLLW-OPE
What changed. public.eql_v3_text_search — the combined domain carrying equality,
ordering, and bloom fuzzy match in one type — swapped its ordering term from
block-ORE to CLLW-OPE, exactly as _ord did in U-005. Its equality (hm) and
match (bf) terms are untouched. The block-ORE shape is preserved under a
new name, public.eql_v3_text_search_ore.
public.eql_v3_text_search (changed) | public.eql_v3_text_search_ore (new) | |
|---|---|---|
| CHECK requires | hm + op + bf | hm + ob + bf |
| ordering extractor | eql_v3.ord_term(col) → eql_v3_internal.ope_cllw | eql_v3.ord_term_ore(col) → eql_v3_internal.ore_block_256 |
| equality extractor | eql_v3.eq_term(col) | eql_v3.eq_term(col) |
| match extractor | eql_v3.match_term(col) | eql_v3.match_term(col) |
| empty string | accepted, sorts first | rejected (check_violation) |
The operator surface of the two domains is identical (= <> < <= > >=
@@). Equality still routes through hm on both — text ordering terms are
not equality-lossless — and fuzzy match still routes through bf (via the @@
operator / eql_v3.matches, renamed from @>/contains in U-008).
Why. The same reason as U-005, and it is not about operator capability.
text_search ordered through block-ORE, whose btree operator class is
hand-written for a composite type and installed by a DO block that silently
skips on insufficient_privilege — the ordinary case on Supabase and other
managed PostgreSQL. With that opclass absent, an ordered functional index over
text_search still succeeds at CREATE INDEX (PostgreSQL binds
record_ops) and then never engages. eql_v3_internal.ope_cllw is a
DOMAIN ... AS bytea and inherits bytea_ops, the base type's default opclass,
so the index engages with no special privilege.
Who is affected. Anyone with a column typed public.eql_v3_text_search, a client
emitting ob into one, or an index / extractor-form ORDER BY written against
eql_v3.ord_term_ore(col) for such a column.
What to do.
-
Choose a domain. Keep the OPE default (re-encrypt so rows carry
op), or retype to the block-ORE sibling and change nothing else:ALTER TABLE docs ALTER COLUMN body TYPE public.eql_v3_text_search_ore;Retyping is the zero-re-encryption path, and is also the rollback.
-
Repoint extractor-form ordering sites.
eql_v3.ord_term_ore(public.eql_v3_text_search)no longer exists. On an OPEtext_searchcolumn the ordering extractor iseql_v3.ord_term(col); ontext_search_oreit stayseql_v3.ord_term_ore(col). Bare-form predicates (WHERE col < \$1) and bareORDER BY colneed no change. Equality and@@(fuzzy match) index recipes are unaffected on both domains (the match operator/function was renamed@>/contains→@@/matches— see U-008). -
Rebuild ordered indexes on
text_search. The DDL text is unchanged, but the expression now returns a different type over a different payload term. -
Note the empty-string change.
public.eql_v3_text_searchnow accepts the empty string and sorts it first, for the same reasontext_orddoes (issue #262): OPE encrypts""to a well-formed one-byteopterm, whereas block-ORE produces an unorderable empty array (ob: []). If you relied on the database rejecting empty strings in a search column, enforce it with your ownCHECK, or usepublic.eql_v3_text_search_ore, which still rejects it.
Verification.
-- text_search orders through the OPE extractor; the ORE overload is gone for it.
SELECT to_regprocedure('eql_v3.ord_term(public.eql_v3_text_search)') IS NOT NULL; -- t
SELECT to_regprocedure('eql_v3.ord_term_ore(public.eql_v3_text_search)') IS NULL; -- t
-- the block-ORE sibling exists and keeps the ORE extractor.
SELECT to_regprocedure('eql_v3.ord_term_ore(public.eql_v3_text_search_ore)') IS NOT NULL; -- t
-- both still support equality and bloom fuzzy match.
SELECT to_regprocedure('eql_v3.match_term(public.eql_v3_text_search)') IS NOT NULL; -- t
SELECT to_regprocedure('eql_v3.match_term(public.eql_v3_text_search_ore)') IS NOT NULL; -- t
-- An ordered functional index binds bytea_ops, not record_ops, as a plain role.
CREATE TABLE t (c public.eql_v3_text_search);
CREATE INDEX t_s ON t USING btree (eql_v3.ord_term(c));
SELECT oc.opcname -- expect: bytea_ops
FROM pg_index x
JOIN pg_class i ON i.oid = x.indexrelid
JOIN pg_opclass oc ON oc.oid = x.indclass[0]
WHERE i.relname = 't_s';
-- Payload shape: text_search wants `op`, text_search_ore wants `ob`.
SELECT '{"v":3,"i":{},"c":"x","hm":"aa","op":"00ff","bf":[1]}'::jsonb::public.eql_v3_text_search; -- ok
SELECT '{"v":3,"i":{},"c":"x","hm":"aa","ob":["aa"],"bf":[1]}'::jsonb::public.eql_v3_text_search; -- check_violation
SELECT '{"v":3,"i":{},"c":"x","hm":"aa","ob":["aa"],"bf":[1]}'::jsonb::public.eql_v3_text_search_ore; -- ok
Rollback. ALTER TABLE … ALTER COLUMN … TYPE public.eql_v3_text_search_ore and
rebuild the ordered index against eql_v3.ord_term_ore(col). The block-ORE
surface is still installed, so no EQL downgrade is required.
U-007: the encrypted-JSON domains are renamed, and public.eql_v3_json is now storage-only
What changed. The JSON family now follows the same naming convention as every scalar family — the bare family name is the storage-only domain, and a suffix marks the searchable variant. Three renames plus one new domain:
| Before (3.0.0 pre-release) | After | What it is |
|---|---|---|
public.eql_v3_json | public.eql_v3_json_search | searchable SteVec document (@>, path access, ordered field compare) |
public.eql_v3_jsonb_entry | public.eql_v3_json_entry | one sv element (returned by ->) |
eql_v3.query_jsonb | eql_v3.query_json | containment needle (query operand) |
| — (new) | public.eql_v3_json | ciphertext-only, storage-only encrypted JSON ({v,i,c}, no index terms) |
The bare public.eql_v3_json name no longer means the searchable document — it is
now a storage-only domain (the JSON analogue of public.eql_v3_boolean): one
opaque ciphertext for encrypt-at-rest with no server-side searchability. Every
comparison/containment native jsonb operator (=, @>, <@, #>, ||) is
blocked on it.
Why. Consistency. For every scalar family the bare name is storage-only and a
suffix (_eq, _ord, _search) adds capability; the JSON family was the lone
exception, where the bare name was the fully-searchable document and there was no
storage-only form. _search mirrors text's combined _search domain, and the new
bare public.eql_v3_json fills the previously-missing ciphertext-only slot.
Who is affected. Anyone with a column typed public.eql_v3_json (the searchable
document), a public.eql_v3_jsonb_entry reference, or a query cast to
eql_v3.query_jsonb — all from a 3.0.0 pre-release. The old names never shipped in
a final release.
What to do. Retype searchable columns to public.eql_v3_json_search and update
containment casts to eql_v3.query_json:
ALTER TABLE t ALTER COLUMN doc TYPE public.eql_v3_json_search;
-- containment queries:
SELECT * FROM t WHERE doc @> \$1::eql_v3.query_json;
Use the new bare public.eql_v3_json only for encrypt-at-rest JSON you never search:
-- storage-only: accepts the {v,i,c} envelope, rejects a SteVec document
SELECT '{"v":3,"i":{},"c":"x"}'::jsonb::public.eql_v3_json; -- ok
SELECT '{"v":3,"i":{},"sv":[{"s":"a","hm":"h"}]}'::jsonb::public.eql_v3_json; -- check_violation
Rollback. Downgrade the EQL installation to the pre-release you came from
first, then retype the columns back. Unlike U-005 / U-006, the old surface is not
still installed: 3.0.0 renames these domains, so public.eql_v3_jsonb_entry and
eql_v3.query_jsonb no longer exist to retype to, and the bare
public.eql_v3_json now resolves to the storage-only domain, which rejects a
SteVec document. No envelope or term-key change rides along, so the stored
ciphertext itself needs no re-encryption.
U-008: text bloom fuzzy match is renamed to @@ / eql_v3.matches
What changed. On the text bloom-match domains — public.eql_v3_text_match,
public.eql_v3_text_search, public.eql_v3_text_search_ore — the fuzzy match was
previously spelled with the containment operators @> / <@, backed by the public
wrappers eql_v3.contains / eql_v3.contained_by. It is now a single directional
operator @@, backed by eql_v3.matches. The reverse contained_by direction is
dropped, and @> / <@ now raise operator ... is not supported on these
domains (they are generated as blockers).
| before (3.0.0 pre-release) | now | |
|---|---|---|
| operator | col @> \$1 (contains), col <@ \$1 | col @@ \$1 (matches) |
| function | eql_v3.contains(col, \$1), eql_v3.contained_by(col, \$1) | eql_v3.matches(col, \$1) |
@> / <@ on the domain | supported (were the match operators) | raise ("not supported") |
| GIN index recipe | GIN (eql_v3.match_term(col)) | unchanged |
Why. The operation is not containment. col @@ needle reduces to
eql_v3.match_term(col) @> eql_v3.match_term(needle) — array containment over two
bloom filters built from the downcased 3-gram token set. That is a probabilistic,
order- and multiplicity-insensitive, one-sided fuzzy match: a true may be a
false positive; a false never is. Naming it contains / @> promised
substring/containment semantics it does not provide, and collided with the genuine
containment surface on encrypted JSON (public.eql_v3_json_search,
eql_v3.ste_vec_contains), which is unaffected and keeps @> / <@. This is the
SQL-layer counterpart of the SDK's contains() → matches() rename.
Who is affected. Anyone with a query, view, or SDK/adapter that emits
eql_v3.contains(col, …) / eql_v3.contained_by(col, …) or the @> / <@
operators against a text_match / text_search / text_search_ore column.
What to do. Repoint match predicates from @> / contains to @@ / matches.
The functional GIN index needs no change — col @@ \$1 inlines through match_term
to the same array-containment the index supports:
-- before:
SELECT * FROM t WHERE name @> \$1::eql_v3.query_text_match;
SELECT * FROM t WHERE eql_v3.contains(name, \$1::eql_v3.query_text_match);
-- now:
SELECT * FROM t WHERE name @@ \$1::eql_v3.query_text_match;
SELECT * FROM t WHERE eql_v3.matches(name, \$1::eql_v3.query_text_match);
Coordinate with the client: an SDK still emitting eql_v3.contains( for text match
must switch to eql_v3.matches( in the same release, since contains is no longer
a callable public function on these domains.
Verification. \df eql_v3.matches lists the wrapper; col @@ \$1 engages the
eql_v3.match_term(col) GIN index (EXPLAIN shows a Bitmap/Index Scan); col @> \$1
now raises operator ... is not supported. Genuine JSON containment is unchanged:
WHERE doc @> \$1::eql_v3.query_json still works.
Rollback. Downgrade the EQL installation to 2.x first, then revert queries from
@@ / matches back to the @> / contains forms. Reverting the queries alone is
not enough: under 3.0.0 @> / contains raise on these domains (see
Verification above), so the old query form fails until the old surface is
reinstalled. There is no envelope, term-key, or index change — only the
operator/function name moved — so no re-encryption or index rebuild is involved.
U-009: the uninstaller can exceed Postgres's default lock budget
What changed. The v3 surface has grown to the point where the shipped
uninstaller — whose core is DROP SCHEMA eql_v3 CASCADE followed by
DROP SCHEMA eql_v3_internal CASCADE — takes 6,433 locks when run as a
single transaction (measured on this release: one per dropped function,
operator, type, and cast, all held until commit). Note the shipped file
contains no BEGIN/COMMIT of its own: run it with
psql --single-transaction -f cipherstash-encrypt-uninstall.sql to get the
atomic behaviour this note describes — under plain autocommit each DROP
commits separately (fewer locks at once, but eql_v3 can drop while
eql_v3_internal survives if the second statement fails). PostgreSQL sizes
its shared lock table once at startup as max_locks_per_transaction × (max_connections + max_prepared_transactions); at the defaults (64 × 100)
that is 6,400 slots for the whole cluster, so a full single-transaction
uninstall now exceeds what the default configuration nominally affords.
Who is affected. Anyone running cipherstash-encrypt-uninstall.sql on a
cluster with the default max_locks_per_transaction = 64 — in practice only
when the cluster is doing other work at the same time. A lone uninstall on a
quiet cluster normally succeeds anyway, because Postgres lets the lock table
overflow into spare shared memory; concurrent transactions holding locks shrink
that headroom, and the uninstall (or an innocent bystander transaction) then
fails with:
ERROR: out of shared memory
HINT: You might need to increase max_locks_per_transaction.
Nothing is left half-dropped — under --single-transaction the transaction
aborts atomically — but the error is confusing and the standard remedy needs a
server restart, which is worth knowing before scheduling an uninstall on
a production cluster.
What to do. Either:
- Uninstall during a quiet window (the usual case — no configuration change needed, spare shared memory absorbs the overflow), or
- Raise
max_locks_per_transactionahead of time if the cluster cannot be quiesced — e.g.ALTER SYSTEM SET max_locks_per_transaction = 256;followed by a restart (the setting cannot be changed at runtime). The cost is modest (~64 bytes per slot of shared memory).
Installation is unaffected: CREATE takes no per-object lock, so the installer
runs comfortably inside any default configuration.
Verification. After a successful uninstall, \dn eql* lists no EQL schemas.
To preview the lock footprint without dropping anything:
BEGIN; \i cipherstash-encrypt-uninstall.sql then
SELECT count(*) FROM pg_locks WHERE pid = pg_backend_pid(); and ROLLBACK;.
Rollback. Not applicable — this is an operational note, not a behaviour change. Re-running the installer restores the surface.
U-010: SteVec value-inclusive selectors and the envelope wire format
What changed. Two coupled changes to the encrypted-JSONB (SteVec) surface, both in this release:
-
Exact equality is value-selector presence, for every type. Each JSON node now emits a value entry whose selector
SEL(type-tag ‖ path ‖ canonical(value))bakes the value into the selector, so the selector's presence in a stored document is an exact, injective match. The per-valuehmequality term is retired — an entry carries only an optionalopordering term (number/string path entries). Field equality is now document containment on the value selector:WHERE col @> \$1::eql_v3.query_json, where the client emits a value-selector needle. This is exact fortext,bigint, andnumerictoo — where the oldop-based comparison collided ("café"=="cafe",9007199254740993==…992). Correspondingly, extract-surface equality is blocked:col -> 'sel'::text = \$1::…raisesoperator is not supportedfor every family (an extracted path entry carries no value selector). Range queries on the extract surface are unchanged and still index-accelerated. -
The document wire format is restructured. A SteVec document is now
{v, k:"sv", i, h, sv:[…]}. The key-retrieval material (IV, tag, descriptor, keyset) is hoisted once to the document-levelhkey instead of being repeated inside every entry; there is no rootc(the root document ciphertext is the rootsventry). Each entry'scis the raw AEAD output only, and its nonce is derived from the entry's own selector (nonce = hex_decode(s)[..12]) rather than a shared per-document IV. Value entries encrypt a fixed versioned sentinel (not the empty string), so a value entry stays distinguishable from a genuine""leaf.eql_v3.jsonb_array_elements_text(a bare-ciphertextSETOF text) is removed — an entry's ciphertext is not independently decryptable.
Why. (1) op-based field equality was lossy for collated text and
large integers; value selectors make it exact and injective for every type.
(2) Under the old format every entry repeated the full key header and shared
one IV-derived nonce, so equal values at different paths — and the constant
value-entry plaintext — produced byte-identical ciphertexts (an intra-document
equality leak the value selectors otherwise avoid). Selector-derived nonces
under the single document key give every entry a distinct ciphertext, bind each
ciphertext to its selector (a within-document graft fails to decrypt), and let
the key header be stored once — shrinking a document ~30%.
Who is affected. Every producer and consumer of public.eql_v3_json_search
/ public.eql_v3_json_entry payloads; anyone doing encrypted-JSON field
equality on the extract surface; anyone calling eql_v3.jsonb_array_elements_text.
What to do.
- Upgrade the encryption client (CipherStash Stack / protect-ffi /
CipherStash Proxy via cipherstash-client) to a version that emits the new
envelope wire format, and re-encrypt stored SteVec documents — there is no
mechanical conversion (the key material, entry ciphertexts, and nonces all
change;
eql-bindings'from_v2fails closed withUnconvertibleSteVecDocumenton a v2 SteVec document, andUnconvertibleSteVecQueryon every v2 SteVec query). Legacy queries carry only path selectors and index terms; they cannot supply the plaintext needed to derive v3's value-inclusive selector. - Rewrite encrypted-JSON field equality from any extract-surface form to
containment on the value selector:
WHERE col @> \$1::eql_v3.query_json(the client builds the needle for the field/value). A functional GIN index oneql_v3.to_ste_vec_query(col)::jsonbengages it. - Replace
eql_v3.jsonb_array_elements_text(col)witheql_v3.jsonb_array_elements(col)— its entry rows carry the grafted key header and are the decryptable unit. - Range queries, document-to-document containment (
@>/<@,eql_v3.ste_vec_contains), field/array access (->,->>), ordered entry-to-entry comparison,eql_v3.ord_term, andmin/maxare unchanged. Entry-to-entry=/<>now fail loudly; use value-selector containment for exact equality.
Verification.
-- A v3 document carries the key header h and no root c:
SELECT '{"v":3,"k":"sv","i":{},"h":"kh","sv":[{"s":"aa","c":"ct"}]}'::jsonb::public.eql_v3_json_search;
-- A document WITHOUT h is rejected:
SELECT '{"v":3,"k":"sv","i":{},"sv":[{"s":"aa","c":"ct"}]}'::jsonb::public.eql_v3_json_search; -- check_violation
-- An hm-bearing entry is rejected (hm retired):
SELECT '{"v":3,"k":"sv","i":{},"h":"kh","sv":[{"s":"aa","c":"ct","hm":"01"}]}'::jsonb::public.eql_v3_json_search; -- check_violation
-- Must fail with "function does not exist":
SELECT eql_v3.jsonb_array_elements_text('{}'::jsonb);
Rollback. Not applicable within 3.0.0 — the wire format is part of the
breaking release. Downgrading to a v2-emitting client and the eql_v2 surface
is the v2.x → v3 rollback path (a full re-encryption in reverse).
U-011: An empty-bloom fuzzy-match term no longer matches every row
What changed. eql_v3.matches — the @@ fuzzy-match operator on
public.eql_v3_text_match, public.eql_v3_text_search, and
public.eql_v3_text_search_ore — reduces to bloom array-containment
match_term(a) @> match_term(b) on the extracted bf terms. An empty needle
bloom ({}) is contained by every value ('[1,2,3]'::jsonb @> '[]'::jsonb is
true), so a query term with no n-gram tokens matched every row in the table,
silently and with no error. A term has no tokens when its plaintext is below the
tokeniser's floor (e.g. a 2-character search string, which produces no trigrams
and encrypts to bf: []).
The wrapper now guards the empty-needle case with LIKE-shaped semantics:
| stored value bloom | needle bloom | LIKE analogue | result |
|---|---|---|---|
| empty | empty | '' LIKE '' | true |
| non-empty | empty | 'catty' LIKE '' | false (was true) |
| empty | non-empty | '' LIKE 'cat' | false |
| non-empty | non-empty | — | bloom containment |
Only the second row changed. An empty needle now matches only a value whose own bloom is also empty, instead of everything.
Why. This is the server-side backstop for an invariant the SDK already enforces client-side ("short needles are rejected rather than silently matching every row"). It holds regardless of which client constructed the term.
Who is affected. Anyone issuing a fuzzy match (col @@ \$1, or the by-name
eql_v3.matches(col, \$1)) where the query term can be a sub-floor value with no
n-gram tokens. In practice this surfaced as a free-text search over an encrypted
column returning the entire table (capped only by the query's LIMIT) where the
plaintext baseline returned a handful of rows.
What to do. Nothing — the fix is transparent for well-formed queries. No re-encryption, no schema change, no query rewrite. In particular:
- Normal queries still engage a
Bitmap Index Scanon the documented functional index (GIN (eql_v3.match_term(col))): the top-levelmatch_term(col) @> needleconjunct is preserved as an indexable qualifier, soWHERE col @@ \$1(a bind parameter, or a literal needle) inlines as before and the guard rides along as a cheap recheck filter. - One narrow exception:
eql_v3.matchesis now non-STRICT(aSTRICTSQL function whose body carries the guard'sAND/ORwill not inline) and references the needle twice, so a needle supplied as an uncorrelated subquery —WHERE col @@ (SELECT …)— no longer inlines and falls back to a sequence scan. The result is still correct; only the index acceleration is lost. Supply the needle as a bind parameter or a literal (the normal shapes) to keep the index. - Stored empty blooms (legitimate sub-floor values already in a column) are
unaffected: they never matched a populated needle before and still don't, and
matches/NOT matchescontinue to partition the non-NULL rows exactly as before.
Verification.
-- Internal reproduction of the bug (a populated value @> an empty needle):
SELECT eql_v3_internal.bloom_filter('{"bf":[10,20,30]}'::jsonb)
@> eql_v3_internal.bloom_filter('{"bf":[]}'::jsonb); -- true (raw containment, unchanged)
-- But the guarded operator no longer matches:
SELECT eql_v3.matches(
'{"v":"3","i":{},"c":"x","bf":[10,20,30]}'::jsonb::public.eql_v3_text_match,
'{"v":"3","i":{},"c":"x","bf":[]}'::jsonb::public.eql_v3_text_match); -- false (was true)
-- An empty needle still matches an empty value ('' LIKE ''):
SELECT eql_v3.matches(
'{"v":"3","i":{},"c":"x","bf":[]}'::jsonb::public.eql_v3_text_match,
'{"v":"3","i":{},"c":"x","bf":[]}'::jsonb::public.eql_v3_text_match); -- true
Rollback. Not applicable — the change only narrows a degenerate empty-needle result set that was never intentional. A prior release is the only way back to the match-everything behaviour, which is the bug being fixed.