awkrs compatibility vs BSD awk, mawk, and gawk
August 30, 2026 · View on GitHub
This document is a feature matrix, not a proof of correctness. awkrs does not claim bit-identical behavior, zero defects, or complete coverage of every extension in three other implementations. Where behavior is unspecified by POSIX (random number sequences, hash iteration order, subtle printf rounding), differences are expected.
Legend
| Cell | Meaning |
|---|---|
| Match | Intended to follow the reference; covered by tests or explicit design. |
| Part | Subset, different edge cases, or alternate diagnostics. |
| Ext | Extension in that engine; POSIX awk may lack it. |
| No | Not supported or incompatible. |
| — | Not applicable. |
References: special variables and builtins lists in src/compiler.rs (SPECIAL_VARS) and src/namespace.rs (BUILTIN_NAMES, SPECIAL_GLOBAL_NAMES). CLI surface in src/cli.rs.
1. Executive summary
| Topic | awkrs stance |
|---|---|
| POSIX core | Large subset implemented; -P/posix toggles some ordering rules (e.g. for (i in a) without gawk-style PROCINFO["sorted_in"] sorting). |
BSD awk (e.g. nawk) | Many gawk-only features in awkrs are not in BSD awk; matrix below marks Ext for gawk. |
| mawk | Fast awk; extension set differs; awkrs accepts some -W tokens for CLI compatibility only. |
| gawk | Highest overlap; awkrs implements many gawk builtins and globals directly or as Rust builtins (see src/gawk_extensions.rs). |
@load | awkrs inlines .awk sources or maps known gawk module names; does not load arbitrary .so extensions (src/source_expand.rs). |
Parallel records (-j) | awkrs-only execution path when the program is parallel-safe (parallel::record_rules_parallel_safe); can diverge from any sequential reference. |
2. Command-line interface
| Flag / option | POSIX awk | BSD awk | mawk | gawk | awkrs |
|---|---|---|---|---|---|
-f program file | Yes | Yes | Yes | Yes | Match — an empty program file is a program with no rules: it runs, reads nothing and exits 0. awkrs decided whether a program had been supplied from the assembled program bytes, so awk -f empty.awk data.txt took data.txt as the program text and died on a parse error, while awk -f empty.awk alone reported "no program given". An empty program on argv (awk '' data.txt) already worked. |
-F FS | Yes | Yes | Yes | Yes | Match — POSIX defines -F sepstring as the assignment FS=sepstring, so the value gets the same escape processing as -v: -F '\t' is a one-character tab and -F '\\.' the two characters \. (a regex for a literal dot). awkrs stored the argument verbatim, so length(FS) was 2 for the tab and backslashes reached the regex compiler undecoded, while the identical value written -v FS='\t' came out right. |
-v var=val | Yes | Yes | Yes | Yes | Match — the value is processed as a string literal, so -v 's=a\tb\n' is 4 characters. awkrs stored the raw argument and answered 6; it now runs the value through the lexer's own escape table (lexer::unescape_assignment_value) rather than a second copy of the rules. The name has to be an awk identifier, optionally qualified as namespace::name; anything else is fatal (-v 1x=3 → "`1x' is not a legal variable name", exit 2), which is what all three references do. awkrs accepted every spelling and created a variable under a name no program can write, so a mistyped flag ran silently with the variable unset. |
| Program + file operands | Yes | Yes | Yes | Yes | Match — the operand - names standard input and reports FILENAME as -, so awk '…' data.txt - < more.txt appends a pipe to a file list. awkrs opened a file literally named - and died with a fatal "cannot open file"; getline < "-" was already redirected but the operand was not. |
var=value operand (assignment between files) | Yes | Yes | Yes | Yes | Match — an operand whose left side is a valid identifier assigns instead of naming a file, takes effect at the position it occupies, is a POSIX numeric string, and gets the same escape processing as -v. When every operand is an assignment the program still reads standard input. awkrs read them as file names and failed with cannot open file "v=1". |
-e / -i | — | — | Part | Yes | Match (multiple -e/-i) |
-b characters-as-bytes | — | — | — | Yes | Part (wired into runtime; verify vs release I/O paths) |
-c traditional | — | — | — | Yes | Part (reserved; stricter rules incremental) |
-C copyright | — | — | — | Yes | Match (prints the awkrs copyright line and exits) |
-d dump-variables | — | — | — | Yes | Part (dump after run; format awkrs-specific) |
-D debug | — | — | — | Yes | Part (listing/dump; not gawk’s debugger) |
-E exec | — | — | — | Yes | Match (program from FILE; remaining args are data) |
-g gen-pot | — | — | — | Yes | Match (awkrs POT generator) |
-I trace | — | — | — | Yes | No (parsed for CLI compatibility; no runtime effect — Args::trace is never read outside src/cli.rs) |
-k / --csv | — | — | — | Yes | Match (CSV / FPAT mode per Runtime::csv_mode) |
-l load / AWKPATH | — | — | — | Yes | Part (library search; no dynamic .so) |
-L lint | — | — | — | Yes | Part (lint_warn / fatal modes) |
-M bignum | — | — | — | Yes | Part (MPFR path; PROCINFO["prec"] / roundmode) |
-N use-lc-numeric | — | — | — | Yes | Match (formatting path; string→number still . per cli.rs docs) |
-n non-decimal-data | — | — | — | Yes | Match (set_numeric_parse_mode) |
-o pretty-print | — | — | — | Yes | Part (AST listing; not gawk’s --pretty-print text) |
-O optimize | — | — | — | Yes | Match (accepted; JIT on unless -s) |
-p profile | — | — | — | Yes | Part (awkrs wall-clock summary; not gawk profiler format) |
-P posix | — | — | — | Yes | Part (runtime flag; incremental strictness) |
-r re-interval | — | — | — | Yes | Match (no-op; intervals always on) |
-s no-optimize | — | — | — | Yes | Match (disables JIT) |
-S sandbox | — | — | — | Yes | Part (require_unsandboxed_io; system() blocked, etc.) |
-t lint-old | — | — | — | Yes | Part |
-W opt (mawk) | — | — | Yes | — | Part (help/version/exec= merged; other tokens ignored) |
-j / --threads | — | — | — | — | Ext (awkrs parallel pool) |
--read-ahead | — | — | — | — | Ext (stdin chunking with -j) |
--repl | — | — | — | — | Ext (reedline REPL; also the default on a bare tty) |
--lsp | — | — | — | — | Ext (Language Server over stdio) |
--dap [HOST:PORT] | — | — | — | — | Ext (Debug Adapter over stdio or TCP) |
--aot OUT | — | — | — | — | Ext (AOT-compile a BEGIN-only program to a native executable) |
--dump-tokens / --dump-ast / --dump-bytecode / --disasm | — | — | — | — | Ext (compiler introspection; each prints and exits) |
--tiers | — | — | — | — | Ext (reports which fusevm execution tier took each chunk) |
3. Source directives and namespaces
| Feature | BSD | mawk | gawk | awkrs |
|---|---|---|---|---|
@include "file" | No | No | Yes | Match (pre-parse expand) |
@load "x.awk" / bundled names | No | No | Yes | Part (.awk inline only; no .so) |
@namespace "ns" | No | No | Yes | Match (apply_default_namespace) |
ns::name identifiers | No | No | Yes | Match (lexer / namespace pass) |
4. Language constructs (selected)
| Construct | BSD | mawk | gawk | awkrs |
|---|---|---|---|---|
BEGIN / END | Yes | Yes | Yes | Yes |
BEGINFILE / ENDFILE | No | No | No | Yes (Ext) |
Range patterns (pat1,pat2) | Yes | Yes | Yes | Match |
Regex record patterns + compound (/re/ && expr) | Yes | Yes | Yes | Match (tests in tests/extra_integration.rs) |
next / nextfile / exit | Yes | Yes | Yes | Match |
User functions / return | Yes | Yes | Yes | Match |
delete a[k] / delete a | Yes | Yes | Yes | Match |
for (i in a) order | Unspecified | Unspecified | gawk sorts / sorted_in | Part (hash order vs PROCINFO["sorted_in"]; -P skips gawk ordering) |
switch | No | No | Yes | Yes |
Indirect function call (@ / function pointer) | No | No | Yes | Yes |
Coprocess (|&) | No | No | Yes | Part (runtime has coproc types; parity not guaranteed) |
getline variants | Yes | Yes | Yes | Part (incl. PROCINFO timeout/retry — see runtime.rs) |
5. Special variables
| Variable | BSD | mawk | gawk | awkrs |
|---|---|---|---|---|
NR FNR NF $0 $n | Yes | Yes | Yes | Match (invalid NF / negative fields fatal like gawk — tested) |
FS RS OFS ORS OFMT CONVFMT | Yes | Yes | Yes | Match — including streaming multi-char RS, regex RS, and paragraph mode (RS == "") over both stdin and files (trailing-newline trim matches gawk). |
FILENAME ARGC ARGV ENVIRON | Yes | Yes | Yes | Match — ARGV is consulted as awk walks the operands, not snapshotted at startup, so BEGIN { delete ARGV[1] } skips that file, setting an element to "" skips it, and rewriting one redirects the read. awkrs iterated its own argv and read a deleted file anyway. |
SUBSEP | Yes | Yes | Yes | Match |
RSTART RLENGTH | Yes | Yes | Yes | Match |
RT | No | Part | Yes | Match |
ARGIND | No | No | Yes | Match |
ERRNO | No | No | Yes | Match |
PROCINFO | No | No | Yes | Part (keys: sorted_in, read timeout, errno, FS mode, bignum, identifiers, etc. — not every gawk key) |
SYMTAB FUNCTAB | No | No | Yes | Part (reflection best-effort) |
FIELDWIDTHS FPAT | Part | Part | Yes | Match — FIELDWIDTHS accepts gawk's width, skip:width, and * tokens; the last entry is clamped to its declared width (no auto-extend), so any trailing input bytes are left unused like gawk. |
IGNORECASE | Part | Part | Yes | Match — applies to multi-char regex FS, match/sub/gsub/split/gensub, and ~/!~. Single-char string FS (and single-char split separator) is always literal, independent of IGNORECASE (gawk parity). |
BINMODE | No | No | Yes | Part |
LINT | No | No | Yes | Part |
TEXTDOMAIN | No | No | Yes | Part (gettext path) |
6. Built-in functions
Columns: P = POSIX / universal core, B = BSD awk, M = mawk, G = gawk extension (approximate; BSD may add some).
| Builtin | P | B | M | G | awkrs |
|---|---|---|---|---|---|
atan2 cos sin exp log sqrt int | * | * | * | * | Match (negative log/sqrt: warn + NaN like gawk — runtime::warn_builtin_negative_arg) |
rand srand | * | * | * | * | Part (sequence not guaranteed to match any one engine) |
length / length() | * | * | * | * | Match (bare length → $0 — parser.rs) |
index substr sprintf | * | * | * | * | Match |
match sub gsub split | * | * | * | * | Match / Part (regex engine = Rust regex; subtle differences possible). gsub(//, …) produces gawk's zero-width matches at every position; split(s, a, fs, seps) populates the 4th-arg seps array with the actual separator strings between fields. |
tolower toupper | * | * | * | * | Match |
system close | * | * | * | * | Match — system() flushes buffered stdout / pipes / files before invoking the subprocess; close() returns -1 for an unopened name and the exit code / 0 for a clean close (gawk parity, runtime::close_handle). A child killed by a signal reports 256 + signo from both (system("kill -TERM $$") → 271), the encoding all three references use; it lived only in close_handle and system() answered -1, so both now share runtime::awk_process_status. |
strtonum | *¹ | Part | Part | Yes | Match |
asort asorti | — | — | — | Yes | Match |
gensub patsplit | — | — | — | Yes | Part |
mktime strftime systime gettimeofday | — | — | Part | Yes | Part |
and or xor compl lshift rshift | — | — | — | Yes | Match |
isarray typeof mkbool | — | — | — | Yes | Match / Part |
intdiv intdiv0 | — | — | — | Yes | Match |
bindtextdomain dcgettext dcngettext | — | — | — | Yes | Part (gettext_util / stubs) |
chdir stat statvfs fts | — | — | — | Ext / Yes | Match / Part (gawk_extensions.rs) |
readfile ord chr sleep | — | — | — | Ext | Match (as builtins) |
revoutput revtwoway rename | — | — | — | Ext | Match |
inplace_tmpfile inplace_commit | — | — | — | Ext | Match |
writea reada | — | — | — | Ext | Match |
intercept intercept_proceed intercept_list intercept_remove intercept_clear | — | — | — | — | awkrs-only — aspect-oriented before/after/around advice on user-function calls (ported from zshrs; no POSIX/gawk counterpart). See §0x03 of the README. |
¹ strtonum appears in POSIX awk revision used by gawk; older texts omit it.
7. printf / print / numeric formatting
| Topic | awkrs |
|---|---|
%g / %G | Match — precision is total significant digits (C99/POSIX); the fixed-vs-e form decision uses the rounded exponent (so %.1g of 9.5 is 1e+01, not 10). Precision 0 is treated as 1. |
%u on negative values | Match — wraps via i64→u64 two's complement (gawk parity), not clamped to 0. |
0 flag on %s / %c | Match — POSIX says the flag is for numeric conversions only; awkrs pads with spaces for string/char conversions. |
Unknown conversion letters (%q, %v, …) | Match — emit the literal %X without consuming an argument (gawk parity). |
%a / %A hex float | Match (format_hex_float in src/format.rs; gawk parity confirmed) |
Non-finite floats (±inf, ±nan) across %f/%e/%g/%a | Match (gawk-style +inf / -inf / +nan / -nan, with INF / NAN for uppercase variants — format_non_finite in src/format.rs) |
print of non-finite values | Match — format_number in src/runtime.rs emits the same +inf / +nan spelling so print x and printf "%s", x agree |
LC_NUMERIC (-N) | Part (documented split: format vs parse) |
%' flag thousands grouping | Match — consults localeconv()->thousands_sep regardless of -N (gawk parity). Empty in LC_ALL=C → no grouping; "," in en_US.UTF-8 → comma grouping. |
== / < / > of Num vs string literal | Match — string-compare fallback stringifies the number via CONVFMT (not the default %.6g). E.g. BEGIN{CONVFMT="%.2f"; print 3.14159=="3.14"} prints 1. |
a % 0 / a %= 0 | Match — fatal "division by zero attempted in `%'" (was previously NaN). |
Numeric coercion of "inf" / "nan" | Match — bare special names coerce to 0; only signed three-letter inf / nan (case-insensitive) are accepted. "+infinity" is rejected like in gawk. |
lshift / rshift / compl negative args | Match — fatal "negative values are not allowed". |
typeof($field) of noisy numeric text (e.g. "42abc") | Match — reports "string" (numeric prefix alone is not enough); field comparisons against numbers use string-compare. Pure-numeric text ("42") still reports "strnum". |
match(str, re, arr) start/length subscripts | Match — writes arr[i, "start"] (1-based char index) and arr[i, "length"] for each successful submatch; unmatched optional groups have NO entries. |
mktime(spec [, utc]) | Match — optional second argument forces UTC interpretation when truthy; one-arg form remains local-time. |
Assignment in ternary else-branch (1 ? x=1 : x=2) | Match — the else-branch parses as an assignment-expression (gawk grammar). Previously rejected as "invalid assignment target". |
asort / asorti on unassigned name | Match — treats missing slot as an empty array (returns 0). Scalar values still raise the "first argument is not an array" fatal. Compiler tracks these positions for array-slot promotion. |
Numeric == precision | Match — bit-exact (POSIX). Previously used a fuzzy f64::EPSILON tolerance, so 0.1 + 0.2 == 0.3 returned true (the difference is ~5.55e-17, below EPSILON). Now matches gawk's 0. |
Paragraph-mode RT (RS == "") | Match — captures the FULL run of trailing newlines from the last content line plus the blank lines separating records (b\n\nc → RT == "\n\n"). The last record also captures EOF-trailing newlines into RT. |
PROCINFO["strftime"] default | Match — "%a %b %e %H:%M:%S %Z %Y" (gawk's date(1)-equivalent default), not "%c". |
printf("fmt", a, b) function-call form | Match — equivalent to printf "fmt", a, b. Previously rejected as "parenthesized comma list is not allowed". Mixed paren-args + bare args (printf(a,b), c) still rejected. |
| Builtin called with wrong arity | Match (no panic) — uniform "N is invalid as number of arguments for X" error across tolower, toupper, index, substr, length, system, close, rand, srand, asort, asorti, split, match, sub, gsub, exp, log, sin, cos, sqrt, atan2, int. Earlier awkrs panicked on some, silently ignored extras on others, and used a non-gawk wording on the math functions. |
delete x / delete x[k] on a scalar | Match — fatal "attempt to use scalar `x' as an array". Unassigned names still silently no-op (POSIX). |
| What counts as a POSIX numeric string | Match — only input-derived values (fields, getline targets, split elements, ARGV/ENVIRON) are strnum. A computed string never is, so substr("065",1,2) == 6, sprintf("%s","06") == 6, toupper("06") == 6 and $1 "" == 6 are all string compares and answer 0. awkrs previously carried the strnum-capable Value::Str out of substr/sprintf/toupper/tolower/gensub/strftime and out of concatenation, and answered 1. typeof reports "strnum" on the same rule the comparisons use. |
| Empty record field count | Match — an empty record has NF == 0 under every FS. The single-char and regex splitters previously pushed one empty range and reported NF == 1 for a blank line under FS=":". |
sub / gsub that matches nothing | Match — the target is left completely untouched, so an uninitialized variable stays uninitialized (sub(/x/,"y",z); z == 0 is still 1) and a number stays a number. The unchanged string used to be written back, demoting strnum to string. |
Bare return (and falling off the end of a function) | Match — yields the uninitialized value, equal to both 0 and "". |
split(s, a, /re/) with a regex literal separator | Match — always a regex, so the FS shorthands never apply: / / is one literal space (split(" a b ", A, / /) is 7, not 2) and /./ is any-character. An empty separator (// or "") still splits into characters. |
Multidimensional subscripts and CONVFMT | Match — each subscript converts like a single subscript: integral values exactly, everything else through CONVFMT. CONVFMT="%.2f"; A[1.234,2] keys on 1.23<SUBSEP>2. |
CONVFMT subscript in every subscript operation | Match — the CONVFMT rendering is the array's identity, so k in a, delete a[k], a[k] op= v, a[k]++/-- and typeof(a[k]) all key exactly as the a[k] = … that created the entry. CONVFMT="%.2f"; x=1.23456; A[x]=5; A[x]+=1 leaves one entry A["1.23"] of 6. awkrs previously converted the key differently in those five operations, so x in A was false and the compound assignment created a second entry under the full-precision spelling. |
CONVFMT in every string builtin | Match — POSIX gives one rule for turning a number into a string outside print, and length, substr, index (both operands), toupper, tolower, split's subject, sub/gsub's target and replacement, and gensub's subject all follow it. CONVFMT="%.2f"; x=1.23456 gives length(x)==4, substr(x,3)=="23", index("a1.23b",x)==2 and leaves gsub(/3/,"9",x) as 1.29. awkrs previously read the number at full f64 precision in all of them (length(x) was 7, gsub left 1.29456), so CONVFMT was honoured by concatenation, comparison and subscripts but ignored one call away. |
CONVFMT for a dynamic regex | Match — a dynamic regex is the string value of its operand, so a numeric pattern converts the same way: CONVFMT="%.2f"; x=1.23456; "a1.23b" ~ x is true, and match, split's separator, sub/gsub's pattern and patsplit agree. Only the subject side of ~/!~ used to convert this way, so "a1.23b" ~ x was false while "a1.23b" == x — the same coercion one operator apart — was true. |
CONVFMT for a getline redirect target | Match — getline … < expr and expr | getline name a file or command as a string, so a numeric operand opens the CONVFMT rendering. awkrs previously looked for the full-precision spelling and returned −1. |
When the CONVFMT coercion is performed | Match — at the point of use, never cached at assignment: CONVFMT="%.2f"; x=1.23456; a=length(x); CONVFMT="%.4f"; b=length(x) yields 4 6 in all three references. Integral values bypass the format entirely, and an input-derived value keeps its original text ($1 of the record 1.23456 is still 7 characters under "%.2f") — only computed Num/Mpfr values are rendered. |
printf "%c" of a numeric string | Match — a strnum operand is numeric, so echo 65 | awk '{printf "%c", \$1}' prints A, while the string literal "65" prints 6. |
printf negative * precision | Match — ISO C: a negative precision argument is taken as if the precision were omitted, so printf "%.*f", -2, 3.14159 prints 3.141590. awkrs previously clamped it to 0. |
; as a control-flow body | Match — POSIX makes ; a statement, so if (c) ;, while (c) ;, for (…) ; and else ; all parse. awkrs previously rejected every one of them. |
split(s, a) on an empty string | Match — the target becomes an (empty) array, so typeof reports "array" rather than "untyped". |
Scalar used as array (x[k]=…, x[k], k in x, for (k in x)) | Match — fatal "attempt to use scalar x' as an array". Earlier awkrs silently auto-promoted on write, returned empty on read, returned 0 from in`, and ran zero iterations on for-in. |
printf "%u" of values past | Match — falls back to %g-style formatting (2^65 → "3.68935e+19"). The exact boundary still prints as the u64::MAX digit string. Earlier awkrs saturated all over-u64 values at u64::MAX. |
MPFR (-M) | Part (precision / rounding via PROCINFO) |
printf precision on d i o u x X | Match — the precision is a minimum digit count reached by zero-padding the magnitude, and while it is present the 0 flag is ignored, so %08.2d of 42 is 42. The # prefix goes outside that padding (%#.5x of 255 is 0x000ff), and on %o it raises the precision far enough to force a leading zero, so %#.0o of 0 is 0 where plain %.0o is empty. awkrs previously ignored the precision entirely on o/u/x/X and let the 0 flag win on all six. |
printf rounding at an exact half | Match — %e/%f/%g round the exact binary value, halves to even, as C does: %.1g of 2.5 is 2 and of 4.5 is 4, while %.2g of 1.35 is 1.4 because 1.35 is a shade above the half. %g previously rounded in arithmetic (scale, f64::round, unscale), which both rounded halves away from zero and moved the value before rounding — %.1g of 0.15 came out 0.2 because 0.15 * 10 is exactly 1.5 in f64 even though 0.15 is not. |
| Byte-exact strings | Match — values hold a byte string (AwkStr), so a byte that is not part of valid UTF-8 survives $0, fields, substr, index, length, toupper, concatenation, array subscripts, split elements, ~, sub/gsub/gensub, printf/sprintf and print, and is accepted in the program text itself. See the byte-exact-strings entry in §9 for the verified matrix and what still renders. |
| Regex acceptance set | Match — ~ accepts what the references accept rather than what Rust's parser does. (?:…) and (?i)… are fatal (ERE has no non-capturing group and no inline flags, so the ? has no preceding expression); )), {, a{ and \Qa\E are literal text; a reversed range like [z-a] is the characters as a set (mawk and one-true-awk, against gawk's fatal). Inside a bracket the character escapes keep their character but the class shorthands do not name a class — [\t] is a tab while [\w] is the letter w, matching neither a backslash nor a digit. |
-M float literals | Match — a literal holds the decimal that was written, not the f64 nearest it, so ten additions of 0.1 come to exactly 1 as in gawk. %.*f and %.*e emit the number of digits asked for; rug's formatting precision counts significant digits, so %.4f of 2.5 used to print 2.500 and %.0f the whole binary expansion. |
printf "%c" of a number | Match — the low byte in a single-byte locale (233 → \351, 955 → \273), the UTF-8 encoding of the code point in a UTF-8 locale. That is gawk in both locales and mawk / one-true-awk in the C locale, which is the only rule no reference contradicts. awkrs previously emitted the encoding regardless of locale. |
| Output already printed when a fatal is raised | Match — a fatal does not un-print what the program already wrote, at any phase. Verified on BEGIN { print "A"; printf "%d\n" } (a format that outruns its arguments, which gawk, mawk and one-true-awk all treat as fatal): every reference writes A and exits 2, and so does awkrs. Output is buffered, and the flush used to happen on a normal exit and on the record loop's error path only, so a fatal raised in BEGIN, BEGINFILE, ENDFILE or END dropped the buffer on the way to the process exit and the run appeared to print nothing — silent data loss in a pipeline. Every phase now flushes what it printed before reporting the fatal (flush_if_err! in src/lib.rs), and the diagnostic still wins over any error the flush itself hits. |
8. Regular expressions
| Topic | awkrs |
|---|---|
| Engine | Rust regex crate (not literal GNU regex copy). |
Interval quantifiers {m,n} | Enabled ( -r is no-op). |
IGNORECASE | Supported for split/match contexts that consult runtime. |
. matches \n | Match — all built regexes use dot_matches_new_line(true) (gawk ERE convention). |
Backreferences in patterns (e.g. (.)\1) | No — Rust regex is linear-time and does not support pattern-side backrefs. (Backrefs in replacement text via gensub \1..\9 and & are supported.) |
POSIX character classes ([[:digit:]], etc.) | Match |
| NUL bytes / binary | Part (-b / BINMODE — exercise before relying on). |
9. Known intentional or unavoidable divergences
-
JIT (fusevm's Cranelift, via
src/fusevm_bridge.rs): When enabled, must match interpreter; if a mismatch is found, treat as a bug in JIT, not as "gawk is wrong." Eligibility is an allowlist of numeric ops (is_fusevm_eligible); AWK-specific ops including~/!~regex match lower tofusevm::Op::Extendedand run on the interpreter, not the JIT. -
Parallel mode (
-j): Record rules may run concurrently; programs with side effects or dependence on global order are unsafe. -
Dynamic extensions: gawk
@load "foo.so"has no equivalent in awkrs. -
Process / locale / OS:
PROCINFO["platform"]mapping usesposix/mingwstyle (procinfo.rs), not necessarily gawk’s host string for every OS. -
For-in order: Without
-P, gawk-stylesorted_inand user comparators apply; hash order still differs across engines when sorting is off. -
Exit status: fatal conditions (runtime faults, an unreadable
-ffile, an input file that cannot be opened, output-redirection I/O errors) exit 2, matching all three reference awks. Parse diagnostics exit 1, matching gawk; mawk and one-true-awk exit 2 there. SeeError::exit_statusinsrc/error.rs. -
Builtin arity is checked at run time, not at parse time.
substr(),substr("a"),length("a","b"),index("a")andsin()are all rejected by gawk before the program runs — exit 1, nothing executed — while awkrs compiles them and raises when the call is reached, so aBEGINblock that printed first has already written its output and the exit is 2 rather than 1.split()is the one already rejected at parse time. The check needs to move to the compiler, where the reference does it; the diagnostic text differs in either case, so the divergence is when it is reported and what has run by then. -
printf "%c"with an empty string: emits nothing, matching POSIX ("the first character of the string value") and one-true-awk. gawk and mawk emit a NUL byte. -
"0x10" + 0:0, matching POSIX, gawk and mawk. one-true-awk'sstrtodaccepts the0xprefix and yields 16. -
Namespace-qualified command-line assignments:
-v ns::x=1assigns into the namespace as gawk does, but the operand form (awk '…' ns::x=1) is still read as a file name, and gawk'sawk::alias for the default namespace is not recognised (-v awk::x=1leavesxunset where gawk sets it). mawk and one-true-awk cannot parse a qualified name at all, so there is no three-way rule here — only a gawk gap. -
printfunsigned conversions of a negative argument (%x%o%X): converted as a 64-bit unsigned value (printf "%x", -3→fffffffffffffffd), matching gawk. one-true-awk and mawk both print0. -
OFMT/CONVFMTset to a non-floating-point conversion (e.g."%d"): undefined by POSIX, and all three references differ — one-true-awk ignores the setting, mawk prints a garbage integer, gawk warns and prints0. awkrs produces gawk's value without the warning. -
Paragraph mode field splitting: with
RS == ""a single-characterFSgains<newline>as an additional separator (gawk and one-true-awk both do this; mawk does not). A regexFSis left alone in every reference, so an embedded newline stays inside the field. -
Character semantics are UTF-8, not locale-driven:
length/substr/index/toupper/tolower— andmatch'sRSTART/RLENGTH, which report 1-based character positions in the same unit — count and fold Unicode scalar values regardless ofLC_ALL, solength("é")is 1 even underLC_ALL=Cwhere gawk reports 2.-bselects byte semantics explicitly, and it now governs case folding as well as counting: under-b,toupper/tolowerfold ASCII only, sotoupper("café")isCAFéandtoupper("Straße")isSTRAßE— reproducing all three references in the C locale, and keeping the result the same length as its input where Unicode'sß→SSwould grow it.-bpreviously switched the counting builtins but left folding Unicode-aware, so a single-brun reportedlength("café") == 5whiletoupper("café")returnedCAFÉ— the byte world and the character world in one program. Without-bthe fold is the Unicode simple (1:1) mapping, which is the one gawk applies:toupper("ß fi ʼn")comes back unchanged andtolower("İ")isi, so a fold never changeslength(). awkrs previously used the full mapping fromSpecialCasing.txtand grew the string (ß→SS,fi→FI,ʼn→ʼN,İ→iplus a combining dot), which no reference awk does in any locale.printf "%c"of a numeric argument is the one part of this entry that is not locale-independent, because the references do not agree on a single answer: in a single-byte locale gawk, mawk and one-true-awk all emitN & 0xFF, and in a UTF-8 locale gawk emits the UTF-8 encoding of the code point while the other two stay on bytes. awkrs follows the locale, which matches gawk in both and the other two in the one they agree with it on. The rule in a single-byte locale is the low byte for anyN, not just below 256:233→e9,300→2c,511→ff,955→bb,1000→e8. The only values the references split on there are the ones whose low byte is00(256,512,1024,65536): gawk and mawk emit the NUL, one-true-awk emits nothing — the same split this section already records for%cof an empty string. awkrs used to emit the UTF-8 encoding regardless of locale, which was a genuine C-locale gap rather than a reference disagreement; see the byte-exact-strings entry below. -
A function name used as a variable (
function f(){} BEGIN{ f = 1 }) is rejected before the program runs, matching all three references. The status is a three-way split: gawk exits 1 (it diagnoses at parse time), mawk and one-true-awk exit 2. awkrs exits 2 with the other two, because the check runs invalidate_programafter parsing rather than inside the grammar. A function parameter that shadows a function name is legal everywhere and stays legal here. -
close()of a pipe returns the command's exit status, matching gawk and mawk; one-true-awk returns 0. A command killed by a signal reports256 + signoin gawk, mawk and one-true-awk alike, and awkrs matches fromclose()andsystem()both. -
print > "-"creates a file named-, matching mawk and one-true-awk; gawk writes to standard output instead. The input side has no such split —getline < "-"reads standard input in all three references, and in awkrs. -
print > "/dev/stdout"writes to the program's own standard output, interleaved with plainprintin program order, which is what all three references do.close("/dev/stdout")is where they part: gawk flushes and answers 0 with the stream still usable, while mawk and one-true-awk really close descriptor 1 — one-true-awk silently drops every later line and mawk reportswrite failure (Bad file descriptor). awkrs follows gawk, the only reading under which a program can keep printing. -
When an output pipe flushes pending standard output: opening
print … | "cmd"flushes whatever awk has buffered, so the child's output cannot overtake lines the program printed first.close()does not flush again — that is mawk and one-true-awk's timing; gawk flushes at both points, soprint "1"; print "P1" | "cat"; print "2"; close("cat")orders2before the child's output in gawk and after it here. -
A regex
RSthat can match the empty string: zero-length matches are not separators.RS="z?"readsabcas a single record in gawk, mawk, one-true-awk and awkrs. Where a pattern mixes empty and real matches the references split:RS="x*"overaXbxcgives mawk and one-true-awk two records (aXb,c) by ignoring the empty matches, while gawk emits an empty record per position; awkrs follows mawk and one-true-awk, which is the same rule that makes theRS="z?"case unanimous. -
getline < <directory>reads the directory's entries, one file name per record, sorted. This is an awkrs extension (the same datareaddir()returns); gawk and mawk return −1 for a directory and one-true-awk reports an I/O error. A script that means to read a file and is handed a directory therefore sees records here where the references see a failure. -
Record splitting reads a numeric
FS/RSwithoutCONVFMT— the one part of theCONVFMT-coercion rule above that is still open, and it is deliberately partial.BEGIN { CONVFMT="%.2f"; FS=1.23456 }splits records on the full-precision1.23456where all three references split on1.23;RSbehaves the same way, andOFS/ORSmatch mawk rather than gawk and one-true-awk. The explicit separator forms are all correct —split(s, a, fs), and also the two-argumentsplit(s, a)that falls back toFS— so within awkrs the sameFSvalue can separate asplit()call and a record differently. That inconsistency is the smaller of the two available ones: readingFSis not a single site (the record splitter, the$0-assignment path, the field-rebuild path and the JIT host each read it independently, and the splitter caches the value per record), so converting at only some of them would put the interpreter and the JIT tier into disagreement, which §9 treats as a bug in its own right. Converting at assignment is not the answer either:print FSusesOFMTin every reference (CONVFMT="%.2f"; OFMT="%.3f"; FS=1.23456; print FSprints1.235), so the variable has to keep its numeric identity.length(FS)and the other string builtins are already correct, because those go through the coercion above. Repro:printf 'a1.23b\n' | awk 'BEGIN{CONVFMT="%.2f"; FS=1.23456}{print NF}'—2in gawk/mawk/one-true-awk,1here. -
typeofof a never-assigned function parameter: gawk turns a parameter from"untyped"into"unassigned"the first time it is read; awkrs reports"untyped"throughout. Global scalars and array elements do make that transition (a per-slot "touched" bit inRuntime::slot_touched); function locals live in a per-call frame map that has nowhere to record it, and adding a parallel per-frame structure would cost work on every user-function call for one value of one gawk-only introspection builtin. -
typeof(\$0)before any record is read: gawk reports"unassigned", awkrs reports"string"—$0starts as an empty record rather than a distinct never-assigned state. -
Byte-exact strings. awkrs values hold an
AwkStr(src/awkstr.rs), a byte string, so a byte that is not part of valid UTF-8 travels through the program the way gawk, mawk and one-true-awk pass it. It used to be a RustString, which by construction cannot hold one, so every such byte becameU+FFFDon the way in — three bytes out where all three references emit the one they were given, and a value that could never match the byte it came from.Vec<u8>was chosen overbstr::BString(a dependency for an API written here in a few hundred lines, and it derefs to[u8]too, so it breaks the same call sites) and over an enum ofUtf8(String) | Bytes(Vec<u8>)(one logical string with two representations: equality, hashing and array-subscript identity would all have to normalise across the variants, and a site that forgot would answer wrong silently).Verified against all three references under
LC_ALL=C, on the inputa\351b c\377d— every line below is unanimous, andtests/posix_parity_regressions.rspins them over bytes, because a lossy comparison would turn the byte under test intoU+FFFDand hold against exactly the bug it exists to catch:what result { print },{ print \$1 },{ print \$2 }the bytes, unchanged { print length(\$0), NF, length(\$1), index(\$0,"b") }7 2 3 3{ print substr(\$0,2,1) }the single \351{ print toupper(\$0) }A\351B C\377D— there is no case mapping for an unpaired byte, andU+FFFDis not one{ printf "%s\n", \$0 }and{ printf "%c\n", substr(\$0,2,1) }the bytes { x = \$1 "-" \$2; print x }and{ x = sprintf("%s",\$0); print x }the bytes { a[\$1]=1; for (k in a) print k }the subscript that was stored { split(\$0,p," "); print p[2] }c\377d{ print (\$0 ~ /^a.b$/) }ona\351b1{ print (\$1 ~ \$2) }where$2is\3511length("\351")andprintf "%s", "\351"1, and the single byte —"\xNN"/"\NNN"in a literal name a byte, not a character{ print (\$0 ~ "\xe9") }ona\351b1{ gsub(/b/,"Z"); print }and the same forsub,$1, a variable and an array elementa\351Z c\377d— the bytes it did not replace{ gsub(/[ab]/,"[&]"); print \$1 }[a]\351[b]—&stands for a matched bytea program holding the byte in a string literal, a regex literal or a comment runs, from argvand from-falike-v x=a\351bthe three bytes the byte in code position syntax error, as in all three a binary line \377\376\0Aintact (one-true-awk truncates at the NUL; gawk and mawk do not, so the majority rules) printf "%c"of a number follows the locale, which is the only behaviour no reference contradicts: in a single-byte locale all three emitN & 0xFF(233→\351, and it stays the low byte above 255 —300→\054,955→\273), while in a UTF-8 locale gawk emits the encoding of the code point and the other two stay on bytes. The regex engine takes the same switch — with Unicode mode off,.and the character classes work in single bytes, which is what all three do in a single-byte locale.locale_numeric::ctype_is_utf8resolvesLC_ALL, thenLC_CTYPE, thenLANG, the precedence gawk and one-true-awk were both observed to use.What is still rendered rather than carried, each for a named reason:
- The fusevm backend's
sprintfandawk_keys.fusevm::Valuecarries aString, so a value crossing into that backend is rendered.printfon that backend is unaffected — it writes intoprint_bufdirectly. Closing this needs a byte string in fusevm itself, which is upstream of this crate, and it is a shared-VM limit rather than an awkrs one: the same ceiling shows up as lone surrogates in node-js'sJSON.stringifyand in tclrs's%c. rust { }blocks and@include/@namespace/@load. Those four rewrite the program text before lexing, so they cannot run over bytes they cannot name. A program that needs both a raw byte and one of them is refused by name rather than silently losing one; a program with neither — which is every ordinary one — is unaffected. The introspection flags (--dump-tokens,--dump-ast,--dump-bytecode,--disasm,--tiers) and the AOT builder take source as&strand so see a rendering; none is on the execution path.
- The fusevm backend's