Benchmarks

Generated by composer bench. Do not edit by hand - re-run the command.

The question underneath all of this is the one the compile-time generics discussions keep stalling on: monomorphization is assumed to cost a copy of the code per instantiation. Here it does not, because the specialized class entry shares its method bodies with the template, and these tables are the evidence.

Environment

• PHP — 8.4.19
• Thread safety — NTS
• Debug build — no
• Architecture — x86_64 (64-bit)
• OPcache — loaded, disabled for CLI
• JIT — off
• zend.assertions — -1
• CPU — Intel(R) Xeon(R) Processor @ 2.10GHz
• sizeof(zend_class_entry) — 520 bytes
• sizeof(zend_op_array) — 256 bytes
• sizeof(zend_property_info) — 72 bytes
• sizeof(zend_arg_info) — 32 bytes
• sizeof(zend_op) — 32 bytes

Struct sizes are read from the running engine. They are part of the result: a byte count only means something next to the size of the things being counted.

Memory per specialization

What does one specialization cost, and does that cost depend on how large the methods are?

slot	M	K	opcodes/method	bytes/specialization	once called	fitted slope	struct floor	overhead	r2
none	1	4	10	3016	0	3416	848	3.56x	0.99919
none	1	200	206	3112	0	3544	848	3.67x	0.99822
none	8	4	10	7592	0	8797	2640	2.88x	0.99558
none	8	200	206	7592	0	8426	2640	2.88x	0.9828
none	32	4	10	23912	0	27907	8784	2.72x	0.99279
none	32	200	206	23912	0	28541	8784	2.72x	0.96822
property-only	1	4	12	3112	655	3302	912	3.41x	0.99959
property-only	1	200	208	3112	0	3302	912	3.41x	0.99959
property-only	8	4	12	7592	655	7782	3152	2.41x	0.99993
property-only	8	200	208	7592	655	7782	3152	2.41x	0.99993
property-only	32	4	12	23912	655	24102	10832	2.21x	0.99999
property-only	32	200	208	23912	655	38222	10832	2.21x	0.93731
class-parameter	1	4	12	3448	0	3638	912	3.78x	0.99966
class-parameter	1	200	208	3448	0	3638	912	3.78x	0.99966
class-parameter	8	4	12	10280	655	10470	3152	3.26x	0.99996
class-parameter	8	200	208	10280	0	10470	3152	3.26x	0.99996
class-parameter	32	4	12	34664	1310	34854	10832	3.20x	1
class-parameter	32	200	208	34664	655	34854	10832	3.20x	1
builtin-parameter	1	4	12	3640	0	3830	1296	2.81x	0.99969
builtin-parameter	1	200	208	11448	0	11638	7568	1.51x	0.99997
builtin-parameter	8	4	12	11816	0	12006	6224	1.90x	0.99997
builtin-parameter	8	200	208	74280	0	74470	56400	1.32x	1
builtin-parameter	32	4	12	40808	655	50260	23120	1.77x	0.89312
builtin-parameter	32	200	208	290664	655	290854	223824	1.30x	1

• none: growing the bodies by 50x changed the cost per specialization by at most 1.03x. Bodies are shared, so body size does not appear in the cost.
• property-only: growing the bodies by 50x changed the cost per specialization by at most 1.00x. Bodies are shared, so body size does not appear in the cost.
• class-parameter: growing the bodies by 50x changed the cost per specialization by at most 1.00x. Bodies are shared, so body size does not appear in the cost.
• builtin-parameter: growing the bodies by 50x changed the cost per specialization by at most 7.12x. This is the one slot kind that un-shares the opcode array, so it is expected to track body size - and the number says how much that costs.
• once called is part of the price, and it is quantized. A copied method starts with a null run-time cache and a null static-variable table, and the engine materializes both on first call - so a freshly minted specialization is cheaper than one an application is using. The engine takes that memory from its arena in 64 KiB chunks, which over 100 specializations makes the column resolvable only to about 655 bytes: a 0 means "below that", not "free".
• The headline column is the median of the per-specialization deltas, not a least-squares slope. EG(class_table) is a hash and rehashes as it fills, which puts a step into the series that a straight line has to absorb; a median ignores it and a slope does not. The fitted slope and its r2 are kept beside it so the two can be compared - where they disagree, the rehash is why.

Monomorphization vs code generation

How does a class-entry copy compare to emitting and compiling the specialized source?

M	K	opcodes/method	specialize (bytes)	codegen (bytes)	ratio
1	4	12	3232	1680	0.5x
1	200	208	3232	13456	4.2x
8	4	12	10064	6160	0.6x
8	200	208	10064	100368	10.0x
32	4	12	34448	22480	0.7x
32	200	208	34448	399312	11.6x

• At K=4 statements per method, code generation used between 0.5x and 0.7x what a specialization used. Below 1x the copy is the more expensive of the two: a class entry has a fixed cost that a body this small never earns back.
• At K=200 statements per method, code generation used between 4.2x and 11.6x what a specialization used.
• The direction is what matters: the ratio is a function of body size, because only one of the two approaches copies the body. Everything else was held constant between the two columns.
• The comparison is memory only. Code generation also pays a full compile per specialization, which the latency scenario measures separately.

Specialization latency

How long does minting a specialization take, and how long does asking for an existing one take?

slot	M	K	median (us)	p95 (us)	memoized (us)
class-parameter	1	20	285.9	424.6	1.559
class-parameter	8	20	1051.4	1338.4	1.599
class-parameter	32	20	4012.4	4839.8	2.721
builtin-parameter	1	20	281.5	424.9	1.538
builtin-parameter	8	20	995.5	1130.8	1.606
builtin-parameter	32	20	3414.9	3949.6	1.548

• class-parameter: about 129 us fixed per specialization plus 121 us per own method (r2 1.000). Every method pays it whether or not it has a substituted slot, because every one has its zend_op_array struct copied.
• builtin-parameter: about 184 us fixed per specialization plus 101 us per own method (r2 1.000). Every method pays it whether or not it has a substituted slot, because every one has its zend_op_array struct copied.
• Those are the numbers behind the "specialize at worker boot, not per request" advice: minting is not a hot-path operation.
• The work is a long sequence of individual FFI calls from userland rather than one engine-side copy, which is where most of that time goes. It is a property of driving the engine through FFI, not of monomorphization.
• Asking again costs about 1.8 us regardless of shape - the factory resolves and mangles the name, then finds it in the cache. That is what makes of() safe to write wherever a generic type is needed.
• Timings are sensitive to zend.assertions: with assertions on, z-engine verifies every relocated operand of a copied opcode array. The environment block above records which setting produced these numbers.

Scaling out to many specializations

What do 1000 live specializations cost, and do they slow the class table down?

metric	value
Specializations	1000
Template shape	M=4 K=20 P=1 class-parameter
Total memory	5.9 MiB
Memory per specialization	6.1 KiB
RSS delta	6.1 MiB
Total time	664.6 ms
Time per specialization	664.6 us
Class lookup before	66.4 ns
Class lookup after	58.2 ns

• Memory per specialization here is a plain total divided by N, not a fitted slope, so it carries the one-off costs the memory scenario deliberately cancels. Read that scenario for the marginal number and this one for the bill.
• The class table is a hash, so the lookup figures are expected to match. They are measured because "the class table gets slow" is the objection this approach would otherwise have to answer with an assurance.

Property writes in steady state

Does writing a specialized property cost more than writing a compiled one?

property	subject	ns/call	vs hand-written
class-typed	specialization	80.8	2.55x
class-typed	hand-written	31.7	1.00x
class-typed	mixed (unchecked)	29.3	0.93x
builtin-typed	specialization	27.9	0.99x
builtin-typed	hand-written	28.2	1.00x
builtin-typed	mixed (unchecked)	27.7	0.98x

• A class-typed specialized property wrote at 2.55x the cost of the same property on a compiled class. That is not parity, and the next lines are the cause.
• A builtin-typed specialized property wrote at 0.99x the cost of the same property on a compiled class. That is parity within run-to-run variation: the engine tests the value against a type mask and never looks at where the class came from.
• Lengthening the type argument's class name from a short one to 120 characters moved the write by +47.4 ns/call (78.8 -> 126.2). A compiled class is flat under the same change.
• That probe uses a class-typed property. A builtin-typed one has no name to resolve, which is why it sits at parity above and why the attribute form is the cheaper of the two whenever the type argument is a builtin.
• Cost that tracks name length is cost spent resolving the name, which means the specialization is looking its property type up on every write while a compiled class resolves it once. The engine reaches that fast path through a class-entry cache attached to interned strings, and the name z-engine writes into a substituted type is created at run time rather than interned. That is the suspected mechanism and the obvious place to look first; it is not something this harness has proven. Filed as z-engine#130.
• What is being lengthened here is the type argument's name - the name written into the property's type - not the specialization's own mangled name, which no property write ever reads. The two coincide for a nested generic, where the argument is a specialization and its angle-bracket name is long by construction: Box<Box<int>> pays this on every write to its inner slot.