Latest run

updated with actions and can be found here and fancy histogram version are here

What is all about?

General idea is to hide implementation of each ECS under context abstraction and work with it from benchmark implementations.

Benchmarks design follow 2 rules which I try to balance with:

• Strict usage - to ensure all benchmarks are running with same flow to avoid cheating.
• Features utilization - to allow implementations to run in perfomant way.

General flow of any benchmark execution is divided into 3 steps:

• Preparation
  • Creating world
  • Creating initial entities if needed
  • Initialize filters and queries or other stuff which used to gain perfomance
• Benchmark call
  • Acquiring lock of world
  • Run main logic
  • Commiting changes
• Cleanup - mostly omitted

Implemented contexts

ECS	Version	Implemented	Verified	Notes
ArrayECS	1.0.0	✅	✅	dumb ecs on arrays
Arch	2.1.0-beta	✅	❌	Source Generator 2.1.0 / Systems 1.1.0
fennecs	0.6.9	✅	❌	N/A
Morpeh	stage-2024.1.1	✅	❌	N/A
DragonECS	1.0.1	✅	❌	N/A
LeoECS	2023.6.22	✅	❌	N/A
LeoECSLite	2024.5.22	✅	❌	N/A
DefaultECS	0.18.0-beta01	✅	❌	Analyzer 0.17.0
FlecsNET	4.0.4-build.548	✅	❌	N/A
TinyEcs	main	✅	❌	No releases in nuget since 2024 so simply use some commit from main
Xeno	0.1.9	✅	✅	N/A
FriFlo	3.6.0	✅	❌	N/A
StaticEcs	2.0.2	✅	✅	StaticPack 1.1.0
Massive ECS	v20.0.0	✅	✅	N/A

Implemented benchmarks

Benchmark	Description
Create Empty Entity	Creates [EntityCount] empty entities
Create Entity With N Components	Creates [EntityCount] entitites with N components
Add N Components	Adds N components to [EntityCount] entities
Remove N Components	Adds N components to [EntityCount] entities
Structural Changes	Adds \ Removes 1\2\3 components from 1-2-3-4 component entities (tests archetype migration
System with N Components	Performs simple operations on entities (numbers sum)
System with N Components Multiple Composition	Same as System with N Components but with mixture of other components
MultiSystem	C1 \ C2 \ C1C2 entities iteraing with 3 different systems
Filter Mismatch	100 archetypes and all not match 100 systems (empty iterations)

Running

Just call Benchmark.sh from terminal.

Command line args:

arg	description	sample
benchmark	allow to specify single benchmark to run	benchmarks=CreateEmptyEntities,Add1Component
benchmarks	allow to specify benchmarks to run	benchmark=CreateEmptyEntities
contexts	allow to specify contexts to run	contexts=Morpeh,Fennecs,...
--list	prints all benchmarks name	--list

Since all comparisons is made by string contains you can simply write something like contexts=Morpeh instead of context=MorpehContext and benchmarks=With1,With2 to launch subset of benchmarks. Selected benchmarks and contexts will be logged to console. BUT benchmark arg requires exact name match with those printed with --list

Contribution

• Fork
• Implement
• Create PR

Problems

1. Because of nature of BenchmarkDotNet there's sequential iteration of creating entities happening. This leads to case where, for example we're creating 100k entities in benchmark, it's properly cleared in Setup and Cleanup but benchmark itself will be called multiple times which will lead to creating 100k entities, then another 100k and in some cases lead to millions of entities in the world which can affect performance of creation and deletion on certain ECS implementations.
2. System benchmarks which uses Padding property produces up to 1.100.000 entities each because of logic of padding generation. It affects runs duration but for now i'm not sure about correct way do fix that (maybe keep entire entities count up to EntityCount so it'll not affect speed, but it'll reduce actual entity count to about 9.9k so archetype ecs implementation will gain significant boost).
3. Because some framework deleting entity on delete last components there are differences in behaviours in tests and benchmarks. For example RemoveComponent benchmark will work faster with Arch and Fennecs because they're not deleting entity. Because of that special property called DeletesEntityOnLastComponentDeletion is required to be implemented in each context.