Horse Integrity and Resilience Testing

July 16, 2026 ยท View on GitHub

Horse includes an automated suite of integrity tests designed to validate the framework's behavior in real network scenarios, as well as analyze its resilience under fatal system failures and critical memory overflows.

These tests reside in the folder: samples/delphi/console_complete/

Why are Integrity Tests Useful?

Unlike traditional unit tests (which test individual classes and methods in isolation), integrity tests simulate a real HTTP client consuming an active Horse API over the network.

They are extremely useful for:

  • Preventing Regressions: Guaranteeing that changes in the HTTP parser, router, middleware, or memory buffers do not break the expected behavior of real network calls.
  • Validating Verbs and Headers: Verifying correct parsing of query strings, route parameters, case-insensitive headers, and the custom QUERY HTTP verb.
  • Testing real uploads: Validating actual file transmissions sent as multipart/form-data.
  • Resilience Analysis (Robustness): Assessing how the Horse server handles fatal errors inside the application logic (such as null pointers) without crashing the service.

What is Tested?

The integrity test script (test_integrity.ps1) runs two consecutive validation passes: one compiling the sample server (ConsoleComplete.dpr) with the Default Router (RouterTree), and another with the Radix Router (RadixRouter). Each pass performs the following real HTTP requests using Invoke-RestMethod and curl:

  1. GET /ping: Basic request flow and quick ping-pong verification.
  2. GET /resource/:id: Validates URL parameter extraction, query strings, and case-insensitive authorization headers.
  3. POST /resource: Transmission of raw payload inside the request body.
  4. PUT, PATCH, DELETE /resource/:id: RESTful verbs for resource modification and deletion.
  5. QUERY /search: Custom QUERY verb sending search payloads.
  6. POST /upload: Actual multipart file upload.
  7. GET /error-trigger: Clean, formatted exceptions raised using EHorseException.
  8. GET /clientes, GET /pessoas, GET /qualquercoisa: Route matching priority, checking that exact paths are resolved before fallback wildcard (*) matching.

Resilience and Limitations (Fatal System Failures)

The integrity suite includes specific endpoints to trigger catastrophic crashes and analyze the Horse server's resilience:

1. Intercepting Access Violations (AV)

Accessing /av-trigger intentionally performs a write operation on a null pointer reference:

var
  LPointer: PInteger;
begin
  LPointer := nil;
  LPointer^ := 42; // Access Violation!
end;
  • Horse Behavior: Horse successfully intercepts the violation on the request handler thread level. It blocks the main executable from crashing, returns an HTTP 500 Internal Server Error message, and keeps the server active to continue handling other requests.

2. Stack Overflow

Accessing /stack-trigger triggers an infinite recursion that consumes the thread's memory stack allocation limit:

var
  LRecurse: TProc;
begin
  LRecurse := procedure
    begin
      LRecurse();
    end;
  LRecurse(); // Stack Overflow!
end;
  • SO Limitations: A Stack Overflow represents a hardware/OS-level resource limit. When the thread memory stack size limit is exceeded, the OS intervenes and terminates the server process immediately for stability. The integrity script correctly logs and handles this crash scenario.

Running Locally

To run the integrity tests locally in a Windows environment using PowerShell:

cd samples/delphi/console_complete
.\test_integrity.ps1

The script manages the compilation of both routers using the Delphi compiler (dcc32), launches the background process for each scenario, executes the validation requests, terminates the servers, and prints a final combined success/failure summary.

Running via Docker

For development isolation, or to avoid installing FPC and PAServer on your local host, you can orchestrate your Linux testing environment using the provided docker-compose.yml inside the tests/ directory.

1. Running Lazarus / FPC Tests

To compile and run the full test suite under Lazarus/FPC on Linux (Ubuntu environment):

cd tests
docker compose run --build tests-lazarus

This command automatically:

  • Resolves unit test dependencies via Boss.
  • Clones DUnitX.
  • Compiles the console test runner.
  • Executes the tests and outputs the results to your terminal.

2. Running PAServer for Delphi (Cross-Compilation / Debugging)

If you are developing on Windows and want to target, deploy, and debug your Delphi application directly on a Linux container:

cd tests
docker compose up -d paserver

This will launch the Delphi Platform Assistant (PAServer) on port 64211 without a password. To connect your Delphi IDE (Windows) to this PAServer instance:

  1. Open Delphi and go to Tools > Options > Deployment > Connection Profile Manager.
  2. Add a new Connection Profile:
    • Platform: 64-bit Linux
    • Host name: localhost (or the IP of your Docker host)
    • Port: 64211
    • Password: Leave empty.
  3. In your Delphi Project Manager, select Linux 64-bit as the target platform, choose the newly created connection profile, and run/debug.

---

## Static Cross-Compilation Matrix

In addition to runtime network integration testing, Horse includes an automated static compilation verification suite. This is designed to ensure that all combinations of Providers and Routers compile correctly and without syntax regressions across multiple compilers and operating systems.

These tests reside in the folder: `tests/`

### Why is the Compilation Matrix Important?
In the Delphi and Lazarus/FPC ecosystems, compilers implement *smart linking* and demand-driven compilation. If a secondary provider (such as `HttpSys` or `IOCP`) or a specific router type is not explicitly referenced in the project's default unit test suite, severe syntax errors in those components can remain undetected during standard continuous integration (CI/CD) pipelines.

The matrix forces the full static compilation graph of all framework `.pas` source files in an isolated, parallel, and cross-platform manner.

### What is Tested?
The `run_compile_matrix.ps1` script orchestrates compiling the stub `tests/src/CompileCheck.dpr` across all combinations of:
*   **Compilers**: Delphi 10 Seattle, Delphi 11 Alexandria, Delphi 12 Athens, Delphi 13 Florence (local) and Lazarus/FPC (Linux).
*   **Providers**: Default (Indy), IOCP, HttpSys, Apache, CGI, ISAPI, Daemon, VCL, LCL, Epoll.
*   **Routers**: Default (`RouterTree`) and Radix Router (`RadixRouter`).

### How to Run the Matrix Locally

To run the full static compilation matrix check in Windows using PowerShell, execute the following command from the repository root:

```powershell
powershell -File .\tests\run_compile_matrix.ps1

The script will automatically auto-detect your local Delphi installations, check if Docker Desktop is available (to compile Lazarus/FPC targets in an isolated Ubuntu container), run all compilation variations, and print a consolidated success/failure summary table.