Observability

June 10, 2026 · View on GitHub

DataSurface integrates with standard .NET observability infrastructure: structured logging via ILogger, OpenTelemetry metrics, distributed tracing, health checks, and audit logging.

Structured Logging

Both EfDataSurfaceCrudService and DynamicDataSurfaceCrudService emit structured logs for every operation:

[DBG] List User page=1 pageSize=20
[DBG] List User completed in 45ms, returned 20/142 items
[INF] Created User in 12ms
[INF] Updated User id=5 in 8ms
[INF] Deleted User id=5 in 3ms

Log Levels

Level	Operations
`Debug`	Operation start, read completions (List, Get)
`Information`	Mutating operations (Create, Update, Delete)

Structured Properties

All log entries include structured properties for filtering and querying:

Property	Description
`{Resource}`	Resource key (e.g., `"User"`)
`{Id}`	Entity ID (when applicable)
`{ElapsedMs}`	Operation duration in milliseconds
`{Count}` / `{Total}`	List result counts

These properties work with any structured logging sink (Serilog, Seq, Application Insights, etc.).

Metrics

OpenTelemetry-compatible metrics via DataSurfaceMetrics:

Setup

builder.Services.AddSingleton<DataSurfaceMetrics>();

builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics => metrics.AddMeter("DataSurface"));

Available Metrics

Metric	Type	Description
`datasurface.operations`	Counter	Total CRUD operations by resource and operation
`datasurface.errors`	Counter	Failed operations by resource, operation, and error type
`datasurface.operation.duration`	Histogram	Operation duration in milliseconds
`datasurface.rows_affected`	Counter	Rows affected by operations

Tags/Labels

Each metric includes tags for filtering:

resource — Resource key
operation — CRUD operation name
error_type — Error classification (for error counter)

Feature Flag

opt.Features.EnableMetrics = true;  // default: true in Standard/Full

When EnableMetrics is false, the CRUD and streaming services do not record metrics even if DataSurfaceMetrics is registered.

Distributed Tracing

Activity/span integration via DataSurfaceTracing:

Setup

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing.AddSource("DataSurface"));

Trace Attributes

Each CRUD operation creates a span with these attributes:

Attribute	Description
`datasurface.resource`	Resource key
`datasurface.operation`	CRUD operation
`datasurface.entity_id`	Entity ID (when applicable)
`datasurface.rows_affected`	Rows returned or affected
`datasurface.query.page`	Page number (List)
`datasurface.query.page_size`	Page size (List)
`datasurface.query.filter_count`	Number of active filters (List)
`datasurface.query.sort_count`	Number of sort fields (List)

Traces integrate with any OpenTelemetry-compatible backend (Jaeger, Zipkin, Azure Monitor, etc.).

Feature Flag

opt.Features.EnableTracing = true;  // default: true in Standard/Full

When EnableTracing is false, the CRUD and streaming services do not start spans even if a listener is subscribed to the DataSurface activity source.

Health Checks

Built-in IHealthCheck implementations for monitoring:

Setup

builder.Services.AddHealthChecks()
    .AddCheck<DataSurfaceDbHealthCheck>("datasurface-db")
    .AddCheck<DataSurfaceContractsHealthCheck>("datasurface-contracts")
    .AddCheck<DynamicMetadataHealthCheck>("datasurface-dynamic-metadata")
    .AddCheck<DynamicContractsHealthCheck>("datasurface-dynamic-contracts");

Available Health Checks

Check	Description
`DataSurfaceDbHealthCheck`	Database connectivity — verifies the DbContext can connect
`DataSurfaceContractsHealthCheck`	Static contracts loaded — verifies at least one contract is registered
`DynamicMetadataHealthCheck`	Dynamic entity definitions table accessible
`DynamicContractsHealthCheck`	Dynamic contracts loaded from metadata

Use the standard ASP.NET Core health check endpoint:

app.MapHealthChecks("/health");

Audit Logging

Track all CRUD operations with full before/after state using IAuditLogger:

Setup

using DataSurface.EFCore.Interfaces;

public class DatabaseAuditLogger : IAuditLogger
{
    private readonly AppDbContext _db;
    private readonly IHttpContextAccessor _http;

    public DatabaseAuditLogger(AppDbContext db, IHttpContextAccessor http)
    {
        _db = db;
        _http = http;
    }

    public async Task LogAsync(AuditLogEntry entry, CancellationToken ct)
    {
        _db.AuditLogs.Add(new AuditLog
        {
            UserId = _http.HttpContext?.User.FindFirst("sub")?.Value,
            Operation = entry.Operation.ToString(),
            ResourceKey = entry.ResourceKey,
            EntityId = entry.EntityId,
            Timestamp = entry.Timestamp,
            Success = entry.Success,
            Changes = entry.Changes?.ToJsonString(),
            PreviousValues = entry.PreviousValues?.ToJsonString()
        });
        await _db.SaveChangesAsync(ct);
    }
}

builder.Services.AddScoped<IAuditLogger, DatabaseAuditLogger>();

AuditLogEntry Properties

Property	Type	Description
`Operation`	`CrudOperation`	The operation performed
`ResourceKey`	`string`	Resource that was accessed
`EntityId`	`string?`	Entity ID (if applicable)
`Timestamp`	`DateTime`	UTC timestamp
`UserId`	`string?`	User who performed the operation
`IpAddress`	`string?`	Client IP address
`Success`	`bool`	Whether the operation succeeded
`ErrorMessage`	`string?`	Error message when the operation failed
`Changes`	`JsonObject?`	Fields written (create/update)
`PreviousValues`	`JsonObject?`	Previous field values (update)

Audit entries are written for cache hits too — a read served from the query cache is still audited. Inside bulk transactions, audit entries (and webhooks) are buffered and emitted only after a successful commit; a rollback discards them.

Feature Flag

opt.Features.EnableAuditLogging = true;  // default: true in Standard/Full

Observability Stack Example

A complete observability setup combining all features:

// Metrics + Tracing
builder.Services.AddSingleton<DataSurfaceMetrics>();
builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics => metrics.AddMeter("DataSurface"))
    .WithTracing(tracing => tracing.AddSource("DataSurface"));

// Audit logging
builder.Services.AddScoped<IAuditLogger, DatabaseAuditLogger>();

// Health checks
builder.Services.AddHealthChecks()
    .AddCheck<DataSurfaceDbHealthCheck>("datasurface-db")
    .AddCheck<DataSurfaceContractsHealthCheck>("datasurface-contracts");

// Feature flags — all observability enabled
opt.Features = new DataSurfaceFeatures
{
    EnableAuditLogging = true,
    EnableMetrics = true,
    EnableTracing = true
};

Feature Flags — Enable/disable individual observability features
Request Lifecycle — Where observability hooks into the pipeline