Changelog

May 8, 2026 · View on GitHub

All notable changes to Morphium will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[6.2.4] - 2026-05-08

Added

MorphiumDocumentTooLargeException for BSON size limits (#199)

Introduced a dedicated MorphiumDocumentTooLargeException that is thrown when a document exceeds the 16MB BSON limit. This replaces generic MorphiumDriverException for these cases, allowing callers to programmatically handle oversized documents.

Messaging: Server-side recipient/sender filtering (#200)

SingleCollectionMessaging now uses a server-side $match stage in its change stream pipeline. This significantly reduces wire traffic and client-side decoding overhead by filtering out messages not intended for the current node directly on the MongoDB server.

Messaging: Passive liveness watchdog and cursor recovery (#201)

Added a watchdog that monitors the health of the messaging change stream. It can detect when a cursor has fallen behind or stalled and automatically restarts it to ensure timely message delivery.

Aggregator: Field name translation support (#198)

The Aggregator pipeline now supports field name translation, ensuring that Java camelCase field names are correctly mapped to their MongoDB snake_case counterparts during aggregation.

Fixed

Messaging: Robustness against Errors in main loop (#202)

The messaging main loop now catches Throwable instead of just Exception. This prevents the messaging thread from dying silently due to Errors (like OutOfMemoryError), keeping the system more resilient.

Field translation in Query.distinct() (#197)

Fixed a bug where Query.distinct() and explainDistinct() did not translate Java field names, leading to incorrect results when using camelCase names.

Messaging: Thread liveness check

Added a FATAL log message when the messaging main thread terminates unexpectedly, improving visibility into component failures.

[6.2.3] - 2026-04-20

Added

defaultQueryTimeoutMS configuration (#182)

A new defaultQueryTimeoutMS setting decouples the query/operation timeout from the connection pool wait time. Previously both reused maxWaitTime, making it impossible to wait long for a connection while still timing out individual queries quickly. Applied as fallback to both Query execution and aggregation commands.

storeList(..., continueOnError) for partial-failure batch stores (#190)

New overload storeList(List<T>, String collection, boolean continueOnError) continues processing remaining entities when individual stores fail, mirroring MongoDB's ordered: false insert semantics. Successful entities are persisted; failures are reported via the returned result. As part of this work, entity classification logic was refactored into a shared helper using Java records instead of Object[].

Batched versioned-entity updates in store(List) (#185)

Versioned-entity updates within a store(List) are now batched per connection instead of executing one round-trip per entity, reducing pool overhead noticeably for large lists.

Fixed

Connection swap in StoreMongoCommand not propagated to caller (#191)

When StoreMongoCommand swapped to a fresh connection (e.g. after a network error), the new connection reference was not returned to the caller. The caller continued using the stale reference, leading to inconsistent connection state. The swap is now propagated back correctly.

Transient WriteConflict (error 112) not retried (#184)

Single-document writes hitting a transient WriteConflict outside a transaction were surfaced to the caller instead of being retried. WriteMongoCommand now retries on error 112 — except inside an explicit transaction, where the caller must own the retry decision.

null collation sent in write commands (CosmosDB compatibility) (#186)

Write commands serialized an explicit collation: null field when no collation was set. CosmosDB rejects this with a parse error. Null collations are now omitted from the command document.

Insert/upsert writeErrors not surfaced as structured errors (#187, #188)

  • InsertMongoCommand and WriteMongoCommand failures now attach a structured writeErrors list to the thrown MorphiumDriverException, matching MongoDB's response format.
  • InMemoryDriver.insert() now produces proper writeError documents (with index, code, errmsg) for duplicate-key failures.
  • FindAndModifyMongoCommand now throws MorphiumDriverException with structured writeErrors on failure instead of returning a partial result.
  • Dead writeErrors checks following InsertMongoCommand.execute() were removed (the command now throws instead of returning errors).

InMemoryDriver insert did not honor ordered: false (#189)

When ordered=false was requested, InMemoryDriver.insert() still aborted at the first failure like the ordered case. It now continues inserting remaining documents and returns all writeErrors together, matching MongoDB semantics.

Missing return in save(T, String, AsyncOperationCallback) (#183)

A missing return after the saveList() call caused execution to fall through and double-process the entity.

PoppyDB startup checks and status command

Stabilized PoppyDB startup checks and added the missing status command implementation.

[6.2.2] - 2026-03-31

Fixed

PoppyDB: Update operations now return correct matched/modified counts

The InMemoryDriver returned "matched" instead of the MongoDB-standard "n" key in update results. This caused all update-based operations (inc, set, sequence, bulk updates) to fail with "Update failed" or "Error - not updated" when running against PoppyDB over the wire protocol.

PoppyDB: Find queries now respect batchSize (server-side cursor support)

processFindDirect previously returned all matching documents in a single firstBatch regardless of the requested batchSize, with cursor ID always 0. This broke iterators and cursors that rely on batched fetching. PoppyDB now returns only the requested batch and registers a server-side cursor for getMore requests.

PoppyDB: Insert error response includes nModified field

Duplicate-key error responses from insert operations were missing the nModified field, causing a NullPointerException in ThrowOnError predicates that call Number.intValue() on the missing map entry.

Expr.arrayExpr() parse roundtrip

ArrayExpr.toQueryObject() used Arrays.asList(stream.toArray()) which wrapped the result array as a single element instead of unpacking it. Also fixed Expr.parse(List) which returned List<Expr> objects instead of mapped query objects, and added proper evaluate() overrides for both ArrayExpr and parsed list expressions.

IndexDescription.equals() false mismatches

The comparison treated null and false/0 as different values for boolean and integer fields (e.g., background, sparse, unique). Since MongoDB may return explicit false for fields that Java leaves null, this caused indices to appear "missing" on every startup, triggering repeated create-index attempts that fail with "Index already exists". Also removed a stale log.info() call inside equals() that logged every single index comparison at INFO level.

PoppyDB: Upsert operations now correctly report document count

Upserted documents were not included in the "n" count of update responses. MongoDB returns n: 1 for a successful upsert (even though matchedCount is 0), but PoppyDB returned n: 0. This broke storeMap() assertions and any code that checks the update result count after an upsert.

PoppyDB: Wire protocol corruption on concurrent change stream responses

The CompletableFuture.whenComplete() callback for watch/tailable cursor getMore responses wrote directly to the Netty channel from a background thread. When a change stream event arrived while the I/O thread was writing another response on the same connection, the bytes were interleaved, producing corrupted wire protocol messages (Illegal opcode 0, wrong section ID). Responses are now dispatched back to the Netty event loop thread, serializing all writes per connection.

PoppyDB: writeErrors from InMemoryDriver not forwarded

processUpdateDirect in the Netty command handler silently dropped writeErrors returned by the InMemoryDriver (e.g., duplicate key errors on upsert). These errors are now included in the wire protocol response, matching MongoDB behavior.

Thread leak in PooledDriver.close() and ReplicationManager reconnect

PooledDriver.close() did not signal waitCounterCondition, leaving ConnectionWaiter threads blocked forever. Over time this accumulated thousands of leaked threads. Fixed by calling signalAll() before shutdown. Additionally, ReplicationManager.replicationLoop() now calls disconnectFromPrimary() before connectToPrimary() to prevent accumulating stale Morphium instances on repeated reconnects.

Change stream events lost after collection drop and resume

Several race conditions in the InMemoryDriver's change stream implementation could cause events to be lost or duplicated after a collection drop:

  • Stale async events: Events dispatched by virtual threads after a collection drop could sneak into the change stream history with tokens from the pre-drop era. Fixed by advancing the sequence counter by 100 on drop and filtering events whose tokens fall below the drop boundary.
  • Resume-after replay: replayHistory() now uses the maximum of the resume token and the drop boundary sequence, preventing stale events from being replayed.
  • History purge: drop() now purges the change stream history for the dropped collection both before and after the drop notification, ensuring no stale events survive.

ChangeStreamMonitor race condition on startup

running was set to true after Thread.start(), creating a window where the run() method could see running=false and exit immediately. Fixed by setting running=true before calling Thread.start().

PoppyDB: Tailable cursor events not delivered from direct insert path

The performance-optimized direct insert path (processInsertDirect) did not call notifyTailableCursorsOnInsert(). Only the generic command path had this notification. Tailable cursors on capped collections never received new documents, causing TailableQueryTests to fail on all PoppyDB phases.

PoppyDB: Hostname 0.0.0.0 in hello response breaks client connections

When PoppyDB binds to 0.0.0.0, the hello response reported hosts: ["0.0.0.0:17017"]. Clients tried connecting to 0.0.0.0 which is unreachable from remote hosts. PoppyDB now resolves 0.0.0.0 to the actual hostname via InetAddress.getLocalHost().

PoppyDB: Raft election flapping under load

Three nodes on the same host with equal priority (50) caused endless split-vote elections. Combined with onLeaderDiscovered firing on every heartbeat (not just on changes) and non-atomic isLeader()/getCurrentLeader() reads in getHelloResult(), the PooledDriver saw rapid primary flapping ("Primary failover?" multiple times per second). Fixed by:

  • Election timer generation guard prevents stale timer callbacks from triggering spurious elections
  • cancel(true) instead of cancel(false) for all timer tasks
  • getLeaderSnapshot() provides atomic leader state reads
  • onLeaderDiscovered only fires on actual leader changes
  • RS nodes should use different priorities (e.g. --rs-priorities 100,75,50)

Wire protocol corruption: concurrent writes on shared connection

SingleMongoConnection.sendQuery() was not synchronized. When the PooledDriver gave the same connection to multiple threads, their bytes interleaved on the wire, producing corrupted messages (Illegal opcode 0 with responseTo=0x6B6C0000 — bytes from $clusterTime mid-stream). Fixed by synchronizing sendQuery, sendCommand, and sendAndWaitForReply.

Network retry on closed connection reuses dead connection

When a MorphiumDriverNetworkException closed the connection (e.g. corrupt stream), the NetworkCallHelper retried on the same dead connection — guaranteed to fail again. MongoCommand.executeAsync() and WriteMongoCommand.execute() now check isConnected() before each retry and get a fresh connection from the pool if needed.

MongoCommand.getLog() StackOverflow

MongoCommand had a log field initialized via getLog() which recursively called itself. Fixed to use LoggerFactory.getLogger() directly.

Count command Long/Integer cast

processCountDirect in InMemoryDriver returned long but CountMongoCommand.getCount() cast to Integer, causing a ClassCastException. Now returns as int.

Changed

  • WriteSafety downgrade message (standalone MongoDB) reduced from WARN to DEBUG
  • Index creation message (CREATE_ON_STARTUP) reduced from WARN to INFO; WARN_ON_STARTUP remains WARN as intended
  • MultiCollectionMessaging fallback poll interval reduced from 5000ms to 1000ms for faster message delivery when change streams are unavailable
  • SingleMongoConnectDriver reconnect sleep reduced from 1000ms to 200ms for faster failover detection

Performance

ClassGraphCache: 4.7x faster Morphium startup

Introduced a JVM-wide singleton cache for ClassGraph classpath scan results. Previously, each new Morphium() triggered 2–4 full classpath scans (~100–500ms each), which dominated test setup time and slowed down applications that create multiple Morphium instances. The scan now happens once per JVM; all subsequent instances reuse cached results. In tests, BasicFunctionalityTest dropped from 67s to 14s.

  • Zero-copy BSON decoder, reduced BsonEncoder allocations per document
  • Shallow copy instead of deep copy for change stream events
  • Direct dispatch for hot-path commands (insert, update, delete, find, count, distinct)
  • PoppyDB: fixed thread pool instead of virtual threads (prevented OOM under load)
  • PoppyDB: orphaned cursor cleanup on client disconnect
  • PoppyDB: 3x faster than MongoDB for individual operations (insert 0.74ms vs 4.48ms, find 0.45ms vs 1.95ms, update 0.66ms vs 5.19ms in local benchmarks)

[6.2.0]

Breaking Changes

PoppyDB: Server extracted into separate module (renamed from MorphiumServer)

The server component has been extracted into its own Maven module and renamed to PoppyDB.

Why? The server pulled in dependencies (Netty, etc.) that 90% of Morphium users don't need — most projects just use the core library to talk to MongoDB. By extracting PoppyDB into a separate module, de.caluga:morphium stays lean. Beyond testing, PoppyDB is a fully functional MongoDB-compatible server — particularly useful as a messaging backend, providing a lightweight messaging solution without requiring a full MongoDB deployment. Add it as a test dependency or use it standalone:

<dependency>
    <groupId>de.caluga</groupId>
    <artifactId>poppydb</artifactId>
    <version>6.2.0</version>
    <scope>test</scope>
</dependency>

This also makes standalone deployment and testing of PoppyDB much simpler.

What changed:

  • Module: de.caluga:poppydb (was part of de.caluga:morphium)
  • Package: de.caluga.poppydb (was de.caluga.morphium.server)
  • CLI JAR: poppydb-<version>-cli.jar (was morphium-<version>-server-cli.jar)
  • Main classes: PoppyDB / PoppyDBCLI (were MorphiumServer / MorphiumServerCLI)
  • Netty handlers → de.caluga.poppydb.netty, election → de.caluga.poppydb.election
  • Morphium core library (de.caluga:morphium) is unaffected
  • Wire protocol backward compatible: server sends both poppyDB: true and morphiumServer: true in hello response

Multi-module Maven structure

The project is now a multi-module build:

  • morphium-parent — parent POM (de.caluga:morphium-parent)
  • morphium-core — the core library, artifactId stays de.caluga:morphium
  • poppydb — the server (de.caluga:poppydb)

Dependency coordinates for the core library are unchanged: de.caluga:morphium:6.2.0

MongoField.not() return type changed from Query<T> to MongoField<T>

The not() method now returns MongoField<T> instead of Query<T>, enabling fluent chaining:

// now compiles and works correctly
query.f("field").not().eq("val");

Migration: Any code that captured the return value of not() as a Query<T> must be updated to MongoField<T>. In practice not() was always intended to be chained with an operator (.eq(), .gt(), etc.), so no valid use of the old return type exists.

MorphiumDriverException is now unchecked (extends RuntimeException)

Aligns with MongoDB Java driver (MongoException), JPA, jOOQ, and Spring Data conventions.

Migration:

  • catch (MorphiumDriverException e) blocks continue to work — no changes needed
  • catch (RuntimeException | MorphiumDriverException e) must be simplified to catch (RuntimeException e)
  • Code inspecting getCause() for wrapped exceptions must catch MorphiumDriverException directly

Entity instantiation: ReflectionFactory → Unsafe.allocateInstance()

Replaced sun.reflect.ReflectionFactory (progressively hidden since JDK 17) with sun.misc.Unsafe.allocateInstance() for creating entity instances without no-arg constructors. This matches what Spring, Jackson, Gson, and Hibernate use. Best practice: add a no-arg constructor to @Entity classes.

Added

@Reference cascade features and cycle detection

  • cascadeDelete = true — Referenced entities are automatically deleted when the parent is deleted. Supports single references, collections, and maps.
  • orphanRemoval = true — References removed from a collection after update are automatically deleted.
  • Cycle detection — Circular @Reference chains (A→B→A) are detected during serialization and deserialization. Objects with IDs return {_id: ...}; objects without IDs throw IllegalStateException.
  • New CascadeHelper utility with ThreadLocal-based cycle detection.
  • Documentation: docs/howtos/references-and-relationships.md

@AutoSequence annotation — zero-boilerplate sequence assignment

@Entity
public class ImportRecord {
    @Id private MorphiumId id;
    @AutoSequence(name = "import_number", startValue = 1000, inc = 1)
    private Long importNumber;
}
  • Supported field types: long, Long, int, Integer, String
  • Explicit values are never overwritten — only null (or 0 for primitives) triggers assignment
  • Batch optimization: storeList() allocates all sequence numbers in a single round-trip via SequenceGenerator.getNextBatch()

Automatic CosmosDB backend detection

  • BackendType enum (MONGODB, COSMOSDB, POPPY_DB, UNKNOWN) in the driver layer
  • Auto-detected from hello handshake response (CosmosDB: msg field, PoppyDB: poppyDB field)
  • morphium.isCosmosDB() / driver.isPoppyDB() for application-level checks
  • Supports Azure sovereign cloud domains

@CreationTime / @LastChange enhancements

  • LocalDateTime support as a fourth field type (alongside long, Date, String)
  • Field-only usage — class-level @CreationTime annotation is no longer required; the field annotation alone is sufficient
  • Preset values preserved — explicitly set @CreationTime values are no longer overwritten on insert

resetThreadLocalOverrides()

New method to clean up all per-thread boolean overrides (disableAutoValuesForThread(), disableReadCacheForThread(), etc.) in a single call. Prevents state leaking between requests in thread-pool and virtual-thread environments.

@Version annotation — Optimistic Locking

Full optimistic locking via @Version on Long fields. On insert, version is initialized to 1; on update, a version-match filter is added and the version incremented atomically. VersionMismatchException on concurrent modification.

Other additions

  • MONGODB-X509 client-certificate authentication
  • mongodb+srv:// connection string support for MongoDB Atlas (pure-Java DNS, no JNDI)
  • Configurable LocalDateTimeMapper storage format (Date vs. ISO-8601 string)
  • SequenceGenerator.getNextBatch(int) for bulk sequence allocation in a single round-trip

Changed

Lazy-loading proxies: spring-cglib → ByteBuddy

Replaced org.springframework:spring-core (cglib) with net.bytebuddy:byte-buddy for lazy-loading proxy generation. ByteBuddy is actively maintained, has native Java 21 support, and requires no --add-opens JVM flags. Proxy classes are cached per entity type via ConcurrentHashMap to avoid Metaspace leaks. The new MorphiumProxyMarker interface replaces the fragile $$EnhancerByCGLIB$$ string check for proxy detection.

DNS SRV resolver logging

DnsSrvResolver now logs SRV resolution at INFO (start/result), DEBUG (per-server queries), WARN (failures), and TRACE (raw hex dump) for diagnosing Atlas connectivity in containers.

Fixed

PoppyDB: wrong BSON limits caused write failures

maxBsonObjectSize was reported as 10KB (should be 16MB) and maxMessageSizeBytes as 100KB (should be 48MB) in PoppyDB's hello response. The MongoDB driver uses these values to validate documents — the tiny limits caused BSON assertion errors and silent write failures under normal load.

PoppyDB: idle timeout killed change stream connections

Default idle connection timeout was 60 seconds. Change stream connections are idle by design between getMore polls — the short timeout killed them mid-wait, causing "Broken pipe" cascades. Increased to 300 seconds.

PoppyDB: stale primary status after elections

The primary boolean was a snapshot from connection init and became stale after replica set elections. Write-concern handling now uses the dynamic isCurrentPrimary() check via ElectionManager.

PoppyDB: aggressive connection close on parse errors

The wire protocol decoder closed the entire connection on unknown opcodes or payload parse errors. Now skips the malformed message (bytes are consumed so the stream stays in sync) and only closes on irrecoverable stream corruption or I/O errors. Prevents cascade failures where one bad message kills the connection.

Wire protocol: EOF handling and stream corruption

WireProtocolMessage.parseFromStream() could enter an infinite loop when InputStream.read() returned -1 (EOF) during header or body reads. Now returns null gracefully. Added message size validation and diagnostic logging (size, messageId, responseTo) on illegal opcodes.

Thread visibility: volatile running flags

SingleMongoConnection, SingleCollectionMessaging, BufferedMorphiumWriterImpl, WatchingCacheSynchronizer, and CacheHousekeeper used non-volatile running flags read by worker threads in while-loops. Without volatile, the JIT could cache the value and the worker thread would never see the stop signal. (MultiCollectionMessaging already used AtomicBoolean, ChangeStreamMonitor and PooledDriver already used volatile.)

@CreationTime not set on primitive long fields

f.get(o) on a primitive long field returns Long(0) (not null), so the "don't overwrite manually set" check always skipped setting the creation time. Now treats zero as "not set" for numeric types.

MongoField.not() produced wrong BSON structure

not() wrapped $not around the value instead of the operator, producing {$regex: {$not: val}} instead of the correct {$not: {$regex: val}}. Fixed operator grouping and addSimple() for not().eq().

QueryHelper.matchesQuery short-circuit on multi-field queries

The for-loop over query keys returned on the first field match without checking remaining fields, breaking AND semantics. Also fixed the same short-circuit in the Map/array-index pre-loop.

Auto-detect single-node replica sets

When no RS name is configured but the server's hello response contains a setName, the driver now auto-upgrades to RS mode. Covers Docker/Testcontainers setups where the server runs as a single-node replica set.

Index and capped collection checks

  • setAutoIndexAndCappedCreationOnWrite() now also sets CappedCheck (previously only IndexCheck)
  • Missing indices no longer reported for collections that don't exist yet
  • WriteConcern on standalone MongoDB: queries driver.isReplicaSet() instead of config flag; gracefully downgrades w>1 to w:1

Enum serialization/deserialization round-trip in untyped containers

Enums stored in untyped containers (Object, List<Object>, Map<String, Object>) were serialized as {class_name, name} maps, but the deserialization path never routed back through enum handling — causing ClassCastException on read. New deserializeEnumValue() method handles both String and Map formats (backwards-compatible with existing data). Also fixed: enums in typed Map<String, SomeEnum> or List<SomeEnum> were not converted back to their enum type.

Custom TypeMappers ignored in queries

Custom MorphiumTypeMapper implementations were not consulted when resolving field values in MongoFieldImpl. Queries now call ObjectMapperImpl.marshallIfNecessary() during value resolution.

WriteBuffer WAIT strategy lock starvation

The entire switch(strategy) block was wrapped in synchronized(opLog) — the WAIT strategy slept while holding the lock, preventing the flush thread from ever draining the buffer. Also fixed: missing break after WAIT (fall-through to JUST_WARN caused double-add), off-by-one in buffer limit check (> size vs >= size), TOCTOU race in WAIT branch, and WRITE_OLD/DEL_OLD creating plain ArrayList instead of Collections.synchronizedList().

Transaction isolation with write buffer

commitTransaction() called flush(), which drained the shared write buffer from ALL threads into the committing thread's transaction — breaking cross-thread isolation. Fix: startTransaction() now saves and disables the per-thread write buffer, commitTransaction()/abortTransaction() restore the previous state in finally. Also fixed: PooledDriver.markTransactionCommitted() was in the finally block, updating the read-routing timestamp even after a failed commit.

Transient transaction error 251 (NoSuchTransaction) handling

After MongoDB aborts a transaction, the TCP connection's server-side session retains the poisoned state. Subsequent operations on the same pooled connection receive error 251, which was thrown as non-retriable MorphiumDriverException. Fix: detect error 251, close the poisoned connection, throw MorphiumDriverNetworkException (retriable), and retry with a fresh connection. Also fixed: WireProtocolMessage.parseFromStream() and SingleMongoConnection.sendQuery()/readNextMessage() were wrapping MorphiumDriverNetworkException in RuntimeException/MorphiumDriverException, destroying the type information that NetworkCallHelper needs for retry decisions.

RS auto-detect race condition

Concurrent heartbeat threads could race on setReplicaSet()/setReplicaSetName() when auto-detecting a replica set from hello responses. Wrapped in double-checked locking with synchronized(primaryNodeLock).

Concurrent double-write in BufferedMorphiumWriterImpl.flush()

flush() used opLog.get() which returned a live reference. Concurrent calls would write the same entries, causing E11000 duplicate key errors. Fixed via opLog.remove() for atomic ownership transfer.

Quarkus / OSGi ClassLoader compatibility

All Class.forName() call sites now use a centralized helper preferring the thread context ClassLoader. Fixes ClassNotFoundException in Quarkus dev mode, OSGi, and JBoss.

Other fixes

  • @Version hardening: initialized to 1L on insert, $and filter in InMemoryDriver
  • BufferedWriter: immediate execution for non-buffered entities (buffer size = 0)
  • BufferedWriter: setIdIfNull support for UUID and ObjectId ID types
  • Sequence @WriteSafety: changed to BASIC for standalone MongoDB compatibility
  • BsonEncoder java.time type support
  • InMemoryDriver: no-op handler for X509 auth command
  • Multi-collection messaging bootstrapping speedup

Code Quality

  • Resolved all source and test compilation warnings
  • Replaced deprecated MorphiumConfig API calls with new sub-object API
  • CascadeHelper uses @CascadeAware marker annotation instead of ConcurrentHashMap caches

Tests

  • Increased timeouts for flaky messaging, changestream, and LastAccessTest tests
  • Comprehensive failover tests for PoppyDB replica sets
  • InMemory backend detection tests

Dependencies

DependencyPreviousUpdated
io.netty:netty-all4.1.100.Final4.2.9.Final
org.mongodb:bson4.7.14.11.5
org.slf4j:slf4j-api2.0.02.0.17
ch.qos.logback:logback-core1.5.241.5.25
org.assertj:assertj-core3.23.13.27.7
org.springframework:spring-core5.3.39removed
net.bytebuddy:byte-buddy—1.15.11

[6.1.8]

Tests

  • splitting long running tests for better maintainability
  • tuning some timeouts in tests in order to be more resiliant to load related slowdowns

Fixed

Connection Pool counter drift

• PooledDriver: fixes counter drift / incorrect borrowed counter decrement under topology changes (prevents apparent pool exhaustion). • ChangeStreamMonitor: fixes connection release fallback when watch exists but has no connection (prevents lingering borrowed counter of +1).

Heartbeat connection leak on error

• When getHelloResult() or connect() threw an exception during heartbeat, the connection container was polled from the pool but never returned or closed — invisible leak since it was not tracked in borrowedConnections either. Now properly closed in finally.

ReadPreference fall-through clarification

• Explicit fall-through comments for NEAREST → PRIMARY_PREFERRED → SECONDARY cascade in getReadConnection(). No behavioral change — documents the intentional degradation path.

Connection Pool Exhaustion due to Hostname Case Mismatch

  • Pool exhaustion when MongoDB reports hostnames with different casing: When MongoDB's hello response reported hostnames with different casing than the seed list (e.g., SERV-MSG1.example.com vs serv-msg1.example.com), connections were being closed instead of returned to the pool. The borrowed connections counter was not decremented, causing the pool to fill up to maxConnections with all connections appearing "borrowed" but none available.
  • Root cause: The hosts map was keyed by the hostname as reported by MongoDB, but releaseConnection() looked up by the hostname stored in the connection object (from the seed list). Case mismatch caused lookup failures.
  • Fix: All hostname operations now normalize to lowercase:
    • normalizeHostKey() converts to lowercase and ensures port suffix
    • SingleMongoConnection.getConnectedTo()/getConnectedToHost() return lowercase
    • addToHostSeed()/setHostSeed() normalize on write
    • getWaitCounterForHost(), getTotalConnectionsToHost(), onConnectionError() normalize inputs
    • ConnectionWaiter thread normalizes before all host lookups

ChangeStreamHistoryLost

  • forget resume token as it is invalid
  • restart changestream
  • might cause loss of a message or two, but is stable

Messaging Lock TTL Bug

  • Lock expires immediately when message has no timeout: When a message had timingOut=false, the TTL was 0, causing the lock to be created with deleteAt = now. MongoDB's TTL monitor would delete the lock almost immediately, allowing duplicate message processing. Now uses 7 days as fallback TTL for messages without timeout.

ChangeStreamMonitor Stability

  • ChangeStreamMonitor dies on "connection closed": Previously, a "connection closed" exception would cause the ChangeStreamMonitor to stop permanently with no auto-recovery. This is often a transient error (network issues, MongoDB failover). Now the monitor will retry the connection instead of giving up.
  • Improved exit logging: ChangeStreamMonitor now logs at WARN level when it stops, explaining the reason (config null, connection closed, no such host, etc.). Previously most exit conditions were logged at DEBUG level, making it hard to diagnose why messaging stopped working.
  • Resume token support for ChangeStreamMonitor: ChangeStreamMonitor now tracks the resume token from each event and uses it when restarting the watch after connection issues. This prevents duplicate events and ensures no events are missed during reconnection. Also handles ChangeStreamHistoryLost errors gracefully by discarding the stale token and starting fresh.

[6.1.0]

Added

PoppyDB Enhancements

  • Replica set support: PoppyDB now supports replica set configuration with automatic primary election and failover
  • Server CLI: New standalone poppydb-cli.jar for running PoppyDB from command line with --help option
  • Replication: Data replication between PoppyDB instances in a replica set
  • Custom election protocol: Implemented Raft-inspired election system for PoppyDB replica sets with:
    • Configurable election priorities per node
    • Heartbeat-based leader detection
    • Automatic leader election on primary failure
    • Vote request/response protocol for consensus
  • Netty-based wire protocol handler: New MongoCommandHandler using Netty for improved performance and connection handling
  • Messaging optimization: PoppyDB-specific optimizations for messaging workloads

Messaging

  • Topic Registry / Network Registry: New NetworkRegistry implementation for discovering messaging topics across the network
  • MessagingSettings: New configuration class for messaging-related settings

InMemoryDriver

  • Tailable cursor support: InMemoryDriver now supports tailable queries
  • Shared InMemory databases: Multiple Morphium instances can share the same InMemory database (configurable via DriverSettings.setShareInMemoryDatabase())
  • MongoDB-compatible $text query support: Full text search with MongoDB-standard query syntax
    • Root-level queries: { $text: { $search: "search terms" } }
    • Phrase search: { $text: { $search: "\"exact phrase\"" } }
    • Term negation: { $text: { $search: "include -exclude" } }
    • Case sensitivity: { $text: { $search: "...", $caseSensitive: true } }
    • Automatically searches fields defined in text indexes

Driver

  • Host class: New Host class for improved readability in connection pool management
  • Shared connection pools: Connection pool sharing between Morphium instances

PoppyDB

  • SSL/TLS support: PoppyDB can now accept SSL/TLS encrypted connections
    • server.setSslEnabled(true) to enable SSL
    • server.setSslContext(sslContext) for custom SSL configuration
    • Automatic TLS 1.2/1.3 protocol selection
  • Periodic snapshots/persistence: PoppyDB can now dump databases to disk and restore on startup
    • --dump-dir <path> CLI option to enable persistence
    • --dump-interval <seconds> for periodic dumps during runtime
    • Automatic restore from dump files on startup
    • Final dump on graceful shutdown
    • Programmatic API: setDumpDirectory(), setDumpIntervalMs(), dumpNow(), restoreFromDump()

Fixed

  • MultiCollectionMessaging DM polling when change streams disabled: When setUseChangeStream(false) is called on MultiCollectionMessaging, direct messages (DMs) are now also polled instead of using change streams. Previously, DMs were always using change streams regardless of the setting, causing inconsistent behavior. Added new pollAndProcessAllDms() method and updated the poll trigger handler to support "dm_all" triggers
  • Graceful thread pool shutdown in Morphium: Changed asyncOperationsThreadPool.shutdownNow() to graceful shutdown to prevent abrupt task termination
  • PooledDriver NPE and race conditions: Fixed null pointer exception for primaryNode, race condition with primaryNodeLock, and connection cleanup improvements
  • MorphiumWriterImpl graceful shutdown: Added graceful shutdown in close() and onShutdown() methods
  • InMemoryDriver change stream race condition: Fixed race condition in change stream handling (line 633-646)
  • Flaky IteratorTest.concurrentAccessTest: Fixed race condition where multiple threads sharing a single iterator would call hasNext() and next() non-atomically, causing incorrect element counts (e.g., 29130 instead of 25000). The test now properly synchronizes the hasNext+next critical section
  • Parallel test database isolation: Fixed race condition in MultiDriverTestBase where database cleanup would drop ALL databases matching the prefix pattern, including databases from other parallel tests that were still running. Now each test only drops its own database, preventing "expected X but was 0" failures in parallel execution
  • PoppyDB listDatabases: Added explicit handler for listDatabases command in PoppyDB. Previously this command returned null when forwarded through GenericCommand, causing NullPointerException in tests that call morphium.listDatabases()
  • PoppyDB stepDown for standalone servers: Standalone PoppyDB instances (no replica set configured) now immediately become primary again after receiving a replSetStepDown command. Previously, stepDown would leave the server in secondary state with no way to recover, causing "no primary" errors for subsequent operations
  • InMemoryDriver database-level change streams via PoppyDB: Fixed change stream event delivery for database-level watches registered through PoppyDB. When a client creates a database-level watch via the wire protocol, MongoDB convention sets collection to "1". The InMemoryDriver now correctly delivers events to subscribers registered under the db.1 namespace key
  • Message sending to self: Fixed broken message sending when sender equals recipient
  • Deadlocks: Fixed multiple deadlock scenarios in messaging and server components
  • Robust shutdown: Improved shutdown handling across components
  • NPE in QueryHelper.matchesQuery: Fixed null pointer exception when comparing MorphiumId/ObjectId fields against null query values
  • Flaky test fixes: Replaced timing-dependent Thread.sleep() + assertion patterns with TestUtils.waitForConditionToBecomeTrue() polling in messaging and changestream tests
  • Pooled driver updates: Updates now apply proper writeConcern consistently and single-document updates honor sort
  • Buffered writer bulk inserts: Fixed a race where mutating a list after storeList/insert(list) could flush as "0 operations" and/or cause duplicate inserts
  • Change stream lifecycle: ChangeStreamMonitor no longer misses early events as easily and terminates reliably (stops blocking watches on shutdown)
  • PoppyDB dropDatabase handling: Added "dropdatabase" to WRITE_COMMANDS set so database drops are properly forwarded to primary instead of being rejected by secondaries
  • Test database cleanup: Fixed MultiDriverTestBase to clean databases for ALL morphium instances (both PooledDriver and InMemoryDriver), not just the first one. Previously only one storage backend was cleaned, causing test isolation failures
  • GenericCommand key ordering: Changed cmdData from HashMap to LinkedHashMap in GenericCommand.fromMap() to preserve key ordering, which is critical for MongoDB wire protocol where the command name must be the first key
  • Test configuration default hosts: Changed TestConfig to default to single host (localhost:27017) instead of 3-host replica set for simpler test setup. Multi-node replica sets can still be configured via morphium.hostSeed property
  • PoppyDB getMore for regular query cursors: Fixed getMore command to forward regular query cursors to InMemoryDriver instead of only handling change stream cursors. Previously, iterators would hang infinitely when fetching additional batches because non-change-stream cursors were returning empty batches with non-zero cursor IDs
  • PoppyDB replica set replication: Extended change stream replication to handle drop, dropDatabase, replace, and rename operations. Previously only insert, update, and delete were replicated, causing collection drops and document replacements to not sync to secondaries
  • PoppyDB collection metadata forwarding: Added forwarding of listCollections command to primary when running as secondary. This ensures isCapped() checks return correct results for capped collections created on primary
  • InMemoryDriver listCollections capped status: Fixed listCollections response to include capped, size, and max options for capped collections. Previously the options field was always empty, causing isCapped() to return false even for capped collections
  • PoppyDB capped collection replication: Added initial and periodic sync of capped collection metadata from primary to secondaries. Capped collections created on primary are now properly registered on secondaries, ensuring capped behavior is enforced during replication
  • InMemory backend detection for tests: Added isInMemoryBackend() method to MorphiumDriver interface and inMemoryBackend field to hello response from PoppyDB. Tests that need to skip unsupported features (like Collation) can now correctly detect when connected to PoppyDB with InMemory backend, not just when using InMemoryDriver directly
  • PoppyDB changestream event delivery via wire protocol: Fixed changestream events not being delivered to clients connecting via the wire protocol. Watch cursors are now properly created with callbacks, events are queued via LinkedBlockingQueue, and getMore requests correctly return queued events to clients. This enables reliable messaging when using PoppyDB as a messaging hub
  • PoppyDB killCursors command handler: Added missing killCursors command handler to PoppyDB. Without this, watch cursors were never cleaned up when clients disconnected, causing virtual threads to accumulate and eventually block new watch thread creation. The fix properly removes cursors from watchCursors and tailableCursors maps
  • InMemoryDriver watch thread cleanup: Modified watchInternal() to periodically check callback.isContinued() after each wait timeout (max 5 seconds). This ensures watch threads properly terminate when cursors are killed, preventing resource exhaustion when many clients connect/disconnect
  • PooledDriver connection leak: Fixed connection leak in releaseConnection() where connections were removed from inUse set but not returned to the pool when the connection's host was no longer in the valid hosts set. Connections are now properly closed instead of being leaked
  • InMemoryDriver serverMode premature shutdown: Fixed InMemoryDriver to not clear data or shut down when serverMode=true and close() is called. PoppyDB instances now properly maintain their data when client Morphium instances disconnect
  • SingleMongoConnection watch loop termination: Fixed watch loop to check isContinued() after each individual event instead of only after processing the entire batch. This ensures watches terminate immediately when the callback returns false, matching InMemoryDriver behavior
  • ChangeStreamMonitor reconnection loop on shutdown: Fixed ChangeStreamMonitor to stop gracefully when receiving "No such host" errors instead of endlessly retrying. Also added driver connectivity check before attempting to get connections. This prevents resource exhaustion when PoppyDB instances are shut down
  • PooledDriver parallel connection creation: Changed connection creation from sequential to parallel (up to 10 virtual threads) to handle burst scenarios where many connections are needed simultaneously. This prevents connection timeouts when many async operations are queued at once
  • PoppyDB write concern handling with partial replica sets: Fixed write concern handling when configuring a replica set programmatically before all secondaries are started. Previously, writes with w > 1 would block for the full wtimeout (10 seconds) waiting for non-existent secondaries, causing client-side timeouts. The ReplicationCoordinator now fails fast (100ms grace period) when no secondaries have registered, returning a proper writeConcernError response instead of timing out. This enables tests to store documents on a primary before starting secondary nodes
  • Replication staleness detection: Added staleness detection mechanism to ReplicationManager that detects when a secondary's change stream watch connection has gone stale (no response for 30+ seconds). When detected, the connection is forcibly closed and a new one is established. This prevents secondaries from falling behind when connections silently break
  • SingleMongoConnection socket timeout limit: Modified readNextMessage() to limit consecutive socket timeout retries to 100 (approximately 10 seconds with 100ms timeout). After reaching this limit, it returns null to allow the calling code to check isContinued() and handle connection issues. Previously, the method would retry indefinitely, causing watch loops to never detect broken connections
  • Connection pool issues: Fixed multiple connection pool problems including proper connection release, leak prevention, and handling of stale connections
  • Messaging stability: Fixed various messaging issues including connection handling, message processing, and proper cleanup on shutdown
  • Server status on startup: Fixed PoppyDB status reporting during initial startup phase
  • NPE fixes: Fixed null pointer exceptions in various components during edge cases
  • Election priorities: Fixed election priority handling to ensure highest-priority node becomes primary
  • Read preference on secondary: Fixed read preference checks when operating on secondary nodes
  • Flaky CollationTest timing: Added wait conditions for collation queries to handle replica set replication delay. Previously, tests would fail intermittently because collation queries were executed before data was fully replicated
  • Flaky ExclusiveMessageBasicTests timing: Increased timing tolerance from 30s to 35s to account for timing variance in message processing
  • Flaky LastAccessTest assertions: Added better error messages for debugging timing-related assertion failures
  • CacheTests write buffer timeout: Increased write buffer flush timeout from 3s to 10s to handle PoppyDB latency

Added (Tests)

  • Failover tests for PoppyDB replica sets: Added comprehensive failover tests (FailoverTest.java) that verify:
    • Primary election based on configured priorities
    • Automatic failover when primary is terminated
    • Write operations succeed after failover
    • Rejoining nodes integrate correctly into the cluster
    • Tests cover both PooledDriver and SingleMongoConnectDriver

Changed (Test Infrastructure)

  • Unified multi-driver test base: Migrated 72 test classes from MorphiumTestBase to MultiDriverTestBase

    • Converted 356+ test methods from @Test to @ParameterizedTest with @MethodSource
    • Each test now declares driver compatibility via @MethodSource:
      • getMorphiumInstancesNoSingle() - pooled + inmem (default for most tests)
      • getMorphiumInstances() - all drivers including single connection
      • getMorphiumInstancesPooledOnly() - pooled driver only
      • getMorphiumInstancesInMemOnly() - inmem driver only
    • Tests receive Morphium morphium as parameter instead of using inherited field

  • Driver selection via runtests.sh: Tests can now run against different backends:

    # InMemory only (fast, default without --external)
./runtests.sh --driver inmem

# All drivers against external MongoDB
./runtests.sh --uri mongodb://host1,host2/db --driver all

# Against PoppyDB (run separately from MongoDB tests)
./runtests.sh --poppydb --driver pooled

  • Multi-backend testing workflow: To test against all backends:

    1. ./runtests.sh --driver inmem - InMemory driver (fast, no dependencies)
    2. ./runtests.sh --uri mongodb://... --driver all - Real MongoDB with all drivers
    3. ./runtests.sh --poppydb --driver pooled - PoppyDB

  • External test tagging: Added @Tag("external") to driver tests that require a real MongoDB connection (PooledDriverTest, PooledDriverConnectionsTests, SharedConnectionPoolTest). Fixed pom.xml to use correct <excludedGroups> parameter instead of invalid <excludeTags> for Maven Surefire plugin JUnit 5 tag filtering

  • Test script improvements: Major refactoring of runtests.sh for:

    • Modular script architecture with separate utility scripts in scripts/ directory
    • Better temporary file management and cleanup
    • Improved parallel test execution and slot management
    • Enhanced failure reporting and log management
    • Support for different test backends via --driver, --uri, and --poppydb options
    • Memory settings optimization for test execution

Changed

  • Modernized concurrent collections: Replaced legacy Vector with CopyOnWriteArrayList and Hashtable with ConcurrentHashMap for better performance
  • Optimized string operations: Consolidated multiple replaceAll() calls into single regex patterns, replaced replaceAll() with replace() for literal string replacements
  • ChangeStream implementation: Improved change stream handling and event delivery reliability

Dependencies

  • logback-core: Bumped from 1.5.13 to 1.5.19

Performance

InMemoryDriver Optimizations

  • Removed global synchronization on sendCommand(): Operations on different collections can now execute in parallel. Previously all commands were serialized through a single synchronized method, causing unnecessary contention.

  • Optimized find() deep copy behavior: Documents are now only copied after query matching succeeds, and projection-aware copying avoids redundant work:

    • Non-matching documents: No copy (previously copied before match check)
    • Include projections: Only projected fields are copied (previously full document copied twice)
    • Exclude projections: Single copy (previously double copy)

  • Improved index lookups for equality queries: Simple equality queries (e.g., {field: value}) now use fast Objects.equals() instead of full matchesQuery() evaluation. Operator queries ($gt, $lt, etc.) skip the index path entirely to avoid ineffective bucket scanning.

  • Rewrote TTL expiration checking:

    • Collections without TTL indexes have zero overhead (previously all collections scanned every 10 seconds)
    • TTL index info is cached when indexes are created
    • No snapshot copy during expiration check - iterates directly on CopyOnWriteArrayList
    • Auto-cleanup of tracking when collections are dropped

  • $in operator optimization: Changed from O(n*m) to O(n+m) using HashSet lookups

  • Aggregator reuse: Aggregators are now reused to reduce object allocation

  • Subdocument projection support: Improved projection handling for nested documents

  • Stats performance: Improved performance for driver statistics collection

PoppyDB Optimizations

  • Buffered I/O: Added 64KB buffered streams for socket read/write operations
  • ZLIB decompression buffer: Increased from 100 bytes to 8KB with pre-sized output buffer
  • Reduced redundant serialization: Avoid calling bytes() multiple times in logging paths

[6.0.3] - 2025-11-28

Fixed

  • NPE in MultiCollectionMessaging: Fixed null pointer exception in getLockCollectionName() when building lock collection names

[6.0.2] - 2025-10-16

Fixed

  • NPE in Query.set() methods: Changed from Map.of() to Doc.of() to allow null values in set operations
  • NPE in Msg.preStore(): Initialize processedBy list if null before validation

Changed

  • Default queue name handling: Setting queue name to "msg" now resets to default (null) for backward compatibility
  • Build configuration: Added runOrder=filesystem to surefire plugin for consistent test execution

[6.0.1] - TBD

📖 Detailed release notes: docs/releases/CHANGELOG-6.0.1.md 📝 Quick summary: docs/releases/RELEASE-NOTES-6.0.1.md

Breaking Changes

  • Null Handling Behavior Change: Default behavior now matches standard ORM conventions

    • Previous behavior: Null values were NOT stored in the database by default (fields omitted)
    • New behavior: Null values ARE stored as explicit nulls in the database by default
    • Fields WITHOUT annotation: Accept and store null values (standard ORM behavior)
    • Fields WITH @IgnoreNullFromDB: Reject nulls, field omitted when null
    • Migration impact: Existing code that relies on null values being omitted by default may need to add @IgnoreNullFromDB to those fields

  • @UseIfNull Deprecated: Replaced with @IgnoreNullFromDB for clearer semantics

    • Old annotation had inverted logic that was confusing
    • @UseIfNull is now deprecated but still functional
    • Migration: Replace @UseIfNull with @IgnoreNullFromDB and remove the annotation (behavior is inverted)

Added

  • New @IgnoreNullFromDB annotation: Protects fields from null contamination
    • Prevents null values from being stored during serialization (field omitted)
    • Rejects null values during deserialization (preserves default value)
    • Distinguishes between "field missing from DB" vs "field present with null value"
    • Special handling for @Id fields: NEVER stored when null (MongoDB auto-generates)
    • Comprehensive documentation with behavior matrix and use cases
  • Comprehensive test suites for null handling behavior
  • Enhanced documentation for null handling with detailed examples

Changed

  • Default null handling now matches standard ORMs:
    • Serialization: Null values stored as explicit null in database
    • Deserialization: Null values from database accepted and set to null
    • This aligns with Hibernate, JPA, and other standard ORMs
  • @Id field handling: Fields annotated with @Id are NEVER stored when null
    • Ensures MongoDB can auto-generate unique _id values
    • Prevents E11000 duplicate key errors from null _id values
  • runtests.sh: Added local PoppyDB cluster convenience mode (--poppydb-local) with optional auto-start (--start-poppydb-local)
    • Auto-start logs now go to .poppydb-local/logs/
    • Auto-start is idempotent and keeps a locally started cluster running by default

Fixed

  • Socket timeout handling in SingleMongoConnection - automatic retry on timeout exceptions
  • Better timeout detection in watch operations
  • Multi-collection messaging error handling and lock release
  • Connection management in message rejection handler
  • PoppyDB: fix replica set startup to avoid ending up with no primary
  • PoppyDB: support aggregate command over the wire (enables aggregation stage tests against PoppyDB)
  • Bulk operations now return proper operation counts: runBulk() now returns statistics including num_inserted, num_matched, num_modified, num_deleted, num_upserts, and upsertedIds

Performance

  • Added collection name caching to reduce reflection overhead

Known Issues

Messaging with PoppyDB Replicaset

  • ExclusiveMessageTests#exclusivityTest: This test is flaky when running with multiple Morphium instances connecting to a PoppyDB replicaset. The test sometimes passes and sometimes times out due to slower message processing compared to real MongoDB. Change stream events ARE being delivered correctly, but processing throughput with PoppyDB is lower than with real MongoDB, causing occasional timeouts with the default test timeout.
    • Workaround: Increase test timeout or use InMemoryDriver directly for messaging tests, or use a real MongoDB replicaset
    • Status: Performance issue, not a correctness issue

Test Suite Notes

  • ShardingTests: These tests require a sharded MongoDB cluster and will fail on standalone or replica set deployments
  • SharedConnectionPoolTest: Infrastructure test that requires specific connection pool setup
  • TopicRegistryTest: Network registry discovery tests may fail due to timing issues in some environments

Test Results Summary (v6.1.0)

BackendTests RunPassedErrorsSkipped
InMemory Driver10469290105
MongoDB (Replicaset)10469330105
PoppyDB (Replicaset)10241024092

[6.0.0] - 2024-XX-XX

Major Release

  • Java 21+ requirement
  • Significant architectural improvements
  • Enhanced driver support
  • SSL/TLS support: Added SSL/TLS support for secure connections to MongoDB
    • driver.setUseSSL(true) to enable SSL connections
    • driver.setSslContext(sslContext) for custom SSL configuration
    • driver.setSslInvalidHostNameAllowed(true) to disable hostname verification
    • New SslHelper utility class for creating SSLContext from keystores
  • Improved documentation

For detailed release notes, see individual release documentation in docs/releases/.