๐Ÿ”€ crdt-merge

March 27, 2026 ยท View on GitHub

๐Ÿ”€ crdt-merge

Conflict-free merge, dedup & diff for any dataset โ€” powered by CRDTs

npm version TypeScript License Tests: 97/97

Merge any two datasets in one function call. No conflicts. No coordination. No data loss.

Quick Start โ€ข Why CRDTs โ€ข API Reference โ€ข All Languages


๐ŸŒ Available in Every Language

LanguagePackageInstallRepo
Python ๐Ÿcrdt-mergepip install crdt-mergecrdt-merge
TypeScriptcrdt-mergenpm install crdt-mergeYou are here
Rust ๐Ÿฆ€crdt-mergecargo add crdt-mergecrdt-merge-rs
Java โ˜•io.optitransfer:crdt-mergeMaven / Gradlecrdt-merge-java
CLI ๐Ÿ–ฅ๏ธincluded in Rustcargo install crdt-mergecrdt-merge-rs

๐Ÿค— Try it in the browser โ†’


๐ŸŽฏ The Problem

You have two versions of a dataset. Maybe two services updated the same records. Maybe two contributors edited the same file. Maybe you're merging data from multiple sources.

Today: Write custom merge scripts, lose data, or block on a coordinator.

With crdt-merge: One function call. Zero conflicts. Mathematically guaranteed.

import { merge } from 'crdt-merge';

const merged = merge(dataA, dataB, { key: 'id' }); // done.

โšก Quick Start

npm install crdt-merge

Merge Two Datasets

import { merge } from 'crdt-merge';

const teamA = [
  { id: 1, name: 'Alice', role: 'engineer' },
  { id: 2, name: 'Bob', role: 'designer' },
];

const teamB = [
  { id: 2, name: 'Robert', role: 'designer' },  // Updated name
  { id: 3, name: 'Charlie', role: 'pm' },        // New member
];

const merged = merge(teamA, teamB, { key: 'id' });
// id=1: Alice (only in A โ€” preserved)
// id=2: Robert (B wins โ€” latest)
// id=3: Charlie (only in B โ€” preserved)

Deduplicate Anything

import { dedup } from 'crdt-merge';

const items = ['Hello World', 'hello  world', 'HELLO WORLD', 'Something else'];
const { unique, duplicates } = dedup(items);
// unique = ['Hello World', 'Something else']

// Fuzzy dedup with custom threshold
const { unique: fuzzyUnique } = dedup(items, { threshold: 0.7 });

See What Changed

import { diff } from 'crdt-merge';

const changes = diff(oldData, newData, { key: 'id' });
console.log(changes.summary);
// "+5 added, -2 removed, ~3 modified, =990 unchanged"

changes.modified.forEach(m => {
  console.log(`Row ${m.key}:`, m.changes);
});

Deep-Merge JSON/Objects

import { mergeDicts } from 'crdt-merge';

const configA = { model: { name: 'bert', layers: 12 }, tags: ['nlp'] };
const configB = { model: { name: 'bert-large', dropout: 0.1 }, tags: ['qa'] };

const merged = mergeDicts(configA, configB);
// { model: { name: 'bert-large', layers: 12, dropout: 0.1 }, tags: ['nlp', 'qa'] }

Use CRDT Types Directly

import { GCounter, PNCounter, LWWRegister, ORSet } from 'crdt-merge';

// Distributed counter
const counterA = new GCounter();
counterA.increment('server-1', 100);

const counterB = new GCounter();
counterB.increment('server-2', 200);

const merged = counterA.merge(counterB);
console.log(merged.value); // 300

// Last-writer-wins register
const regA = new LWWRegister('Alice', 1000);
const regB = new LWWRegister('Alicia', 2000);
console.log(regA.merge(regB).value); // 'Alicia' (later timestamp wins)

// Add-wins set
const setA = new ORSet<string>();
setA.add('item1');
const setB = new ORSet<string>();
setB.add('item2');
console.log(setA.merge(setB).value); // Set { 'item1', 'item2' }

๐Ÿง  Why CRDTs

CRDT = Conflict-free Replicated Data Type. A data structure with one mathematical superpower:

Any two copies can merge โ€” in any order, at any time โ€” and the result is always identical and always correct.

Three mathematical guarantees (proven, not hoped):

PropertyWhat it means
Commutativemerge(A, B) == merge(B, A) โ€” order doesn't matter
Associativemerge(merge(A, B), C) == merge(A, merge(B, C)) โ€” grouping doesn't matter
Idempotentmerge(A, A) == A โ€” re-merging is safe

This means: zero coordination, zero locks, zero conflicts.

Built-in CRDT Types

TypeUse CaseExample
GCounterGrow-only countersDownload counts, page views
PNCounterIncrement + decrementStock levels, balances
LWWRegisterSingle value (latest wins)Name, email, status fields
ORSetAdd/remove setTags, memberships, dedup sets
LWWMapKey-value mapRow merges, config objects

๐Ÿ“– API Reference

merge<T>(a: T[], b: T[], options?): T[]

Merge two arrays of objects using CRDT semantics.

  • key (string, default: "id"): Primary key field for matching rows.
  • strategy ('lww' | 'keep_a' | 'keep_b', default: 'lww'): Conflict resolution strategy.
  • timestamps ({ a?: number; b?: number }): Timestamps for LWW resolution.

dedup<T>(items: T[], options?): DedupResult<T>

Deduplicate an array using exact or fuzzy matching.

  • key (string | (item: T) => string): Field name or custom key extractor.
  • threshold (number, default: 0.85): Similarity threshold (0โ€“1). Use 1.0 for exact only.
  • caseSensitive (boolean, default: false): Whether matching is case-sensitive.

Returns { unique: T[], duplicates: Array<{ item, matchedWith, similarity }> }.

diff<T>(a: T[], b: T[], options?): DiffResult<T>

Compute structural diff between two datasets.

  • key (string, default: "id"): Primary key field.

Returns { added, removed, modified, unchanged, summary }.

mergeDicts<T>(a: T, b: T, options?): T

Deep-merge two objects with CRDT semantics.

  • strategy ('lww' | 'deep' | 'keep_a' | 'keep_b', default: 'deep'): Merge strategy.
  • timestamps ({ a?: number; b?: number }): Timestamps for LWW resolution.

CRDT Types

All CRDT types support merge(), value getter, toJSON(), and static fromJSON().

  • GCounter: increment(nodeId, amount?), value (sum)
  • PNCounter: increment(nodeId, amount?), decrement(nodeId, amount?), value (pos - neg)
  • LWWRegister<T>: set(value, timestamp?, nodeId?), value, timestamp
  • ORSet<T>: add(element), remove(element), contains(element), value (Set)
  • LWWMap: set(key, value, timestamp?), get(key), delete(key), value (Record)

๐Ÿ“Š Benchmarks

See the Python version benchmarks for reference numbers (320K+ rows/sec, 8.6M CRDT ops/sec).

๐Ÿ“„ License

Licensed under the Apache License, Version 2.0.

Contributing? By opening a pull request, you agree to our Contributor License Agreement.

Copyright 2026 Ryan Gillespie / Optitransfer. See NOTICE for attribution requirements.

For commercial licensing inquiries: rgillespie83@icloud.com, data@optitransfer.ch


Built with math, not hope. ๐Ÿงฌ

โญ Star on GitHub โ€ข ๐Ÿค— Try on HuggingFace โ€ข ๐Ÿ“ฆ npm