Build Log

V3 is our biggest architectural shift yet. Each demo currently lives as a section on the landing page, but they deserve their own space. The plan: • /stress-test — Full-page stress testing dashboard with configurable concurrency, custom operation generators, and exportable benchmark reports • /whiteboard — Standalone collaborative canvas with shareable room URLs, persistent state, and unlimited canvas size • /split-screen — Multi-panel sync demo supporting 3+ clients with network condition simulation • /metrics — Production-grade observability dashboard with Grafana-style time series • /conflicts — Interactive conflict exploration with step-through replay Each page will maintain its own WebSocket connection and room, with URL-based room sharing so visitors can collaborate across browser tabs or devices.

This release was born from actually using the landing page and realizing... nothing worked. The WebSocket demos showed 'disconnected' or 'reconnecting' because the frontend sent commands before the server acknowledged the connection. The architecture diagram was an SVG/CSS coordinate nightmare where nodes and arrows existed in different coordinate spaces. The stress test sent operations typed 'create' but the backend expected 'add'. Key fixes: • Rebuilt the architecture diagram as pure HTML/CSS flexbox — no more SVG coordinate math • Fixed V2 protocol handshake: wait for 'connected' ack before sending room commands • Fixed operation type mapping: 'create'→'add', 'delete'→'remove' • Fixed backend handleCreateRoom: it created rooms but never added the client as a participant • Added dual-bot whiteboard system (BotAlice + BotBob) with start/stop toggles • Added batch card operations (add 1-100 at once) in split-screen • Replaced real metrics/conflict panels with simulated data for the landing page • Removed sustained mode from stress test (it was just a slower burst with no demo value) The lesson: integration testing between frontend protocol assumptions and backend responses is where bugs hide.

V1 was a backend library. V2 turned it into something you can show people. The headline number — 50,000 ops/sec — comes from the native Rust addon (napi-rs). The TypeScript merge path topped out around 5K ops/sec for complex operations. By moving the hot path to Rust, we kept Node.js for what it's good at (I/O, WebSocket routing) and let Rust handle the CPU-bound CRDT math. The V2 protocol consolidates everything into a single WebSocket per room with four logical channels: • Operations — reliable, persisted, ACK'd • Awareness — fire-and-forget at 60fps (cursor positions) • Metrics — server-push at 1s intervals • Control — room management, simulation commands The batch processor buffers operations for 1-5ms windows, then processes them as a single merge-persist-broadcast cycle. Above 50 ops in a batch, it offloads to worker_threads for parallel merge. The frontend showcases all of this through live demos: a stress tester that floods the engine, a whiteboard with simulated collaborators, split-screen bidirectional sync, and live metrics/conflict panels.

CRDTs have mathematical properties that must hold universally. Property-based testing generates thousands of random inputs and checks these invariants: • Commutativity: merge(state, A, B) = merge(state, B, A) • Associativity: merge(merge(state, A, B), C) = merge(state, A, merge(state, B, C)) • Idempotence: merge(merge(state, A), A) = merge(state, A) • Remove-wins: concurrent remove + update → item stays deleted • Queue ordering: persist → restore produces identical ordering • Serialization round-trip: serialize → deserialize = identity • Reconnection cap: only most recent 1000 ops transmitted We use fast-check (TypeScript) for the Node.js codebase. Each property generates arbitrary operations, timestamps, and state configurations. In the first week, PBT found 3 bugs that unit tests missed: 1. An off-by-one in HLC comparison when wallTime was equal 2. A queue capacity edge case at exactly 10,000 operations 3. A delta computation that included unchanged fields The bug-condition methodology in our bugfix spec also uses PBT: write a test that encodes expected behavior, run it against unfixed code (should fail), fix the code, re-run (should pass).

The V1 protocol was simple: open WebSocket, send JSON, get JSON back. V2 needed to support four distinct communication patterns over a single connection: 1. Operations need reliability (ACKs, ordering, persistence) 2. Awareness needs speed (cursor positions at 60fps, no persistence) 3. Metrics need server-push (1-second intervals, no client request) 4. Control needs request/response (room create/join, simulation commands) The protocol handshake: • Client connects with ?token=JWT • Server validates JWT, sends { type: 'control-response', payload: { type: 'connected', clientId } } • Client sends create-room or join-room • Server confirms with room state • Client can now send ops/awareness on that room The sequencing bug that bit us: demo components sent create-room immediately in ws.onopen, before the server's 'connected' acknowledgment arrived. The server rejected these because it hadn't associated the client yet. The fix was straightforward (wait for ack), but finding it required tracing the full message flow from frontend → backend → response. Another bug: the frontend checked for 'room-created' in responses but the backend sent 'create-room' with status 'created'. Type naming mismatches across the protocol boundary are invisible until you trace actual messages.

The goal was clear: build a distributed sync engine from scratch that guarantees convergence without coordination. Core components: • Sync Engine — LWW-Element-Set merge with field-level last-writer-wins. Remove-wins semantics for deletions. Delta computation for minimal broadcasting. • Hybrid Logical Clock — Combines wall-clock time with logical counters for causal ordering. Deterministic tiebreaker via lexicographic nodeId when timestamps are equal. • Connection Manager — WebSocket gateway with JWT auth, 30s heartbeat + 10s timeout, presence tracking, missed-update queuing (max 1000 ops / 24h). • Client SDK — Local replica, 10K-operation offline queue with file-based persistence, exponential backoff reconnection, ACK-based dequeue. • Persistence — PostgreSQL for durable operation log + snapshots (every 10min / 1000 ops), Redis for state cache (<50ms reads) + presence hashes. 21 properties tested: • Merge: commutativity, associativity, idempotence, remove-wins • Clock: total ordering, deterministic tiebreak • Queue: FIFO ordering, persistence round-trip, capacity limits, ACK removal, reconnection cap • SDK: API translation correctness, serialization round-trip, incoming operation application, item validation • Connection: presence data integrity, auth error classification, missed update queue integrity • Persistence: state recovery via replay • Engine: partial batch merge correctness All implemented with vitest + fast-check. The entire backend has zero runtime dependencies beyond ws, ioredis, pg, uuid, and jsonwebtoken.

The first decision: Operational Transformation (OT) or CRDTs? OT (used by Google Docs) transforms operations against concurrent edits. It requires a central server to establish operation order. CRDTs guarantee convergence mathematically — any replica can merge operations in any order and reach the same state. We chose CRDTs because: • No central coordination required (operations can merge in any order) • Natural fit for offline-first (queue locally, merge on reconnect) • Mathematical correctness guarantees (provable convergence) • Educational value (implementing LWW from scratch teaches distributed systems deeply) Within CRDTs, we chose LWW-Element-Set because: • Simple mental model (last edit wins, period) • Efficient (single timestamp comparison per field) • Deterministic (HLC + nodeId tiebreaker means no ambiguity ever) • Good enough for most collaborative apps (documents, inventory, dashboards) The tradeoff: LWW can 'lose' concurrent edits (the earlier write disappears). For text editing you'd want a sequence CRDT (Yjs, Automerge). For our inventory/state sync use case, LWW is perfect. Local-first architecture: • Client applies operations immediately (sub-100ms local feedback) • Operations queued and sent asynchronously • Server merges, persists, broadcasts deltas • On reconnect: replay queued ops, receive missed deltas, converge This gives users instant responsiveness while guaranteeing eventual consistency across all replicas.

Convergence started as a portfolio project with a specific thesis: demonstrate deep distributed systems knowledge through a working product, not just a README. Goals: • Sub-200ms operation broadcast under normal conditions • 5+ concurrent participants editing shared state • Seamless offline-to-online reconciliation without data loss • Deterministic conflict resolution requiring no user intervention • Full property-based test coverage of mathematical invariants Tech stack decisions: • Node.js + TypeScript — WebSocket concurrency via event loop, shared language with frontend SDK, strong typing • ws — Lightweight, spec-compliant WebSocket implementation • PostgreSQL — Durable append-only operation log, snapshot storage • Redis — Sub-50ms state cache, presence tracking, pub/sub • vitest + fast-check — Property-based testing for CRDT correctness • Next.js 14 — App Router for the frontend showcase Development approach: spec-driven with formal requirements → design → implementation plan → property-based tests → implementation. Every feature starts with correctness properties before writing code. The project would evolve through: V1 (backend engine) → V2 (platform with frontend + Rust performance) → V3 (individual app experiences).