4.5 KiB
Sync algorithm
The current API v5 server stores immutable content versions and a separately versioned file manifest. Clients reconcile snapshots and apply them through a durable filesystem journal. The server does not merge competing edits.
Identity and ordering
Each document has a UUID independent of its path. A file manifest maps live UUIDs to paths; removing a mapping deletes membership. Updating content never restores membership. Directories are implicit.
Every accepted content or manifest update appends an event in the same SQLite
transaction. Event IDs order changes and serve as version IDs. A push names its
parent version and a unique request UUID. The server either accepts that exact
snapshot or returns current metadata as StaleBase. Retrying an accepted request
returns its original acknowledgement; changing its payload under the same request
UUID is rejected.
Reconciliation
A new client reads a coherent vault snapshot, applies it durably, then replays subsequent events. Replayed pages are folded to their final document heads and manifest before changing local files. This avoids applying intermediate remote edits that were already reverted.
For text, the client compares the common base, current local bytes and current
remote bytes using reconcile-text. Configured extensions and valid UTF-8 without
NUL bytes determine whether text merging is allowed. Diffs optimize transport;
they do not authorize the server to merge a stale base. Large diff operation sets
are sent as snapshots.
If only one side changed, that side is retained. Concurrent unmergeable changes keep the server version while the journal archives displaced local bytes for recovery. First contact with existing local text has no shared ancestor and uses an empty parent. Concurrent edits may still require human review.
Paths use a separate three-way decision: keep a local change when the server path is unchanged, otherwise prefer the server path if both changed. Portable-name collisions receive deterministic conflict names. A manifest deletion is not reversed merely because another client edited the document; displaced bytes are retained locally.
Filesystem application
Before moving files, the client persists a journal describing sources, destinations and replacement snapshots. It stages sources before installing any destination, so swaps and rename cycles do not overwrite each other. Prepared content is persisted before installation so a crash does not repeat a merge.
External notifications retain logical document identities. Notifications received during asynchronous filesystem reads or staging are incorporated before accepting a source. An editor save to a staged document's old path remains part of that document. Ignored and oversized physical occupants are protected during conflict allocation. Adapters report atomic editor saves as content updates and suppress notifications generated by the engine itself.
The filesystem and metadata store have separate durability boundaries. Recovery replays the journal before scanning, including when network sync is disabled. Adapters must provide complete scans, coherent snapshots, exclusive vault access, no-replace renames and durable writes. Recovery artifacts are excluded from sync.
Server restores and failures
Numeric version equality cannot establish identity after restoring a backup. Successful HTTP responses include a checkpoint with an event-incarnation token. The client persists and echoes it; the server rejects a checkpoint from a discarded history before executing its request. Each new event gets a fresh token even if its numeric ID or request UUID was used on the discarded branch.
After a mismatch, the client durably records recovery intent, retains pending content, clears old history-dependent state and bootstraps against the restored server. Clean files reconcile against the restored snapshot. Unsent edits with an old base are preserved as separate local documents, with conflict names if needed. Interrupted recovery saves are replayable. Backups must be restored with the server stopped.
WebSocket messages wake HTTP replay; missed notifications are recovered by replay and transport liveness checks. Authentication and permanent errors remain visible; transient errors retry according to client settings. Observer callback failures are isolated from the background sync loop.
API v5 clients and servers must be deployed together. V4 databases migrate without removing history. The old Obsidian and CLI adapters still require the filesystem and persistence contract required by the core engine.