85 lines
4.5 KiB
Markdown
85 lines
4.5 KiB
Markdown
# Sync algorithm
|
|
|
|
The current API v5 server stores immutable content versions and a separately
|
|
versioned file manifest. Clients reconcile snapshots and apply them through a
|
|
durable filesystem journal. The server does not merge competing edits.
|
|
|
|
## Identity and ordering
|
|
|
|
Each document has a UUID independent of its path. A file manifest maps live UUIDs
|
|
to paths; removing a mapping deletes membership. Updating content never restores
|
|
membership. Directories are implicit.
|
|
|
|
Every accepted content or manifest update appends an event in the same SQLite
|
|
transaction. Event IDs order changes and serve as version IDs. A push names its
|
|
parent version and a unique request UUID. The server either accepts that exact
|
|
snapshot or returns current metadata as `StaleBase`. Retrying an accepted request
|
|
returns its original acknowledgement; changing its payload under the same request
|
|
UUID is rejected.
|
|
|
|
## Reconciliation
|
|
|
|
A new client reads a coherent vault snapshot, applies it durably, then replays
|
|
subsequent events. Replayed pages are folded to their final document heads and
|
|
manifest before changing local files. This avoids applying intermediate remote
|
|
edits that were already reverted.
|
|
|
|
For text, the client compares the common base, current local bytes and current
|
|
remote bytes using `reconcile-text`. Configured extensions and valid UTF-8 without
|
|
NUL bytes determine whether text merging is allowed. Diffs optimize transport;
|
|
they do not authorize the server to merge a stale base. Large diff operation sets
|
|
are sent as snapshots.
|
|
|
|
If only one side changed, that side is retained. Concurrent unmergeable changes
|
|
keep the server version while the journal archives displaced local bytes for
|
|
recovery. First contact with existing local text has no shared ancestor and uses
|
|
an empty parent. Concurrent edits may still require human review.
|
|
|
|
Paths use a separate three-way decision: keep a local change when the server path
|
|
is unchanged, otherwise prefer the server path if both changed. Portable-name
|
|
collisions receive deterministic conflict names. A manifest deletion is not
|
|
reversed merely because another client edited the document; displaced bytes are
|
|
retained locally.
|
|
|
|
## Filesystem application
|
|
|
|
Before moving files, the client persists a journal describing sources,
|
|
destinations and replacement snapshots. It stages sources before installing any
|
|
destination, so swaps and rename cycles do not overwrite each other. Prepared
|
|
content is persisted before installation so a crash does not repeat a merge.
|
|
|
|
External notifications retain logical document identities. Notifications received
|
|
during asynchronous filesystem reads or staging are incorporated before accepting
|
|
a source. An editor save to a staged document's old path remains part of that
|
|
document. Ignored and oversized physical occupants are protected during conflict
|
|
allocation. Adapters report atomic editor saves as content updates and suppress
|
|
notifications generated by the engine itself.
|
|
|
|
The filesystem and metadata store have separate durability boundaries. Recovery
|
|
replays the journal before scanning, including when network sync is disabled.
|
|
Adapters must provide complete scans, coherent snapshots, exclusive vault access,
|
|
no-replace renames and durable writes. Recovery artifacts are excluded from sync.
|
|
|
|
## Server restores and failures
|
|
|
|
Numeric version equality cannot establish identity after restoring a backup.
|
|
Successful HTTP responses include a checkpoint with an event-incarnation token.
|
|
The client persists and echoes it; the server rejects a checkpoint from a discarded
|
|
history before executing its request. Each new event gets a fresh token even if
|
|
its numeric ID or request UUID was used on the discarded branch.
|
|
|
|
After a mismatch, the client durably records recovery intent, retains pending
|
|
content, clears old history-dependent state and bootstraps against the restored
|
|
server. Clean files reconcile against the restored snapshot. Unsent edits with an
|
|
old base are preserved as separate local documents, with conflict names if needed.
|
|
Interrupted recovery saves are replayable. Backups must be restored with the server
|
|
stopped.
|
|
|
|
WebSocket messages wake HTTP replay; missed notifications are recovered by replay
|
|
and transport liveness checks. Authentication and permanent errors remain visible;
|
|
transient errors retry according to client settings. Observer callback failures
|
|
are isolated from the background sync loop.
|
|
|
|
API v5 clients and servers must be deployed together. V4 databases migrate without
|
|
removing history. The old Obsidian and CLI adapters still require the filesystem
|
|
and persistence contract required by the core engine.
|