no power loss

This commit is contained in:
Andras Schmelczer 2026-09-21 21:29:56 +01:00
commit 2fcdbfd27d
2 changed files with 1374 additions and 3676 deletions

View file

@ -1,8 +1,7 @@
# Sync algorithm
The current API v5 server stores immutable content versions and a separately
versioned file manifest. Clients reconcile snapshots and apply them through a
durable filesystem journal. The server does not merge competing edits.
versioned file manifest. Clients reconcile snapshots with the current local files. The server does not merge competing edits.
## Identity and ordering
@ -19,10 +18,11 @@ UUID is rejected.
## Reconciliation
A new client reads a coherent vault snapshot, applies it durably, then replays
subsequent events. Replayed pages are folded to their final document heads and
manifest before changing local files. This avoids applying intermediate remote
edits that were already reverted.
A new client reads a coherent vault snapshot, applies it, then replays
subsequent events over HTTP. Pages are folded in memory to their final document
heads and manifest before changing local files. Interrupted fetching restarts from
the last incorporated cursor without saved pagination state. This avoids applying
intermediate remote edits that were already reverted.
For text, the client compares the common base, current local bytes and current
remote bytes using `reconcile-text`. Configured extensions and valid UTF-8 without
@ -31,34 +31,36 @@ they do not authorize the server to merge a stale base. Large diff operation set
are sent as snapshots.
If only one side changed, that side is retained. Concurrent unmergeable changes
keep the server version while the journal archives displaced local bytes for
recovery. First contact with existing local text has no shared ancestor and uses
keep the server version. First contact with existing local text has no shared ancestor and uses
an empty parent. Concurrent edits may still require human review.
Paths use a separate three-way decision: keep a local change when the server path
is unchanged, otherwise prefer the server path if both changed. Portable-name
collisions receive deterministic conflict names. A manifest deletion is not
reversed merely because another client edited the document; displaced bytes are
retained locally.
reversed merely because another client edited the document. Local backup archives
are not created.
## Filesystem application
Before moving files, the client persists a journal describing sources,
destinations and replacement snapshots. It stages sources before installing any
destination, so swaps and rename cycles do not overwrite each other. Prepared
content is persisted before installation so a crash does not repeat a merge.
The client applies an in-memory plan directly to the visible namespace, then
saves one metadata snapshot. Swaps and cycles move occupied sources to temporary
conflict names that remain ordinary discoverable files if application stops.
There is no persisted filesystem plan, staged payload, recovery replay or fsync
protocol. An interruption may leave a subset of the operations visible, or partial
file content; the next scan reconciles whatever is present then.
External notifications retain logical document identities. Notifications received
during asynchronous filesystem reads or staging are incorporated before accepting
a source. An editor save to a staged document's old path remains part of that
document. Ignored and oversized physical occupants are protected during conflict
allocation. Adapters report atomic editor saves as content updates and suppress
notifications generated by the engine itself.
Create, delete and move notifications persist logical document identities; content
updates only wake the loop to reread current bytes. Namespace changes detected
before mutations invalidate the plan. Notifications during application are incorporated
between files using their document identities; the next scan rereads content. Content is reread
before replacement. Ignored and oversized occupants stay in place. Adapters
report atomic editor saves as updates and suppress engine-generated notifications.
The filesystem and metadata store have separate durability boundaries. Recovery
replays the journal before scanning, including when network sync is disabled.
Adapters must provide complete scans, coherent snapshots, exclusive vault access,
no-replace renames and durable writes. Recovery artifacts are excluded from sync.
Metadata persistence must replace a complete value atomically, leaving a readable
old/new value after interruption. Metadata and user files need not describe the
same moment: scans repair that disagreement. File creation and rename must reject
occupied destinations; scans must be complete and reads coherent. No guarantee is
made that user bytes or rename identities survive power loss.
## Server restores and failures
@ -68,8 +70,7 @@ The client persists and echoes it; the server rejects a checkpoint from a discar
history before executing its request. Each new event gets a fresh token even if
its numeric ID or request UUID was used on the discarded branch.
After a mismatch, the client durably records recovery intent, retains pending
content, clears old history-dependent state and bootstraps against the restored
After a mismatch, the client records reset intent, clears old history-dependent state and bootstraps against the restored
server. Clean files reconcile against the restored snapshot. Unsent edits with an
old base are preserved as separate local documents, with conflict names if needed.
Interrupted recovery saves are replayable. Backups must be restored with the server

View file

@ -9,8 +9,7 @@ VaultLink works well for most Obsidian vaults, but has some constraints you shou
Only **`.md`** and **`.txt`** files get automatic conflict-free merging.
For other file types, a one-sided change is retained. Concurrent unmergeable
changes keep the server's version and archive displaced local bytes for recovery.
Review recovery artifacts when both devices edit a binary file.
changes keep the server's version. The client does not archive displaced bytes.
### Binary Detection
@ -60,7 +59,7 @@ Rough estimates (varies by vault size and activity):
**Growth**: Version history grows with every change. A 10 MB vault with frequent edits might grow to 100+ MB over months.
Active journals, immutable merge bases and request receipts must not be independently deleted. There is no automatic safe history-pruning protocol.
Immutable merge bases and request receipts must not be independently deleted. There is no automatic safe history-pruning protocol.
### Implications
@ -143,6 +142,14 @@ the restored server; unsent edits are preserved as separate local documents.
Clients and server must be upgraded together. This does not replace independent
backups or protect against arbitrary physical-media corruption.
## Interrupted sync
The client has no filesystem journal or power-loss durability protocol. Interrupted
writes may leave missing, partial or moved files. On restart, normal scans reconcile
the current files, including edits made while the client was stopped. Metadata
saves must remain complete and readable, so the client can continue syncing; exact
pre-interruption file contents and identities are not guaranteed.
## Known Edge Cases
### Simultaneous Deletes and Edits
@ -150,10 +157,10 @@ backups or protect against arbitrary physical-media corruption.
```
User A deletes note.md
User B edits note.md
Result: Manifest deletion removes the file; unsent edited bytes are retained for recovery.
Result: Manifest deletion removes the file, including unsent local edits.
```
Content updates do not restore deleted membership. The client retains displaced bytes in its local recovery directory.
Content updates do not restore deleted membership. There is no local recovery archive.
### Large File Uploads