vault-link/frontend/sync-client/README.md

Sync client (API v4)

The sync coordinator uses one serialized loop per vault and requires Web Crypto (SHA-256). Local notifications and WebSocket vaultChanged hints wake it; complete disk scans and the durable server event log are the sources of truth. Periodic reconciliation is opt-in: set syncIntervalMs to a positive interval to repair missed notifications. Unset or 0 disables polling. Create, delete and move notifications persist logical identity changes before resolving, including while offline. Shutdown waits for those saves. Content-update notifications only wake reconciliation; scans read the latest bytes, including after restart. Notifications describe logical external changes: adapters suppress self-generated create/delete/move notifications, and report atomic editor saves as content updates. This distinguishes a deliberate delete/recreate from an edit when the host supplies reliable logical events. Raw OS watchers instead send content-update hints for every event and let scans infer the current namespace. Use waitUntilFinished() to await the active attempt; errors leave pending requests saved and are reported to the caller. Transient failures retry; authentication and protocol errors wait for an explicit wake or settings change. Pausing or destroying the client aborts its HTTP session, including body reads; resuming starts a fresh session. Connection checks can run while sync is disabled. WebSocket shutdown retires callbacks immediately and drains message handlers already running; it does not wait for the transport's close handshake.

Code layout

Start with src/sync-operations/syncer.ts for the sync workflow: resolve the saved upload, fetch the complete remote history, apply it, and prepare the next upload. It coordinates the following components:

Component	Responsibility
sync-scheduler.ts	Serialize attempts, coalesce notifications, stop, and schedule polling or retries.
local-change-queue.ts	Persist create/delete/move notifications and invalidate plans when identities change.
scan.ts	Inventory disk files, infer identities, and commit the scan before consuming notifications.
remote-changes.ts	Fetch and validate every event page before returning the final remote state.
content-sync.ts	Fetch merge bases, plan content writes, encode uploads, and maintain the optional diff cache.
upload-queue.ts	Persist exact requests, handle acknowledgements, and retain rejected requests for retry.
file-operations/	Apply paths and content to disk, incorporating notifications between files.
persistence/database.ts	Define StoredDatabase and atomically commit or reload it.
persistence/client-metadata-store.ts	Serialize database, settings, checkpoint, and notification saves into the shared store.

requestSync() schedules work; synchronize() runs an attempt; refreshLocalFiles() flushes notifications and calls scanLocalFiles(). initializeFromServerSnapshot() applies the initial server snapshot, while applyRemoteChanges() merges and applies an observed remote state. These operations keep the same persisted database format.

The scheduler's SyncPhase enum makes its transitions explicit:

Event	Transition
Start or explicit request	Idle, waiting, or blocked → run requested
Begin a pass	Run requested → running
Notification during a pass	Running → run requested; repeated notifications coalesce
Successful completion	Running → idle; optionally schedule the polling timer
Transient failure	Running → waiting to retry
Authentication or permanent failure	Running → blocked until an explicit request
Stop	Running → stopping → stopped; idle timers are cancelled immediately

From frontend/, run npm run lint -w sync-client, npm run check -w sync-client, and npm run test -w sync-client. npm run test:harness covers interrupted saves, file application, and recovery. Protocol types under src/services/types/ are generated from Rust and excluded from ESLint; handwritten TypeScript requires braces.

State and policy

The persisted state contains the incorporated server file manifest, the current local identity map, an optional event cursor, and exact pending submissions. An absent cursor means the client must fetch a fresh vault snapshot; there are no separate initialization or recovery flags and no saved bootstrap snapshot. Each document stores its base version/hash, its last observed disk hash, and a remote head only while that content is deferred. The observed hash also records local file presence; missing remote downloads are not local deletions. Physical path exclusions are rebuilt by scans. Rejection details live on the retained request, without duplicate per-document or manifest rejection records. A received server file manifest becomes the merge base when its application metadata is saved. Successful submissions advance the base to the submitted content snapshot, preserving later local edits. Unknown outcomes are retried unchanged before incorporating newer remote state. HTTP is the sole source of remote events; WebSocket messages only wake the loop. Contiguous event pages are folded in memory into their final manifest and content heads before merging. An interrupted fetch restarts from the last incorporated cursor; pagination progress is not persisted. Intermediate changes that the server has already reverted must not overwrite local edits.

A permanent upload rejection pauses all sync, including after restart. The exact pending request and rejection message stay in metadata, and the failure remains visible in history and waitUntilFinished(). Resetting the client or changing settings retries that request before reading newer remote changes. This recovers earlier attempts that committed despite a lost reply without separate receipt bookkeeping. Local edits made while paused are uploaded after the pending request is resolved; editing a file alone does not retry a rejected request.

For each document ID (absence means deletion), use the server path if local is unchanged, use local if the server is unchanged, and use the server when both changed. Then resolve cross-document path conflicts: a retained server path wins, UUID ordering breaks other ties, and displaced documents receive deterministic (conflict UUID) names. This also handles file/ancestor and portable-name aliases. Nonportable local names are repaired client-side before a file manifest is published. Allocated filename components fit within 255 UTF-8 bytes, including conflict suffixes. The protocol does not impose a universal total-path byte limit; filesystem-specific limits surface from the adapter instead of silently flattening a name. Directories are implicit; empty directories are not synced.

• Content is uploaded before new membership; the client chooses and persists UUIDs.
• Text uses three-way reconcile. First contact at a matching path adopts the remote ID and reconciles differing text with an empty parent.
• In-memory snapshots contain bytes and their hash. Base64 encoding is limited to wire payloads; pending requests persist the exact payload needed for retries.
• Concurrent binary/unmergeable edits keep the server version. If only one side changed, keep that side. No local backup archive is created.
• Deletion follows the file manifest decision, including when the file has local edits.
• Notified moves preserve IDs. Offline inference requires a unique nonempty hash match on both sides; uncertain cases become delete/create.
• Ignored and oversized files preserve their disk paths and remote membership. The original manifest base is retained while excluded, so becoming eligible resumes reconciliation without losing local renames. Not-yet-downloaded files are not mistaken for local deletions.
• Bytes and line endings are preserved. File modification times are not tracked or synchronized. Optional editor cursors are repositioned when text is reconciled.

Restored server histories

Successful HTTP responses carry an X-Vault-Link-History checkpoint. The client persists it before consuming the response and supplies it on subsequent reads and writes. WebSocket events wake HTTP replay, which validates that checkpoint. A restored database cannot silently reuse a version number: a missing or different event incarnation returns a history mismatch before executing the request.

On mismatch the client scans local work and records a reset before clearing the old checkpoint. It clears old requests, receipts, cursors, bases and diff-cache entries. Clean files retain only their hash and reconcile against the restored snapshot. A file edited after this reset is also preserved as a separate local document; recovery never persists another copy of its content. Unsent edits with an old base become separate local documents; path conflicts preserve them under conflict names instead of applying a diff to an unrelated restored version. Recovery resumes after interrupted state saves. Only the current persisted state format is supported; old client stores are not converted.

Status observers cannot stop the background engine by throwing or rejecting. Their failures are reported, and remaining observers continue receiving updates. Remote cursor tracking retains the latest future update and the last usable position for each device, so cursors remain visible while file content catches up.

Required adapter contract

FileSystemOperations and PersistenceProvider define the adapter operations. The CLI and desktop Obsidian share a Node filesystem adapter and atomic full-record metadata persistence. Mobile Obsidian uses its host storage APIs. The E2E runner builds and tests both integrations, including actual CLI processes. The sync client does not coordinate separate processes accessing the same vault.

The filesystem adapter must provide complete scans, coherent snapshots, recursive directory creation, direct file writes, atomic text updates, no-replace renames, and deletion of individual regular files. Directory pruning uses only rmdir, never recursive file removal. Reject links, special files and paths escaping the vault. An unreadable file or directory must fail the scan rather than look like a deletion. Optional editor cursor metadata is applied with file content. Text reconciliation runs inside atomicUpdateText against the latest text and cursors. Files are updated before metadata is saved; an interrupted save can safely repeat the content merge.

User-file writes have no power-loss durability requirement. The engine has no filesystem journal, recovery staging area, replay, backup archive, or flush protocol. Swaps and cycles temporarily move occupied files to visible conflict names; a fresh scan can discover them if application stops partway through. Partial writes and edits made before, during or after an interruption are treated as current disk content. The next sync reconciles that content; it does not finish a saved filesystem plan. Power loss may lose bytes or rename identity and may cause a merge to run again.

Metadata is the one atomic boundary: persistence save replaces a complete value and an interrupted/failed save leaves an old or new complete value, never a torn object. A failed save is reloaded before retrying. No ordering between metadata and user-file durability is required. Settings and database saves are serialized. Keep metadata outside the user-file namespace or in .vault-link-sync, which is excluded from scans. Changing a client's vault once it holds identities, offline notifications, pending requests or an incorporated event cursor requires a separate state store.

Filesystem application saves metadata once when it finishes. Namespace changes detected before mutations invalidate the plan. Changes during application are incorporated between files using their document identities; the next scan rereads content. Content is reread before replacement. Unexpected eligible occupants receive visible conflict names; ignored and oversized occupants stay in place. External writers can still change files between observations, so these checks do not promise transactional file contents or perfect identity reconstruction. There is no persisted-state migration.