no power loss
This commit is contained in:
parent
d56084e4bc
commit
2fcdbfd27d
43 changed files with 1374 additions and 3676 deletions
|
|
@ -1,3 +1,8 @@
|
||||||
|
> Historical handover: its client journaling, recovery-archive and filesystem
|
||||||
|
> durability descriptions are superseded by the direct-application model in
|
||||||
|
> [the sync client README](frontend/sync-client/README.md). The current client
|
||||||
|
> saves complete metadata and reconciles current disk contents after interruption.
|
||||||
|
|
||||||
# Sync engine v4 handover
|
# Sync engine v4 handover
|
||||||
|
|
||||||
Prepared on 2026-09-19. This records the implementation delivered in the preceding
|
Prepared on 2026-09-19. This records the implementation delivered in the preceding
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,7 @@
|
||||||
# Sync algorithm
|
# Sync algorithm
|
||||||
|
|
||||||
The current API v5 server stores immutable content versions and a separately
|
The current API v5 server stores immutable content versions and a separately
|
||||||
versioned file manifest. Clients reconcile snapshots and apply them through a
|
versioned file manifest. Clients reconcile snapshots with the current local files. The server does not merge competing edits.
|
||||||
durable filesystem journal. The server does not merge competing edits.
|
|
||||||
|
|
||||||
## Identity and ordering
|
## Identity and ordering
|
||||||
|
|
||||||
|
|
@ -19,10 +18,11 @@ UUID is rejected.
|
||||||
|
|
||||||
## Reconciliation
|
## Reconciliation
|
||||||
|
|
||||||
A new client reads a coherent vault snapshot, applies it durably, then replays
|
A new client reads a coherent vault snapshot, applies it, then replays
|
||||||
subsequent events. Replayed pages are folded to their final document heads and
|
subsequent events over HTTP. Pages are folded in memory to their final document
|
||||||
manifest before changing local files. This avoids applying intermediate remote
|
heads and manifest before changing local files. Interrupted fetching restarts from
|
||||||
edits that were already reverted.
|
the last incorporated cursor without saved pagination state. This avoids applying
|
||||||
|
intermediate remote edits that were already reverted.
|
||||||
|
|
||||||
For text, the client compares the common base, current local bytes and current
|
For text, the client compares the common base, current local bytes and current
|
||||||
remote bytes using `reconcile-text`. Configured extensions and valid UTF-8 without
|
remote bytes using `reconcile-text`. Configured extensions and valid UTF-8 without
|
||||||
|
|
@ -31,34 +31,36 @@ they do not authorize the server to merge a stale base. Large diff operation set
|
||||||
are sent as snapshots.
|
are sent as snapshots.
|
||||||
|
|
||||||
If only one side changed, that side is retained. Concurrent unmergeable changes
|
If only one side changed, that side is retained. Concurrent unmergeable changes
|
||||||
keep the server version while the journal archives displaced local bytes for
|
keep the server version. First contact with existing local text has no shared ancestor and uses
|
||||||
recovery. First contact with existing local text has no shared ancestor and uses
|
|
||||||
an empty parent. Concurrent edits may still require human review.
|
an empty parent. Concurrent edits may still require human review.
|
||||||
|
|
||||||
Paths use a separate three-way decision: keep a local change when the server path
|
Paths use a separate three-way decision: keep a local change when the server path
|
||||||
is unchanged, otherwise prefer the server path if both changed. Portable-name
|
is unchanged, otherwise prefer the server path if both changed. Portable-name
|
||||||
collisions receive deterministic conflict names. A manifest deletion is not
|
collisions receive deterministic conflict names. A manifest deletion is not
|
||||||
reversed merely because another client edited the document; displaced bytes are
|
reversed merely because another client edited the document. Local backup archives
|
||||||
retained locally.
|
are not created.
|
||||||
|
|
||||||
## Filesystem application
|
## Filesystem application
|
||||||
|
|
||||||
Before moving files, the client persists a journal describing sources,
|
The client applies an in-memory plan directly to the visible namespace, then
|
||||||
destinations and replacement snapshots. It stages sources before installing any
|
saves one metadata snapshot. Swaps and cycles move occupied sources to temporary
|
||||||
destination, so swaps and rename cycles do not overwrite each other. Prepared
|
conflict names that remain ordinary discoverable files if application stops.
|
||||||
content is persisted before installation so a crash does not repeat a merge.
|
There is no persisted filesystem plan, staged payload, recovery replay or fsync
|
||||||
|
protocol. An interruption may leave a subset of the operations visible, or partial
|
||||||
|
file content; the next scan reconciles whatever is present then.
|
||||||
|
|
||||||
External notifications retain logical document identities. Notifications received
|
Create, delete and move notifications persist logical document identities; content
|
||||||
during asynchronous filesystem reads or staging are incorporated before accepting
|
updates only wake the loop to reread current bytes. Namespace changes detected
|
||||||
a source. An editor save to a staged document's old path remains part of that
|
before mutations invalidate the plan. Notifications during application are incorporated
|
||||||
document. Ignored and oversized physical occupants are protected during conflict
|
between files using their document identities; the next scan rereads content. Content is reread
|
||||||
allocation. Adapters report atomic editor saves as content updates and suppress
|
before replacement. Ignored and oversized occupants stay in place. Adapters
|
||||||
notifications generated by the engine itself.
|
report atomic editor saves as updates and suppress engine-generated notifications.
|
||||||
|
|
||||||
The filesystem and metadata store have separate durability boundaries. Recovery
|
Metadata persistence must replace a complete value atomically, leaving a readable
|
||||||
replays the journal before scanning, including when network sync is disabled.
|
old/new value after interruption. Metadata and user files need not describe the
|
||||||
Adapters must provide complete scans, coherent snapshots, exclusive vault access,
|
same moment: scans repair that disagreement. File creation and rename must reject
|
||||||
no-replace renames and durable writes. Recovery artifacts are excluded from sync.
|
occupied destinations; scans must be complete and reads coherent. No guarantee is
|
||||||
|
made that user bytes or rename identities survive power loss.
|
||||||
|
|
||||||
## Server restores and failures
|
## Server restores and failures
|
||||||
|
|
||||||
|
|
@ -68,8 +70,7 @@ The client persists and echoes it; the server rejects a checkpoint from a discar
|
||||||
history before executing its request. Each new event gets a fresh token even if
|
history before executing its request. Each new event gets a fresh token even if
|
||||||
its numeric ID or request UUID was used on the discarded branch.
|
its numeric ID or request UUID was used on the discarded branch.
|
||||||
|
|
||||||
After a mismatch, the client durably records recovery intent, retains pending
|
After a mismatch, the client records reset intent, clears old history-dependent state and bootstraps against the restored
|
||||||
content, clears old history-dependent state and bootstraps against the restored
|
|
||||||
server. Clean files reconcile against the restored snapshot. Unsent edits with an
|
server. Clean files reconcile against the restored snapshot. Unsent edits with an
|
||||||
old base are preserved as separate local documents, with conflict names if needed.
|
old base are preserved as separate local documents, with conflict names if needed.
|
||||||
Interrupted recovery saves are replayable. Backups must be restored with the server
|
Interrupted recovery saves are replayable. Backups must be restored with the server
|
||||||
|
|
|
||||||
|
|
@ -9,8 +9,7 @@ VaultLink works well for most Obsidian vaults, but has some constraints you shou
|
||||||
Only **`.md`** and **`.txt`** files get automatic conflict-free merging.
|
Only **`.md`** and **`.txt`** files get automatic conflict-free merging.
|
||||||
|
|
||||||
For other file types, a one-sided change is retained. Concurrent unmergeable
|
For other file types, a one-sided change is retained. Concurrent unmergeable
|
||||||
changes keep the server's version and archive displaced local bytes for recovery.
|
changes keep the server's version. The client does not archive displaced bytes.
|
||||||
Review recovery artifacts when both devices edit a binary file.
|
|
||||||
|
|
||||||
### Binary Detection
|
### Binary Detection
|
||||||
|
|
||||||
|
|
@ -60,7 +59,7 @@ Rough estimates (varies by vault size and activity):
|
||||||
|
|
||||||
**Growth**: Version history grows with every change. A 10 MB vault with frequent edits might grow to 100+ MB over months.
|
**Growth**: Version history grows with every change. A 10 MB vault with frequent edits might grow to 100+ MB over months.
|
||||||
|
|
||||||
Active journals, immutable merge bases and request receipts must not be independently deleted. There is no automatic safe history-pruning protocol.
|
Immutable merge bases and request receipts must not be independently deleted. There is no automatic safe history-pruning protocol.
|
||||||
|
|
||||||
### Implications
|
### Implications
|
||||||
|
|
||||||
|
|
@ -143,6 +142,14 @@ the restored server; unsent edits are preserved as separate local documents.
|
||||||
Clients and server must be upgraded together. This does not replace independent
|
Clients and server must be upgraded together. This does not replace independent
|
||||||
backups or protect against arbitrary physical-media corruption.
|
backups or protect against arbitrary physical-media corruption.
|
||||||
|
|
||||||
|
## Interrupted sync
|
||||||
|
|
||||||
|
The client has no filesystem journal or power-loss durability protocol. Interrupted
|
||||||
|
writes may leave missing, partial or moved files. On restart, normal scans reconcile
|
||||||
|
the current files, including edits made while the client was stopped. Metadata
|
||||||
|
saves must remain complete and readable, so the client can continue syncing; exact
|
||||||
|
pre-interruption file contents and identities are not guaranteed.
|
||||||
|
|
||||||
## Known Edge Cases
|
## Known Edge Cases
|
||||||
|
|
||||||
### Simultaneous Deletes and Edits
|
### Simultaneous Deletes and Edits
|
||||||
|
|
@ -150,10 +157,10 @@ backups or protect against arbitrary physical-media corruption.
|
||||||
```
|
```
|
||||||
User A deletes note.md
|
User A deletes note.md
|
||||||
User B edits note.md
|
User B edits note.md
|
||||||
Result: Manifest deletion removes the file; unsent edited bytes are retained for recovery.
|
Result: Manifest deletion removes the file, including unsent local edits.
|
||||||
```
|
```
|
||||||
|
|
||||||
Content updates do not restore deleted membership. The client retains displaced bytes in its local recovery directory.
|
Content updates do not restore deleted membership. There is no local recovery archive.
|
||||||
|
|
||||||
### Large File Uploads
|
### Large File Uploads
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -74,7 +74,7 @@ Network retry timers remain enabled. Separate fake-clock tests in
|
||||||
A barrier checks byte-for-byte replica equality **and** the server's canonical
|
A barrier checks byte-for-byte replica equality **and** the server's canonical
|
||||||
UUID/path map, file bytes, independently computed base/observed hashes, content
|
UUID/path map, file bytes, independently computed base/observed hashes, content
|
||||||
and manifest CAS parents, event watermark,
|
and manifest CAS parents, event watermark,
|
||||||
contiguous event log, and absence of pending requests/application journals.
|
contiguous event log, and absence of pending requests.
|
||||||
Internal recovery files are excluded from the user-visible file list, but are
|
Internal recovery files are excluded from the user-visible file list, but are
|
||||||
present in failure disk images. Scenario assertions separately check intended
|
present in failure disk images. Scenario assertions separately check intended
|
||||||
content, document identity, or preservation markers; equality alone is not an
|
content, document identity, or preservation markers; equality alone is not an
|
||||||
|
|
@ -106,8 +106,8 @@ distinct UUIDs at one path.
|
||||||
- `wait-for-response-drop`: explicit checkpoint; every armed fault must fire.
|
- `wait-for-response-drop`: explicit checkpoint; every armed fault must fire.
|
||||||
Every interrupted request must be retried with an identical request ID, target
|
Every interrupted request must be retried with an identical request ID, target
|
||||||
and body before a different CAS is sent. Fault kinds match exactly.
|
and body before a different CAS is sent. Fault kinds match exactly.
|
||||||
- `rename-next-write`: inject an editor rename after the engine's durable install
|
- `rename-next-write`: inject an editor rename after the engine's content write,
|
||||||
rename. The historical name is retained for fixtures; v4 never writes in place.
|
before its metadata save.
|
||||||
- `pause-server`, `resume-server`: SIGSTOP/SIGCONT, not a crash.
|
- `pause-server`, `resume-server`: SIGSTOP/SIGCONT, not a crash.
|
||||||
- `crash-server`, `restart-server`: SIGKILL and restart with the same database/port.
|
- `crash-server`, `restart-server`: SIGKILL and restart with the same database/port.
|
||||||
Server-control scenarios have dedicated processes.
|
Server-control scenarios have dedicated processes.
|
||||||
|
|
|
||||||
|
|
@ -43,12 +43,8 @@ export class DeterministicAgent {
|
||||||
});
|
});
|
||||||
this.disk.boundary = async (label) => {
|
this.disk.boundary = async (label) => {
|
||||||
const rename = this.nextWriteRename;
|
const rename = this.nextWriteRename;
|
||||||
// v4 installs through an exclusive rename, never write-in-place.
|
// Inject after the content write, before the metadata save.
|
||||||
if (
|
if (rename && label === `durable:write:${rename.oldPath}`) {
|
||||||
rename &&
|
|
||||||
label.startsWith("durable:rename:") &&
|
|
||||||
label.endsWith(`->${rename.oldPath}`)
|
|
||||||
) {
|
|
||||||
this.nextWriteRename = undefined;
|
this.nextWriteRename = undefined;
|
||||||
await this.rename(rename.oldPath, rename.newPath);
|
await this.rename(rename.oldPath, rename.newPath);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -13,10 +13,8 @@ export const sameDocIdCollapseAfterRemoteQuickWriteAndPendingRenameTest: TestDef
|
||||||
steps: [
|
steps: [
|
||||||
{ type: "enable-sync", client: 0 },
|
{ type: "enable-sync", client: 0 },
|
||||||
|
|
||||||
// Create a deleted latest version before client 1 joins.
|
// Seed a deleted document before client 1 bootstraps. Later creates
|
||||||
// Catch-up will advance MinCovered with a non-contiguous id,
|
// at the reused path must still receive separate identities.
|
||||||
// keeping client 1's create lastSeen low enough to exercise
|
|
||||||
// the server's same-doc merge path from the e2e failure.
|
|
||||||
{
|
{
|
||||||
type: "create",
|
type: "create",
|
||||||
client: 0,
|
client: 0,
|
||||||
|
|
|
||||||
|
|
@ -33,11 +33,8 @@ export const selfMergePendingRenameAliasesSecondCreateTest: TestDefinition = {
|
||||||
{ type: "enable-sync", client: 1 },
|
{ type: "enable-sync", client: 1 },
|
||||||
{ type: "barrier" },
|
{ type: "barrier" },
|
||||||
|
|
||||||
// Pause client 0's WS so its MinCovered watermark stays at 0
|
// Delay socket notifications while client 0 continues local work.
|
||||||
// through the whole bug sequence. The merge condition the
|
// HTTP catchup must preserve the two generations at the reused path.
|
||||||
// server is going to fire is `creation_vuid > last_seen`; with
|
|
||||||
// a non-zero gap the same-device second create gets merged
|
|
||||||
// into the same-device first create.
|
|
||||||
{ type: "pause-websocket", client: 0 },
|
{ type: "pause-websocket", client: 0 },
|
||||||
|
|
||||||
// Client 1 commits a doc to push the server's vuid above 0.
|
// Client 1 commits a doc to push the server's vuid above 0.
|
||||||
|
|
|
||||||
|
|
@ -134,7 +134,6 @@ export class NodeFileSystemOperations implements FileSystemOperations {
|
||||||
await fs.unlink(fullPath).catch((error: unknown) => {
|
await fs.unlink(fullPath).catch((error: unknown) => {
|
||||||
if (!this.isMissing(error)) throw error;
|
if (!this.isMissing(error)) throw error;
|
||||||
});
|
});
|
||||||
await this.syncDirectory(path.dirname(fullPath));
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
`Failed to delete file ${fullPath}: ${error instanceof Error ? error.message : String(error)}`
|
`Failed to delete file ${fullPath}: ${error instanceof Error ? error.message : String(error)}`
|
||||||
|
|
@ -170,34 +169,13 @@ export class NodeFileSystemOperations implements FileSystemOperations {
|
||||||
const tmpPath = path.join(tmpDir, `atomic-write-${randomUUID()}.tmp`);
|
const tmpPath = path.join(tmpDir, `atomic-write-${randomUUID()}.tmp`);
|
||||||
try {
|
try {
|
||||||
await fs.writeFile(tmpPath, content, encoding);
|
await fs.writeFile(tmpPath, content, encoding);
|
||||||
const fd = await fs.open(tmpPath, "r");
|
|
||||||
try {
|
|
||||||
await fd.datasync();
|
|
||||||
} finally {
|
|
||||||
await fd.close();
|
|
||||||
}
|
|
||||||
await fs.rename(tmpPath, fullPath);
|
await fs.rename(tmpPath, fullPath);
|
||||||
await this.syncDirectory(path.dirname(fullPath));
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
await fs.unlink(tmpPath).catch(() => undefined);
|
await fs.unlink(tmpPath).catch(() => undefined);
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Make the rename durable by fsync'ing the destination's parent directory.
|
|
||||||
// Skipped on Windows: fsync on a directory handle isn't supported there
|
|
||||||
private async syncDirectory(dir: string): Promise<void> {
|
|
||||||
if (process.platform === "win32") {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const fd = await fs.open(dir, "r");
|
|
||||||
try {
|
|
||||||
await fd.sync();
|
|
||||||
} finally {
|
|
||||||
await fd.close();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private async deleteDirectoryTree(fullPath: string): Promise<void> {
|
private async deleteDirectoryTree(fullPath: string): Promise<void> {
|
||||||
const entry = await fs.lstat(fullPath).catch((error: unknown) => {
|
const entry = await fs.lstat(fullPath).catch((error: unknown) => {
|
||||||
if (this.isMissing(error)) return undefined;
|
if (this.isMissing(error)) return undefined;
|
||||||
|
|
@ -220,7 +198,6 @@ export class NodeFileSystemOperations implements FileSystemOperations {
|
||||||
}
|
}
|
||||||
|
|
||||||
await fs.rmdir(fullPath);
|
await fs.rmdir(fullPath);
|
||||||
await this.syncDirectory(path.dirname(fullPath));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private isMissing(error: unknown): boolean {
|
private isMissing(error: unknown): boolean {
|
||||||
|
|
|
||||||
|
|
@ -19,8 +19,8 @@
|
||||||
"dev": "concurrently --kill-others \"npm run dev -w sync-client\" \"npm run dev -w obsidian-plugin\"",
|
"dev": "concurrently --kill-others \"npm run dev -w sync-client\" \"npm run dev -w obsidian-plugin\"",
|
||||||
"test": "npm run test --workspaces",
|
"test": "npm run test --workspaces",
|
||||||
"check:harness": "tsc --noEmit -p test-support/tsconfig.json && tsc --noEmit -p deterministic-tests/tsconfig.json && tsc --noEmit -p test-client/tsconfig.json",
|
"check:harness": "tsc --noEmit -p test-support/tsconfig.json && tsc --noEmit -p deterministic-tests/tsconfig.json && tsc --noEmit -p test-client/tsconfig.json",
|
||||||
"test:harness": "npm run check:harness && tsx --test 'test-support/harness.test.ts' 'test-support/oracle-regressions.test.ts' 'test-support/recovery.test.ts' 'test-support/journal-interleavings.test.ts' 'test-support/observer.test.ts' 'test-support/event-delivery.test.ts' 'test-support/polling.test.ts' 'test-support/engine-regressions.test.ts' 'test-support/sync-audit.test.ts' 'test-client/src/**/*.test.ts' '../scripts/e2e-runner.test.mjs'",
|
"test:harness": "npm run check:harness && tsx --test 'test-support/harness.test.ts' 'test-support/oracle-regressions.test.ts' 'test-support/recovery.test.ts' 'test-support/file-application.test.ts' 'test-support/observer.test.ts' 'test-support/event-delivery.test.ts' 'test-support/polling.test.ts' 'test-support/engine-regressions.test.ts' 'test-support/sync-audit.test.ts' 'test-client/src/**/*.test.ts' '../scripts/e2e-runner.test.mjs'",
|
||||||
"test:protocol": "tsx --test 'test-support/protocol.test.ts' 'test-support/journal-protocol.test.ts' 'test-support/content-limits.test.ts' 'test-support/server-restore.test.ts' 'test-support/protocol-audit.test.ts' 'test-support/real-disk.test.ts'",
|
"test:protocol": "tsx --test 'test-support/protocol.test.ts' 'test-support/restart-reconciliation.test.ts' 'test-support/content-limits.test.ts' 'test-support/server-restore.test.ts' 'test-support/protocol-audit.test.ts' 'test-support/real-disk.test.ts'",
|
||||||
"lint": "eslint --fix sync-client obsidian-plugin test-client deterministic-tests local-client-cli && prettier --write \"**/*.ts\"",
|
"lint": "eslint --fix sync-client obsidian-plugin test-client deterministic-tests local-client-cli && prettier --write \"**/*.ts\"",
|
||||||
"update": "ncu -u -ws"
|
"update": "ncu -u -ws"
|
||||||
},
|
},
|
||||||
|
|
|
||||||
|
|
@ -4,28 +4,30 @@ The engine uses one serialized loop per vault and requires Web Crypto (SHA-256).
|
||||||
messages wake it; complete disk scans and the durable server event log are the
|
messages wake it; complete disk scans and the durable server event log are the
|
||||||
sources of truth. Periodic reconciliation is opt-in: set `syncIntervalMs` to a
|
sources of truth. Periodic reconciliation is opt-in: set `syncIntervalMs` to a
|
||||||
positive interval to repair missed notifications. Unset or `0` disables polling.
|
positive interval to repair missed notifications. Unset or `0` disables polling.
|
||||||
`syncLocallyCreatedFile`, `syncLocallyUpdatedFile`, and `syncLocallyDeletedFile`
|
Create, delete and move notifications persist logical identity changes before
|
||||||
durably record logical file changes before resolving, including while offline.
|
resolving, including while offline. Shutdown waits for those saves. Content-update
|
||||||
Shutdown waits for those saves. Notifications describe logical external changes: adapters suppress
|
notifications only wake reconciliation; scans read the latest bytes, including after
|
||||||
|
restart. Notifications describe logical external changes: adapters suppress
|
||||||
self-generated create/delete/move notifications, and report atomic editor saves
|
self-generated create/delete/move notifications, and report atomic editor saves
|
||||||
as content updates. This distinguishes a deliberate delete/recreate from an edit.
|
as content updates. This distinguishes a deliberate delete/recreate from an edit.
|
||||||
Use `waitUntilFinished()` to await the active attempt; errors leave
|
Use `waitUntilFinished()` to await the active attempt; errors leave
|
||||||
pending work durable and are reported to the caller. Transient failures retry;
|
pending requests saved and are reported to the caller. Transient failures retry;
|
||||||
authentication and protocol errors wait for an explicit wake or settings change.
|
authentication and protocol errors wait for an explicit wake or settings change.
|
||||||
|
|
||||||
## State and policy
|
## State and policy
|
||||||
|
|
||||||
The persisted state contains the incorporated server file manifest, the current
|
The persisted state contains the incorporated server file manifest, the current
|
||||||
local identity map, and the exact pending submission. A received server file
|
local identity map, and the exact pending submission. A received server file
|
||||||
manifest is not a merge base until its local application is durable. Successful
|
manifest becomes the merge base when its application metadata is saved. Successful
|
||||||
submissions advance the base to the submitted content snapshot, preserving later
|
submissions advance the base to the submitted content snapshot, preserving later
|
||||||
local edits. Unknown outcomes are retried unchanged before incorporating newer
|
local edits. Unknown outcomes are retried unchanged before incorporating newer
|
||||||
remote state. Each contiguous event batch is folded into its final manifest and
|
remote state. HTTP is the sole source of remote events; WebSocket messages only
|
||||||
content heads before merging. Intermediate changes that the server has already
|
wake the loop. Contiguous event pages are folded in memory into their final manifest
|
||||||
reverted must not overwrite local edits.
|
and content heads before merging. An interrupted fetch restarts from the last
|
||||||
|
incorporated cursor; pagination progress is not persisted. Intermediate changes
|
||||||
|
that the server has already reverted must not overwrite local edits.
|
||||||
|
|
||||||
A permanent upload rejection is recorded locally, with content retained for
|
A permanent upload rejection is recorded in metadata. An unchanged rejected payload is skipped so other files can sync; the
|
||||||
recovery. An unchanged rejected payload is skipped so other files can sync; the
|
|
||||||
failure remains visible in history and `waitUntilFinished()`. Editing it, resetting
|
failure remains visible in history and `waitUntilFinished()`. Editing it, resetting
|
||||||
the client, or changing settings permits a fresh attempt. Earlier attempts that
|
the client, or changing settings permits a fresh attempt. Earlier attempts that
|
||||||
committed despite a lost reply are incorporated through normal event replay.
|
committed despite a lost reply are incorporated through normal event replay.
|
||||||
|
|
@ -44,10 +46,9 @@ are implicit; empty directories are not synced.
|
||||||
- Content is uploaded before new membership; the client chooses and persists UUIDs.
|
- Content is uploaded before new membership; the client chooses and persists UUIDs.
|
||||||
- Text uses three-way `reconcile`. First contact at a matching path adopts the
|
- Text uses three-way `reconcile`. First contact at a matching path adopts the
|
||||||
remote ID and reconciles differing text with an empty parent.
|
remote ID and reconciles differing text with an empty parent.
|
||||||
- Concurrent binary/unmergeable edits keep the server version and retain
|
- Concurrent binary/unmergeable edits keep the server version. If only one side
|
||||||
displaced local bytes in recovery. If only one side changed, keep that side.
|
changed, keep that side. No local backup archive is created.
|
||||||
- Deletion follows the file manifest decision. Displaced edited bytes are retained
|
- Deletion follows the file manifest decision, including when the file has local edits.
|
||||||
locally, without automatically restoring membership.
|
|
||||||
- Notified moves preserve IDs. Offline inference requires a unique nonempty hash
|
- Notified moves preserve IDs. Offline inference requires a unique nonempty hash
|
||||||
match on both sides; uncertain cases become delete/create.
|
match on both sides; uncertain cases become delete/create.
|
||||||
- Ignored and oversized files preserve their disk paths and remote membership.
|
- Ignored and oversized files preserve their disk paths and remote membership.
|
||||||
|
|
@ -65,9 +66,8 @@ writes. WebSocket events wake HTTP replay, which validates that checkpoint. A
|
||||||
restored database cannot silently reuse a version number: a missing or different
|
restored database cannot silently reuse a version number: a missing or different
|
||||||
event incarnation returns a history mismatch before executing the request.
|
event incarnation returns a history mismatch before executing the request.
|
||||||
|
|
||||||
On mismatch the client first recovers its filesystem journal and scans local
|
On mismatch the client scans local work and records a reset before clearing the
|
||||||
work. It durably records a reset before clearing the old checkpoint, archives
|
old checkpoint. It clears old requests, receipts, cursors, bases and diff-cache
|
||||||
pending content requests, and clears old receipts, cursors, bases and diff-cache
|
|
||||||
entries. Clean files reconcile against the restored snapshot. Unsent edits with
|
entries. Clean files reconcile against the restored snapshot. Unsent edits with
|
||||||
an old base become separate local documents; path conflicts preserve them under
|
an old base become separate local documents; path conflicts preserve them under
|
||||||
conflict names instead of applying a diff to an unrelated restored version.
|
conflict names instead of applying a diff to an unrelated restored version.
|
||||||
|
|
@ -79,59 +79,37 @@ Their failures are reported, and remaining observers continue receiving updates.
|
||||||
|
|
||||||
## Required adapter contract
|
## Required adapter contract
|
||||||
|
|
||||||
`FileSystemOperations` and `PersistenceProvider` define the adapter operations
|
`FileSystemOperations` and `PersistenceProvider` define the adapter operations.
|
||||||
needed by the recovery model. The Obsidian and CLI adapters have intentionally
|
The Obsidian and CLI adapters still need to adopt the current core interface.
|
||||||
not been updated in this rewrite.
|
The sync client does not coordinate separate processes accessing the same vault.
|
||||||
|
|
||||||
The sync client does not coordinate separate processes accessing the same local
|
The filesystem adapter must provide complete scans, coherent snapshots, recursive
|
||||||
vault.
|
directory creation, exclusive file creation, no-replace renames, and deletion of
|
||||||
|
individual regular files. Directory pruning uses only `rmdir`, never recursive
|
||||||
|
file removal. Reject links, special files and paths escaping the vault. An unreadable
|
||||||
|
file or directory must fail the scan rather than look like a deletion. Optional
|
||||||
|
editor cursor metadata is applied with file content.
|
||||||
|
|
||||||
The filesystem adapter must provide:
|
User-file writes have no power-loss durability requirement. There is no filesystem
|
||||||
|
journal, hidden staging area, replay, backup archive, or flush protocol. Swaps and
|
||||||
|
cycles temporarily move occupied files to visible conflict names; a fresh scan
|
||||||
|
can discover them if application stops partway through. Partial writes and edits
|
||||||
|
made before, during or after an interruption are treated as current disk content.
|
||||||
|
The next sync reconciles that content; it does not finish a saved filesystem plan.
|
||||||
|
Power loss may lose bytes or rename identity and may cause a merge to run again.
|
||||||
|
|
||||||
- Complete scans and coherent content snapshots. Missing and unreadable must
|
Metadata is the one atomic boundary: persistence `save` replaces a complete value
|
||||||
be distinguished; reject read races rather than return a partial snapshot.
|
and an interrupted/failed save leaves an old or new complete value, never a torn
|
||||||
- Vault-relative operations that reject symlinks, hardlinks, special files, and
|
object. A failed save is reloaded before retrying. No ordering between metadata and
|
||||||
ancestor symlink traversal. Staging and visible files share one filesystem.
|
user-file durability is required. Settings and engine saves are serialized.
|
||||||
- Atomic, complete, exclusive file creation (a crash leaves absent or complete
|
Keep metadata outside the user-file namespace or in `.vault-link-sync`, which is
|
||||||
bytes, never a partial file) and atomic renames that **never replace** destinations.
|
excluded from scans. Changing a client's vault once it holds identities, offline
|
||||||
Writes apply optional cursor metadata.
|
notifications, pending requests or bootstrap state requires a separate state store.
|
||||||
- `flushPaths` flushes existing files and ancestor directories, including parents
|
|
||||||
of absent paths. Recovery uses it to complete interrupted rename durability.
|
|
||||||
- Durable recursive directory creation, and directory pruning using only `rmdir`
|
|
||||||
operations (never unlink descendant files). All affected data/directory entries
|
|
||||||
must be flushed before a mutation resolves, including intermediate ancestors.
|
|
||||||
|
|
||||||
Keep the persistence store outside the user file namespace or inside the reserved
|
Filesystem application saves metadata once when it finishes. Namespace changes detected
|
||||||
`.vault-link-sync` directory, so writing sync state cannot generate user edits.
|
before mutations invalidate the plan. Changes during application are incorporated
|
||||||
|
between files using their document identities; the next scan rereads content. Content is reread before
|
||||||
Persistence `save` atomically replaces the complete state and resolves only after
|
replacement. Unexpected eligible occupants receive visible conflict names;
|
||||||
its data and directory entry are durable. A failed save may leave either complete
|
ignored and oversized occupants stay in place. External writers can still change
|
||||||
value; the engine reloads before retrying. Settings saves are assumed to succeed
|
files between observations, so these checks do not promise transactional file
|
||||||
and do not reload after errors. Settings and engine saves are serialized.
|
contents or perfect identity reconstruction. There is no automated v2/v3 state migration.
|
||||||
Changing a client's vault once it holds identities, offline notifications,
|
|
||||||
pending requests, or bootstrap/application state requires a separate state store
|
|
||||||
and client. Lifecycle operations are serialized.
|
|
||||||
|
|
||||||
Before changing visible files, the engine persists an application journal. It
|
|
||||||
stages every affected source before installing destinations, so swaps and cycles
|
|
||||||
work. Recovery runs before scanning, including while the server is unavailable.
|
|
||||||
Prepared content is persisted before installation, preventing repeated merges
|
|
||||||
following a crash. Unexpected occupants are preserved and given conflict names.
|
|
||||||
Notifications received while staging rebind the actual source identity before
|
|
||||||
content is merged. Editor saves to a temporarily staged path continue the original
|
|
||||||
document's journal, including saves racing preparation and installation. Ignored
|
|
||||||
or oversized occupants are reserved before conflict allocation.
|
|
||||||
|
|
||||||
`.vault-link-sync/transactions` contains only active recovery artifacts. Staged
|
|
||||||
sources and prepared output are removed after every install is durably recorded;
|
|
||||||
completed journals are not archived. `.vault-link-sync/recovery` retains displaced
|
|
||||||
original bytes and their document/path metadata before staging files are deleted.
|
|
||||||
`.vault-link-sync/requests` retains rejected submission snapshots for explicit
|
|
||||||
recovery. These directories are never scanned
|
|
||||||
as user files. Pending requests and active recovery journals must not be
|
|
||||||
independently discarded.
|
|
||||||
|
|
||||||
These guarantees assume the adapters/storage honor their durability contracts.
|
|
||||||
Arbitrary external writers can change a file between observations; detected races
|
|
||||||
are preserved/reconciled, but unobserved writes and offline identity cannot be
|
|
||||||
reconstructed perfectly. There is no automated v2/v3 persisted-state migration.
|
|
||||||
|
|
|
||||||
|
|
@ -1,428 +0,0 @@
|
||||||
import assert from "node:assert/strict";
|
|
||||||
import { describe, it } from "node:test";
|
|
||||||
import {
|
|
||||||
Database,
|
|
||||||
emptyState,
|
|
||||||
type StoredDatabase
|
|
||||||
} from "../persistence/database";
|
|
||||||
import type { ServerConfig } from "../services/server-config";
|
|
||||||
import { toStoredSnapshot } from "../sync-operations/content";
|
|
||||||
import { isInternalPath } from "../utils/portable-path";
|
|
||||||
import { Logger } from "../tracing/logger";
|
|
||||||
import { FileOperations, type FileWrite } from "./file-operations";
|
|
||||||
import type {
|
|
||||||
FileSnapshot,
|
|
||||||
FileSystemOperations
|
|
||||||
} from "./filesystem-operations";
|
|
||||||
|
|
||||||
const bytes = (text: string) => new TextEncoder().encode(text);
|
|
||||||
const text = (snapshot: FileSnapshot) =>
|
|
||||||
new TextDecoder().decode(snapshot.content);
|
|
||||||
const parent = (path: string) => path.split("/").slice(0, -1).join("/");
|
|
||||||
|
|
||||||
/** Atomic mutations with an injectable interruption before or after durability. */
|
|
||||||
class MemoryFiles implements FileSystemOperations {
|
|
||||||
public readonly files = new Map<string, FileSnapshot>();
|
|
||||||
public readonly directories = new Set([""]);
|
|
||||||
public boundary: (label: string) => void = () => {};
|
|
||||||
|
|
||||||
public change(label: string, action: () => void): void {
|
|
||||||
this.boundary(`before ${label}`);
|
|
||||||
action();
|
|
||||||
this.boundary(`after ${label}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
public put(path: string, value: string): void {
|
|
||||||
for (
|
|
||||||
let directory = parent(path);
|
|
||||||
directory;
|
|
||||||
directory = parent(directory)
|
|
||||||
)
|
|
||||||
this.directories.add(directory);
|
|
||||||
this.files.set(path, { content: bytes(value) });
|
|
||||||
}
|
|
||||||
|
|
||||||
public async listFilesRecursively(root = ""): Promise<string[]> {
|
|
||||||
assert(this.directories.has(root), `Cannot list non-directory ${root}`);
|
|
||||||
return [...this.files.keys()].filter(
|
|
||||||
(path) => !root || path.startsWith(`${root}/`)
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
public async readSnapshot(path: string): Promise<FileSnapshot | undefined> {
|
|
||||||
assert(!this.directories.has(path), `Cannot read directory ${path}`);
|
|
||||||
return structuredClone(this.files.get(path));
|
|
||||||
}
|
|
||||||
|
|
||||||
public async stat(path: string) {
|
|
||||||
for (let ancestor = parent(path); ancestor; ancestor = parent(ancestor))
|
|
||||||
assert(
|
|
||||||
!this.files.has(ancestor),
|
|
||||||
`Cannot traverse file ${ancestor}`
|
|
||||||
);
|
|
||||||
const snapshot = this.files.get(path);
|
|
||||||
if (snapshot)
|
|
||||||
return { kind: "file" as const, size: snapshot.content.length };
|
|
||||||
if (this.directories.has(path))
|
|
||||||
return { kind: "directory" as const, size: 0 };
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
public async exists(path: string): Promise<boolean> {
|
|
||||||
return this.files.has(path) || this.directories.has(path);
|
|
||||||
}
|
|
||||||
|
|
||||||
public async createDirectory(path: string): Promise<void> {
|
|
||||||
this.change(`mkdir ${path}`, () => {
|
|
||||||
for (
|
|
||||||
let directory = path;
|
|
||||||
directory;
|
|
||||||
directory = parent(directory)
|
|
||||||
) {
|
|
||||||
assert(
|
|
||||||
!this.files.has(directory),
|
|
||||||
`File blocks directory ${directory}`
|
|
||||||
);
|
|
||||||
this.directories.add(directory);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public async write(path: string, snapshot: FileSnapshot): Promise<void> {
|
|
||||||
this.change(`write ${path}`, () => {
|
|
||||||
assert(
|
|
||||||
!this.files.has(path) && !this.directories.has(path),
|
|
||||||
`Destination exists: ${path}`
|
|
||||||
);
|
|
||||||
assert(
|
|
||||||
this.directories.has(parent(path)),
|
|
||||||
`Parent missing: ${path}`
|
|
||||||
);
|
|
||||||
this.files.set(path, structuredClone(snapshot));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public async rename(from: string, to: string): Promise<void> {
|
|
||||||
this.change(`rename ${from} -> ${to}`, () => {
|
|
||||||
assert(this.files.has(from), `Source missing: ${from}`);
|
|
||||||
assert(
|
|
||||||
!this.files.has(to) && !this.directories.has(to),
|
|
||||||
`Destination exists: ${to}`
|
|
||||||
);
|
|
||||||
assert(this.directories.has(parent(to)), `Parent missing: ${to}`);
|
|
||||||
this.files.set(to, this.files.get(from)!);
|
|
||||||
this.files.delete(from);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public async flushPaths(paths: readonly string[]): Promise<void> {
|
|
||||||
this.change(`flush ${paths.join(",")}`, () => {});
|
|
||||||
}
|
|
||||||
|
|
||||||
public async delete(path: string): Promise<void> {
|
|
||||||
this.change(`rmdir ${path}`, () => {
|
|
||||||
assert(!this.files.has(path));
|
|
||||||
assert(
|
|
||||||
![...this.files.keys()].some((file) =>
|
|
||||||
file.startsWith(`${path}/`)
|
|
||||||
)
|
|
||||||
);
|
|
||||||
for (const directory of this.directories)
|
|
||||||
if (directory === path || directory.startsWith(`${path}/`))
|
|
||||||
this.directories.delete(directory);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public async deleteFile(path: string): Promise<void> {
|
|
||||||
this.change(`unlink ${path}`, () => {
|
|
||||||
assert(!this.directories.has(path));
|
|
||||||
this.files.delete(path);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function setup(
|
|
||||||
contents: Record<string, string>,
|
|
||||||
local: Record<string, string>
|
|
||||||
) {
|
|
||||||
const fs = new MemoryFiles();
|
|
||||||
for (const [path, value] of Object.entries(contents)) fs.put(path, value);
|
|
||||||
let durable: StoredDatabase = emptyState("test");
|
|
||||||
durable.local = local;
|
|
||||||
for (const [id, path] of Object.entries(local))
|
|
||||||
durable.documents[id] = {
|
|
||||||
materialized: true,
|
|
||||||
observedHash: (await toStoredSnapshot(fs.files.get(path)!)).hash
|
|
||||||
};
|
|
||||||
const config = {
|
|
||||||
getConfig: async () => ({ mergeableFileExtensions: ["md"] })
|
|
||||||
} as ServerConfig;
|
|
||||||
const restart = () => {
|
|
||||||
const database = new Database(
|
|
||||||
new Logger(),
|
|
||||||
structuredClone(durable),
|
|
||||||
async (next) => {
|
|
||||||
fs.change("save", () => {
|
|
||||||
durable = structuredClone(next);
|
|
||||||
});
|
|
||||||
},
|
|
||||||
"test",
|
|
||||||
async () => structuredClone(durable)
|
|
||||||
);
|
|
||||||
return {
|
|
||||||
database,
|
|
||||||
operations: new FileOperations(fs, database, config)
|
|
||||||
};
|
|
||||||
};
|
|
||||||
return { fs, restart, ...restart() };
|
|
||||||
}
|
|
||||||
|
|
||||||
async function scenario() {
|
|
||||||
const fixture = await setup(
|
|
||||||
{
|
|
||||||
"a.md": "A",
|
|
||||||
"b.md": "B",
|
|
||||||
"c.md": "C",
|
|
||||||
"d.md": "Unmanaged",
|
|
||||||
"e.md": "E",
|
|
||||||
"folder/keep.md": "Keep",
|
|
||||||
"deleted.md": "Deleted"
|
|
||||||
},
|
|
||||||
{ a: "a.md", b: "b.md", c: "c.md", e: "e.md", deleted: "deleted.md" }
|
|
||||||
);
|
|
||||||
const next = structuredClone(fixture.database.state);
|
|
||||||
next.local = {
|
|
||||||
a: "b.md",
|
|
||||||
b: "a.md",
|
|
||||||
c: "d.md",
|
|
||||||
e: "folder",
|
|
||||||
added: "new.md"
|
|
||||||
};
|
|
||||||
next.documents.added = { materialized: false };
|
|
||||||
const writes: Record<string, FileWrite> = {
|
|
||||||
a: {
|
|
||||||
expected: await fixture.operations.snapshot("a.md"),
|
|
||||||
replacement: await toStoredSnapshot({ content: bytes("A updated") })
|
|
||||||
},
|
|
||||||
added: {
|
|
||||||
replacement: await toStoredSnapshot({
|
|
||||||
content: bytes("Downloaded")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
};
|
|
||||||
return { ...fixture, next, writes };
|
|
||||||
}
|
|
||||||
|
|
||||||
async function verifyScenario(fs: MemoryFiles, database: Database) {
|
|
||||||
assert.equal(database.state.application, undefined);
|
|
||||||
for (const [id, expected] of Object.entries({
|
|
||||||
a: "A updated",
|
|
||||||
b: "B",
|
|
||||||
c: "C",
|
|
||||||
e: "E",
|
|
||||||
added: "Downloaded"
|
|
||||||
})) {
|
|
||||||
const snapshot = fs.files.get(database.state.local[id])!;
|
|
||||||
assert.equal(text(snapshot), expected, id);
|
|
||||||
assert.equal(database.state.documents[id].materialized, true, id);
|
|
||||||
assert.equal(
|
|
||||||
database.state.documents[id].observedHash,
|
|
||||||
(await toStoredSnapshot(snapshot)).hash,
|
|
||||||
id
|
|
||||||
);
|
|
||||||
}
|
|
||||||
assert.equal(database.state.local.a, "b.md");
|
|
||||||
assert.equal(database.state.local.b, "a.md");
|
|
||||||
assert.equal(database.state.local.c, "d.md");
|
|
||||||
assert.notEqual(database.state.local.e, "folder");
|
|
||||||
assert.equal(text(fs.files.get("folder/keep.md")!), "Keep");
|
|
||||||
assert.equal(fs.files.has("deleted.md"), false);
|
|
||||||
assert.equal(database.state.local.deleted, undefined);
|
|
||||||
const visible = [...fs.files].filter(([path]) => !isInternalPath(path));
|
|
||||||
assert.deepEqual(
|
|
||||||
visible.map(([, value]) => text(value)).sort(),
|
|
||||||
["A updated", "B", "C", "Downloaded", "E", "Keep", "Unmanaged"].sort()
|
|
||||||
);
|
|
||||||
assert.equal(
|
|
||||||
[...fs.files.keys()].some((path) =>
|
|
||||||
path.startsWith(".vault-link-sync/transactions/")
|
|
||||||
),
|
|
||||||
false
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("filesystem journal recovery", () => {
|
|
||||||
it("stages swaps, preserves unexpected files/directories, and cleans recovery artifacts", async () => {
|
|
||||||
const { operations, database, fs, next, writes } = await scenario();
|
|
||||||
await operations.apply(next, writes);
|
|
||||||
await verifyScenario(fs, database);
|
|
||||||
fs.boundary = () => {
|
|
||||||
throw new Error("Completed recovery must not write");
|
|
||||||
};
|
|
||||||
await operations.recover();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("resumes after interruption at every filesystem and metadata mutation boundary", async (t) => {
|
|
||||||
const baseline = await scenario();
|
|
||||||
const boundaries: string[] = [];
|
|
||||||
baseline.fs.boundary = (label) => boundaries.push(label);
|
|
||||||
await baseline.operations.apply(baseline.next, baseline.writes);
|
|
||||||
for (const [index, label] of boundaries.entries()) {
|
|
||||||
await t.test(`${index}: ${label}`, async () => {
|
|
||||||
const fixture = await scenario();
|
|
||||||
const interrupted = new Error("Interrupted");
|
|
||||||
let count = 0;
|
|
||||||
fixture.fs.boundary = () => {
|
|
||||||
if (count++ === index) throw interrupted;
|
|
||||||
};
|
|
||||||
await assert.rejects(
|
|
||||||
fixture.operations.apply(fixture.next, fixture.writes),
|
|
||||||
(error) => error === interrupted
|
|
||||||
);
|
|
||||||
fixture.fs.boundary = () => {};
|
|
||||||
const { database, operations } = fixture.restart();
|
|
||||||
if (
|
|
||||||
!database.state.application &&
|
|
||||||
database.state.local.a === "a.md"
|
|
||||||
)
|
|
||||||
await operations.apply(
|
|
||||||
structuredClone(fixture.next),
|
|
||||||
fixture.writes
|
|
||||||
);
|
|
||||||
else await operations.recover();
|
|
||||||
await verifyScenario(fixture.fs, database);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
it("merges an edit made after planning exactly once across an interrupted install", async () => {
|
|
||||||
const base = "First paragraph.\n\nLast paragraph.\n";
|
|
||||||
const local = "Local first paragraph.\n\nLast paragraph.\n";
|
|
||||||
const remote = "First paragraph.\n\nRemote last paragraph.\n";
|
|
||||||
const fixture = await setup({ "note.md": base }, { note: "note.md" });
|
|
||||||
const expected = await fixture.operations.snapshot("note.md");
|
|
||||||
let edited = false;
|
|
||||||
const interrupted = new Error("Interrupted after install");
|
|
||||||
fixture.fs.boundary = (label) => {
|
|
||||||
if (!edited && label === "after save") {
|
|
||||||
fixture.fs.put("note.md", local);
|
|
||||||
edited = true;
|
|
||||||
}
|
|
||||||
if (
|
|
||||||
label.startsWith("after rename ") &&
|
|
||||||
label.endsWith(" -> note.md")
|
|
||||||
)
|
|
||||||
throw interrupted;
|
|
||||||
};
|
|
||||||
await assert.rejects(
|
|
||||||
fixture.operations.apply(structuredClone(fixture.database.state), {
|
|
||||||
note: {
|
|
||||||
expected,
|
|
||||||
replacement: await toStoredSnapshot({
|
|
||||||
content: bytes(remote)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}),
|
|
||||||
(error) => error === interrupted
|
|
||||||
);
|
|
||||||
fixture.fs.boundary = () => {};
|
|
||||||
const { operations, database } = fixture.restart();
|
|
||||||
await operations.recover();
|
|
||||||
assert.equal(
|
|
||||||
text(fixture.fs.files.get("note.md")!),
|
|
||||||
"Local first paragraph.\n\nRemote last paragraph.\n"
|
|
||||||
);
|
|
||||||
assert.equal(
|
|
||||||
database.state.documents.note.observedHash,
|
|
||||||
(await operations.snapshot("note.md"))!.hash
|
|
||||||
);
|
|
||||||
assert.equal(
|
|
||||||
[...fixture.fs.files.keys()].some((path) =>
|
|
||||||
path.startsWith(".vault-link-sync/transactions/")
|
|
||||||
),
|
|
||||||
false
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("preserves an external deletion between planning and staging", async () => {
|
|
||||||
const fixture = await setup(
|
|
||||||
{ "note.md": "Local" },
|
|
||||||
{ note: "note.md" }
|
|
||||||
);
|
|
||||||
const expected = await fixture.operations.snapshot("note.md");
|
|
||||||
fixture.fs.boundary = (label) => {
|
|
||||||
if (label === "after save") fixture.fs.files.delete("note.md");
|
|
||||||
};
|
|
||||||
await fixture.operations.apply(
|
|
||||||
structuredClone(fixture.database.state),
|
|
||||||
{
|
|
||||||
note: {
|
|
||||||
expected,
|
|
||||||
replacement: await toStoredSnapshot({
|
|
||||||
content: bytes("Remote")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
);
|
|
||||||
assert.equal(fixture.database.state.local.note, undefined);
|
|
||||||
assert.equal(fixture.fs.files.has("note.md"), false);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("preserves a file created after the destination check", async () => {
|
|
||||||
const fixture = await setup(
|
|
||||||
{ "source.md": "Incoming" },
|
|
||||||
{ doc: "source.md" }
|
|
||||||
);
|
|
||||||
const next = structuredClone(fixture.database.state);
|
|
||||||
next.local.doc = "target.md";
|
|
||||||
fixture.fs.boundary = (label) => {
|
|
||||||
if (
|
|
||||||
label.startsWith("before rename ") &&
|
|
||||||
label.endsWith(" -> target.md")
|
|
||||||
)
|
|
||||||
fixture.fs.put("target.md", "Concurrent");
|
|
||||||
};
|
|
||||||
await assert.rejects(
|
|
||||||
fixture.operations.apply(next),
|
|
||||||
/Destination exists/
|
|
||||||
);
|
|
||||||
fixture.fs.boundary = () => {};
|
|
||||||
const { operations, database } = fixture.restart();
|
|
||||||
await operations.recover();
|
|
||||||
assert.equal(text(fixture.fs.files.get("target.md")!), "Incoming");
|
|
||||||
const displaced = Object.entries(database.state.local).find(
|
|
||||||
([id]) => id !== "doc"
|
|
||||||
)!;
|
|
||||||
assert.equal(text(fixture.fs.files.get(displaced[1])!), "Concurrent");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("preserves an unexpected file blocking a destination's ancestor", async () => {
|
|
||||||
const fixture = await setup(
|
|
||||||
{ "source.md": "Incoming", target: "Occupant" },
|
|
||||||
{ doc: "source.md" }
|
|
||||||
);
|
|
||||||
const next = structuredClone(fixture.database.state);
|
|
||||||
next.local.doc = "target/child.md";
|
|
||||||
await fixture.operations.apply(next);
|
|
||||||
assert.equal(
|
|
||||||
text(fixture.fs.files.get("target/child.md")!),
|
|
||||||
"Incoming"
|
|
||||||
);
|
|
||||||
const displaced = Object.entries(fixture.database.state.local).find(
|
|
||||||
([id]) => id !== "doc"
|
|
||||||
)!;
|
|
||||||
assert.equal(text(fixture.fs.files.get(displaced[1])!), "Occupant");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("handles paths becoming directories and directories becoming files", async () => {
|
|
||||||
const fixture = await setup(
|
|
||||||
{ a: "A", "b/deep/child": "B" },
|
|
||||||
{ a: "a", b: "b/deep/child" }
|
|
||||||
);
|
|
||||||
const next = structuredClone(fixture.database.state);
|
|
||||||
next.local = { a: "b", b: "a/deep/child" };
|
|
||||||
await fixture.operations.apply(next);
|
|
||||||
assert.equal(text(fixture.fs.files.get("b")!), "A");
|
|
||||||
assert.equal(text(fixture.fs.files.get("a/deep/child")!), "B");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
File diff suppressed because it is too large
Load diff
|
|
@ -6,10 +6,9 @@ export interface FileSnapshot {
|
||||||
cursors?: CursorPosition[];
|
cursors?: CursorPosition[];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Required adapter contract for API v4; there is deliberately no weak fallback.
|
/** Filesystem operations act on the current user-owned namespace.
|
||||||
* Paths are vault-relative. Never follow symlinks (including ancestor symlinks).
|
* Paths are vault-relative. Never follow symlinks (including ancestor symlinks).
|
||||||
* Only independent regular files are supported; reject hardlinks/special files.
|
* Only independent regular files are supported; reject hardlinks/special files.
|
||||||
* Every mutation must flush file data and affected directories before resolving.
|
|
||||||
* A successful scan is complete: unreadable directories must reject, not vanish.
|
* A successful scan is complete: unreadable directories must reject, not vanish.
|
||||||
*/
|
*/
|
||||||
export interface FileSystemOperations {
|
export interface FileSystemOperations {
|
||||||
|
|
@ -22,24 +21,15 @@ export interface FileSystemOperations {
|
||||||
) => Promise<{ kind: "file" | "directory"; size: number } | undefined>;
|
) => Promise<{ kind: "file" | "directory"; size: number } | undefined>;
|
||||||
/** Includes directories. */
|
/** Includes directories. */
|
||||||
exists: (path: string) => Promise<boolean>;
|
exists: (path: string) => Promise<boolean>;
|
||||||
/** Recursive, idempotent, durable directory creation. */
|
/** Recursive, idempotent directory creation. */
|
||||||
createDirectory: (path: string) => Promise<void>;
|
createDirectory: (path: string) => Promise<void>;
|
||||||
/** Atomically create a complete file: never replace an existing object.
|
/** Create a file exclusively; never overwrite a raced destination.
|
||||||
* A crash leaves the path absent or complete, never a partial file.
|
* Writes apply optional editor cursor metadata. No durability guarantee. */
|
||||||
* Flush data and directory entries before resolving. */
|
|
||||||
write: (path: string, snapshot: FileSnapshot) => Promise<void>;
|
write: (path: string, snapshot: FileSnapshot) => Promise<void>;
|
||||||
/** Atomic, durable rename on the same filesystem; fail if destination exists.
|
/** Move a file without replacing an existing destination. */
|
||||||
* Flush the source's data and both affected directories before resolving. */
|
|
||||||
rename: (from: string, to: string) => Promise<void>;
|
rename: (from: string, to: string) => Promise<void>;
|
||||||
/** Flush existing files and all affected ancestor directories, even when a
|
/** Idempotently unlink one regular file. */
|
||||||
* named file is absent. Recovery uses this after a mutation became visible
|
|
||||||
* before its caller learned that it was durable. */
|
|
||||||
flushPaths: (paths: readonly string[]) => Promise<void>;
|
|
||||||
/** Idempotently unlink one regular file and flush its parent directory.
|
|
||||||
* The sync engine uses this only for its own internal recovery artifacts. */
|
|
||||||
deleteFile: (path: string) => Promise<void>;
|
deleteFile: (path: string) => Promise<void>;
|
||||||
/** Prune an implicit directory tree using rmdir only, durably. Reject
|
/** Prune directories using rmdir only. Reject files and nonempty directories. */
|
||||||
* regular files and never unlink descendant files; reject if files remain.
|
|
||||||
* Partial removal of empty child directories on failure is harmless. */
|
|
||||||
delete: (path: string) => Promise<void>;
|
delete: (path: string) => Promise<void>;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,4 @@
|
||||||
import type { CursorPosition } from "reconcile-text";
|
import type { CursorPosition } from "reconcile-text";
|
||||||
import type { Logger } from "../tracing/logger";
|
|
||||||
import type { DocumentUpdateResponse } from "../services/types/DocumentUpdateResponse";
|
import type { DocumentUpdateResponse } from "../services/types/DocumentUpdateResponse";
|
||||||
import type { DocumentVersionWithoutContent } from "../services/types/DocumentVersionWithoutContent";
|
import type { DocumentVersionWithoutContent } from "../services/types/DocumentVersionWithoutContent";
|
||||||
import type { FileManifest } from "../services/types/FileManifest";
|
import type { FileManifest } from "../services/types/FileManifest";
|
||||||
|
|
@ -7,7 +6,6 @@ import type { FileManifestUpdateResponse } from "../services/types/FileManifestU
|
||||||
import type { PushFileManifest } from "../services/types/PushFileManifest";
|
import type { PushFileManifest } from "../services/types/PushFileManifest";
|
||||||
import type { PutFileContent } from "../services/types/PutFileContent";
|
import type { PutFileContent } from "../services/types/PutFileContent";
|
||||||
import type { VaultSnapshot } from "../services/types/VaultSnapshot";
|
import type { VaultSnapshot } from "../services/types/VaultSnapshot";
|
||||||
import type { EventRecord } from "../services/types/EventRecord";
|
|
||||||
import type { FileManifestEntries } from "../types/file-manifest-entries";
|
import type { FileManifestEntries } from "../types/file-manifest-entries";
|
||||||
|
|
||||||
export type VaultUpdateId = number;
|
export type VaultUpdateId = number;
|
||||||
|
|
@ -23,7 +21,6 @@ export interface StoredSnapshot {
|
||||||
export interface DocumentMetadata {
|
export interface DocumentMetadata {
|
||||||
parentVersionId: number;
|
parentVersionId: number;
|
||||||
hash: string;
|
hash: string;
|
||||||
remoteRelativePath?: string;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface DocumentState {
|
export interface DocumentState {
|
||||||
|
|
@ -54,7 +51,7 @@ export type PendingRequest = { rejection?: string } & (
|
||||||
);
|
);
|
||||||
|
|
||||||
export interface EngineState {
|
export interface EngineState {
|
||||||
/** Durable reset intent; clear the transport checkpoint before bootstrap. */
|
/** Persisted reset intent; clear the transport checkpoint before bootstrap. */
|
||||||
historyRecovery?: boolean;
|
historyRecovery?: boolean;
|
||||||
vaultKey: string;
|
vaultKey: string;
|
||||||
initialized: boolean;
|
initialized: boolean;
|
||||||
|
|
@ -64,17 +61,10 @@ export interface EngineState {
|
||||||
lastSeenUpdateId: number;
|
lastSeenUpdateId: number;
|
||||||
remoteHeads: Record<string, DocumentVersionWithoutContent>;
|
remoteHeads: Record<string, DocumentVersionWithoutContent>;
|
||||||
bootstrap?: VaultSnapshot;
|
bootstrap?: VaultSnapshot;
|
||||||
/** Fold pages durably without applying intermediate remote snapshots. */
|
|
||||||
eventReplay?: {
|
|
||||||
after: number;
|
|
||||||
manifest: FileManifest;
|
|
||||||
heads: Record<string, DocumentVersionWithoutContent>;
|
|
||||||
receipts: EventRecord[];
|
|
||||||
};
|
|
||||||
pending?: PendingRequest;
|
pending?: PendingRequest;
|
||||||
/** A rejected retry does not prove an earlier attempt failed to commit. */
|
/** A rejected retry does not prove an earlier attempt failed to commit. */
|
||||||
unconfirmed?: PendingRequest[];
|
unconfirmed?: PendingRequest[];
|
||||||
/** Commits consumption of the durable notification queue with the identity map. */
|
/** Commits consumption of the saved notification queue with the identity map. */
|
||||||
lastAppliedLocalChangeId?: string;
|
lastAppliedLocalChangeId?: string;
|
||||||
/** Physical exclusions, including files with no sync identity. */
|
/** Physical exclusions, including files with no sync identity. */
|
||||||
protectedPaths?: string[];
|
protectedPaths?: string[];
|
||||||
|
|
@ -83,38 +73,13 @@ export interface EngineState {
|
||||||
rejectedManifest?: { entries: FileManifestEntries; message: string };
|
rejectedManifest?: { entries: FileManifestEntries; message: string };
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface FileStep {
|
export type StoredDatabase = EngineState;
|
||||||
documentId: string;
|
|
||||||
from?: string;
|
|
||||||
to?: string;
|
|
||||||
expected?: StoredSnapshot;
|
|
||||||
replacement?: StoredSnapshot;
|
|
||||||
staged: string;
|
|
||||||
output: string;
|
|
||||||
/** A permanent destination error has already selected a root fallback. */
|
|
||||||
replanned?: boolean;
|
|
||||||
/** Retained input of a later step for this identity; never projected as live. */
|
|
||||||
superseded?: boolean;
|
|
||||||
phase: "planned" | "staged" | "prepared" | "installing" | "installed";
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface ApplicationJournal {
|
|
||||||
id: string;
|
|
||||||
extensions: string[];
|
|
||||||
next: EngineState;
|
|
||||||
steps: FileStep[];
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface StoredDatabase extends EngineState {
|
|
||||||
application?: ApplicationJournal;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Read-only view used by cursor tracking and status reporting.
|
// Read-only view used by cursor tracking and status reporting.
|
||||||
export interface DocumentRecord {
|
export interface DocumentRecord {
|
||||||
documentId: string;
|
documentId: string;
|
||||||
relativePath: string;
|
relativePath: string;
|
||||||
metadata?: DocumentMetadata;
|
metadata?: DocumentMetadata;
|
||||||
isDeleted: boolean;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
export function emptyState(vaultKey: string): StoredDatabase {
|
export function emptyState(vaultKey: string): StoredDatabase {
|
||||||
|
|
@ -134,9 +99,7 @@ function canRebindEmpty(state: Partial<StoredDatabase> | undefined): boolean {
|
||||||
state !== undefined &&
|
state !== undefined &&
|
||||||
!state.initialized &&
|
!state.initialized &&
|
||||||
!state.pending &&
|
!state.pending &&
|
||||||
!state.application &&
|
|
||||||
!state.bootstrap &&
|
!state.bootstrap &&
|
||||||
!state.eventReplay &&
|
|
||||||
(state.unconfirmed?.length ?? 0) === 0 &&
|
(state.unconfirmed?.length ?? 0) === 0 &&
|
||||||
Object.keys(state.local ?? {}).length === 0
|
Object.keys(state.local ?? {}).length === 0
|
||||||
);
|
);
|
||||||
|
|
@ -146,7 +109,6 @@ export class Database {
|
||||||
public state: StoredDatabase;
|
public state: StoredDatabase;
|
||||||
private needsReload = false;
|
private needsReload = false;
|
||||||
public constructor(
|
public constructor(
|
||||||
private readonly logger: Logger,
|
|
||||||
initial: Partial<StoredDatabase> | undefined,
|
initial: Partial<StoredDatabase> | undefined,
|
||||||
private readonly saveData: (data: StoredDatabase) => Promise<void>,
|
private readonly saveData: (data: StoredDatabase) => Promise<void>,
|
||||||
vaultKey: string,
|
vaultKey: string,
|
||||||
|
|
@ -181,7 +143,7 @@ export class Database {
|
||||||
const snapshot = structuredClone(next);
|
const snapshot = structuredClone(next);
|
||||||
if (this.needsReload)
|
if (this.needsReload)
|
||||||
throw new Error(
|
throw new Error(
|
||||||
"Reload durable state before retrying an uncertain save"
|
"Reload saved state before retrying an uncertain save"
|
||||||
);
|
);
|
||||||
this.needsReload = true;
|
this.needsReload = true;
|
||||||
await this.saveData(snapshot);
|
await this.saveData(snapshot);
|
||||||
|
|
@ -204,10 +166,6 @@ export class Database {
|
||||||
return Object.keys(this.state.local).length;
|
return Object.keys(this.state.local).length;
|
||||||
}
|
}
|
||||||
|
|
||||||
public getLastSeenUpdateId(): number {
|
|
||||||
return this.state.lastSeenUpdateId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public getDocumentByDocumentId(id: string): DocumentRecord | undefined {
|
public getDocumentByDocumentId(id: string): DocumentRecord | undefined {
|
||||||
const relativePath = this.state.local[id];
|
const relativePath = this.state.local[id];
|
||||||
if (relativePath === undefined) return undefined;
|
if (relativePath === undefined) return undefined;
|
||||||
|
|
@ -215,11 +173,9 @@ export class Database {
|
||||||
return {
|
return {
|
||||||
documentId: id,
|
documentId: id,
|
||||||
relativePath,
|
relativePath,
|
||||||
isDeleted: false,
|
|
||||||
metadata: base && {
|
metadata: base && {
|
||||||
parentVersionId: base.vaultUpdateId,
|
parentVersionId: base.vaultUpdateId,
|
||||||
hash: base.hash,
|
hash: base.hash
|
||||||
remoteRelativePath: this.state.fileManifest.entries[id]
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
/** save() atomically replaces the entire value and resolves only after both the
|
/** Replace the complete metadata value atomically. Interrupted or failed saves
|
||||||
* data and its directory entry are durable across power loss. Failed saves must
|
* must leave an old/new complete value (or no value for the first save), never
|
||||||
* leave either complete value readable; callers retain their recovery journal.
|
* partially parsed state. No ordering or durability of user files is required.
|
||||||
*/
|
*/
|
||||||
export interface PersistenceProvider<T> {
|
export interface PersistenceProvider<T> {
|
||||||
load: () => Promise<T | undefined>;
|
load: () => Promise<T | undefined>;
|
||||||
|
|
|
||||||
|
|
@ -25,12 +25,18 @@ import { isInternalPath } from "./utils/portable-path";
|
||||||
import { FixedSizeDocumentCache } from "./utils/data-structures/fix-sized-cache";
|
import { FixedSizeDocumentCache } from "./utils/data-structures/fix-sized-cache";
|
||||||
import { globsToRegexes } from "./utils/globs-to-regexes";
|
import { globsToRegexes } from "./utils/globs-to-regexes";
|
||||||
import { Lock } from "./utils/data-structures/locks";
|
import { Lock } from "./utils/data-structures/locks";
|
||||||
import type { LocalChange } from "./sync-operations/local-changes";
|
import {
|
||||||
|
unappliedChanges,
|
||||||
|
type LocalChange
|
||||||
|
} from "./sync-operations/local-changes";
|
||||||
|
|
||||||
type StoredClient = Partial<{
|
type StoredClient = Partial<{
|
||||||
settings: Partial<SyncSettings>;
|
settings: Partial<SyncSettings>;
|
||||||
database: Partial<StoredDatabase>;
|
database: Partial<StoredDatabase>;
|
||||||
localChanges: LocalChange[];
|
localChanges: (
|
||||||
|
| LocalChange
|
||||||
|
| { type: "update"; path: string; changeId?: string }
|
||||||
|
)[];
|
||||||
localChangesVaultKey: string;
|
localChangesVaultKey: string;
|
||||||
historyCheckpoint: { vaultKey: string; checkpoint?: string };
|
historyCheckpoint: { vaultKey: string; checkpoint?: string };
|
||||||
}>;
|
}>;
|
||||||
|
|
@ -96,7 +102,7 @@ export class SyncClient {
|
||||||
let stored = (await persistence.load()) ?? {};
|
let stored = (await persistence.load()) ?? {};
|
||||||
let saving: Promise<void> = Promise.resolve();
|
let saving: Promise<void> = Promise.resolve();
|
||||||
let reloadBeforeSave = false;
|
let reloadBeforeSave = false;
|
||||||
// Settings and engine state share a store. Serialize complete durable
|
// Settings and engine state share a store. Serialize complete metadata
|
||||||
// replacements so one cannot overwrite a newer save from the other.
|
// replacements so one cannot overwrite a newer save from the other.
|
||||||
const save = async (update: StoredClient): Promise<void> => {
|
const save = async (update: StoredClient): Promise<void> => {
|
||||||
const operation = saving.then(async () => {
|
const operation = saving.then(async () => {
|
||||||
|
|
@ -122,7 +128,6 @@ export class SyncClient {
|
||||||
save({ settings: data })
|
save({ settings: data })
|
||||||
);
|
);
|
||||||
const database = new Database(
|
const database = new Database(
|
||||||
logger,
|
|
||||||
stored.database,
|
stored.database,
|
||||||
async (data) => save({ database: data }),
|
async (data) => save({ database: data }),
|
||||||
vaultKey(settings.getSettings()),
|
vaultKey(settings.getSettings()),
|
||||||
|
|
@ -132,8 +137,15 @@ export class SyncClient {
|
||||||
return stored.database as StoredDatabase | undefined;
|
return stored.database as StoredDatabase | undefined;
|
||||||
}
|
}
|
||||||
);
|
);
|
||||||
|
// Older stores also queued content updates. Trim the acknowledged prefix
|
||||||
|
// before discarding them, since its marker may belong to an update.
|
||||||
|
const localChanges = unappliedChanges(
|
||||||
|
database.state,
|
||||||
|
stored.localChanges ?? []
|
||||||
|
// eslint-disable-next-line no-restricted-syntax -- Migration needs a narrowed copy, not removal of a specific object.
|
||||||
|
).filter((change) => change.type !== "update");
|
||||||
if (
|
if (
|
||||||
stored.localChanges?.length &&
|
localChanges.length &&
|
||||||
stored.localChangesVaultKey !== vaultKey(settings.getSettings())
|
stored.localChangesVaultKey !== vaultKey(settings.getSettings())
|
||||||
) {
|
) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
|
|
@ -168,7 +180,6 @@ export class SyncClient {
|
||||||
);
|
);
|
||||||
const serverConfig = new ServerConfig(service);
|
const serverConfig = new ServerConfig(service);
|
||||||
const notifier = new FileChangeNotifier();
|
const notifier = new FileChangeNotifier();
|
||||||
const localChanges = stored.localChanges ?? [];
|
|
||||||
let syncer: Syncer;
|
let syncer: Syncer;
|
||||||
const files = new FileOperations(
|
const files = new FileOperations(
|
||||||
fs,
|
fs,
|
||||||
|
|
@ -283,7 +294,7 @@ export class SyncClient {
|
||||||
this.check();
|
this.check();
|
||||||
if (this.started)
|
if (this.started)
|
||||||
throw new Error("SyncClient has already been started");
|
throw new Error("SyncClient has already been started");
|
||||||
await this.syncer.recoverLocalState();
|
await this.syncer.reloadLocalState();
|
||||||
await this.database.bindVault(
|
await this.database.bindVault(
|
||||||
vaultKey(this.settings.getSettings())
|
vaultKey(this.settings.getSettings())
|
||||||
);
|
);
|
||||||
|
|
@ -310,7 +321,7 @@ export class SyncClient {
|
||||||
await stopped;
|
await stopped;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Restart transports; never discard pending requests or filesystem journals. */
|
/** Restart transports; never discard pending requests. */
|
||||||
public async reset(): Promise<void> {
|
public async reset(): Promise<void> {
|
||||||
await this.lifecycle.withLock(async () => {
|
await this.lifecycle.withLock(async () => {
|
||||||
this.check();
|
this.check();
|
||||||
|
|
@ -335,12 +346,11 @@ export class SyncClient {
|
||||||
this.database.state.initialized ||
|
this.database.state.initialized ||
|
||||||
this.database.state.bootstrap ||
|
this.database.state.bootstrap ||
|
||||||
this.database.state.pending ||
|
this.database.state.pending ||
|
||||||
this.database.state.application ||
|
|
||||||
this.syncer.hasLocalChanges ||
|
this.syncer.hasLocalChanges ||
|
||||||
Object.keys(this.database.state.local).length)
|
Object.keys(this.database.state.local).length)
|
||||||
)
|
)
|
||||||
throw new Error(
|
throw new Error(
|
||||||
"Changing vaults requires a separate state store and client; pending recovery data must not be reset"
|
"Changing vaults requires a separate state store and client; existing sync state must not be reset"
|
||||||
);
|
);
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
|
|
||||||
|
|
@ -47,8 +47,7 @@ export async function mergeContent(
|
||||||
remote.contentBase64
|
remote.contentBase64
|
||||||
].map(base64ToBytes);
|
].map(base64ToBytes);
|
||||||
|
|
||||||
// Concurrent unmergeable edits keep the server's content. The filesystem
|
// Concurrent unmergeable edits keep the server's content.
|
||||||
// journal retains displaced local bytes for recovery.
|
|
||||||
if (!isFileTypeMergable(path, extensions) || inputs.some(isBinary))
|
if (!isFileTypeMergable(path, extensions) || inputs.some(isBinary))
|
||||||
return remote;
|
return remote;
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,6 @@ it("retains future cursors and publishes them when content catches up", async ()
|
||||||
getDocumentByDocumentId: () => ({
|
getDocumentByDocumentId: () => ({
|
||||||
documentId: "document",
|
documentId: "document",
|
||||||
relativePath: "note.md",
|
relativePath: "note.md",
|
||||||
isDeleted: false,
|
|
||||||
metadata: { parentVersionId: 2, hash: contentHash }
|
metadata: { parentVersionId: 2, hash: contentHash }
|
||||||
})
|
})
|
||||||
} as unknown as Database;
|
} as unknown as Database;
|
||||||
|
|
|
||||||
|
|
@ -11,13 +11,12 @@ export type LocalChange = {
|
||||||
| { type: "move"; oldPath: string; relativePath: string }
|
| { type: "move"; oldPath: string; relativePath: string }
|
||||||
| { type: "create"; path: string; documentId?: string }
|
| { type: "create"; path: string; documentId?: string }
|
||||||
| { type: "delete"; path: string }
|
| { type: "delete"; path: string }
|
||||||
| { type: "update"; path: string }
|
|
||||||
);
|
);
|
||||||
|
|
||||||
export function unappliedChanges(
|
export function unappliedChanges<T extends { changeId?: string }>(
|
||||||
state: EngineState,
|
state: EngineState,
|
||||||
changes: readonly LocalChange[]
|
changes: readonly T[]
|
||||||
): LocalChange[] {
|
): T[] {
|
||||||
for (const change of changes) change.changeId ??= uuid();
|
for (const change of changes) change.changeId ??= uuid();
|
||||||
const applied = changes.findIndex(
|
const applied = changes.findIndex(
|
||||||
(change) => change.changeId === state.lastAppliedLocalChangeId
|
(change) => change.changeId === state.lastAppliedLocalChangeId
|
||||||
|
|
@ -63,7 +62,6 @@ export function applyLocalChange(
|
||||||
(path === from || path.startsWith(from + "/"))
|
(path === from || path.startsWith(from + "/"))
|
||||||
)
|
)
|
||||||
);
|
);
|
||||||
if (change.type === "update") return;
|
|
||||||
for (const [id, path] of Object.entries(change.identities)) {
|
for (const [id, path] of Object.entries(change.identities)) {
|
||||||
if (change.type === "delete") {
|
if (change.type === "delete") {
|
||||||
if (ignored(path)) {
|
if (ignored(path)) {
|
||||||
|
|
|
||||||
|
|
@ -17,7 +17,6 @@ for (const change of ["version", "identity", "path", "delete"] as const)
|
||||||
let current: DocumentRecord | undefined = {
|
let current: DocumentRecord | undefined = {
|
||||||
documentId: "a",
|
documentId: "a",
|
||||||
relativePath: "a.md",
|
relativePath: "a.md",
|
||||||
isDeleted: false,
|
|
||||||
metadata: { parentVersionId: 1, hash: digest }
|
metadata: { parentVersionId: 1, hash: digest }
|
||||||
};
|
};
|
||||||
const sent: { documentsWithCursors: DocumentWithCursors[] }[] = [];
|
const sent: { documentsWithCursors: DocumentWithCursors[] }[] = [];
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue