andras/vault-link
andras/vault-link
Watch
1
0
Fork
You've already forked vault-link
 0
Code Issues Pull requests 18 Projects Releases 55 Packages Wiki Activity Actions 33

no power loss

Browse source
This commit is contained in:
Andras Schmelczer 2026-09-21 21:29:56 +01:00
commit 2fcdbfd27d
17 changed files with 1374 additions and 3676 deletions
Download patch file Download diff file Expand all files Collapse all files

116
frontend/sync-client/README.md
View file

@ -4,28 +4,30 @@ The engine uses one serialized loop per vault and requires Web Crypto (SHA-256).
messages wake it; complete disk scans and the durable server event log are the
sources of truth. Periodic reconciliation is opt-in: set `syncIntervalMs` to a
positive interval to repair missed notifications. Unset or `0` disables polling.
`syncLocallyCreatedFile`, `syncLocallyUpdatedFile`, and `syncLocallyDeletedFile`
durably record logical file changes before resolving, including while offline.
Shutdown waits for those saves. Notifications describe logical external changes: adapters suppress
Create, delete and move notifications persist logical identity changes before
resolving, including while offline. Shutdown waits for those saves. Content-update
notifications only wake reconciliation; scans read the latest bytes, including after
restart. Notifications describe logical external changes: adapters suppress
self-generated create/delete/move notifications, and report atomic editor saves
as content updates. This distinguishes a deliberate delete/recreate from an edit.
Use `waitUntilFinished()` to await the active attempt; errors leave
pending work durable and are reported to the caller. Transient failures retry;
pending requests saved and are reported to the caller. Transient failures retry;
authentication and protocol errors wait for an explicit wake or settings change.
## State and policy
The persisted state contains the incorporated server file manifest, the current
local identity map, and the exact pending submission. A received server file
manifest is not a merge base until its local application is durable. Successful
manifest becomes the merge base when its application metadata is saved. Successful
submissions advance the base to the submitted content snapshot, preserving later
local edits. Unknown outcomes are retried unchanged before incorporating newer
remote state. Each contiguous event batch is folded into its final manifest and
content heads before merging. Intermediate changes that the server has already
reverted must not overwrite local edits.
remote state. HTTP is the sole source of remote events; WebSocket messages only
wake the loop. Contiguous event pages are folded in memory into their final manifest
and content heads before merging. An interrupted fetch restarts from the last
incorporated cursor; pagination progress is not persisted. Intermediate changes
that the server has already reverted must not overwrite local edits.
A permanent upload rejection is recorded locally, with content retained for
recovery. An unchanged rejected payload is skipped so other files can sync; the
A permanent upload rejection is recorded in metadata. An unchanged rejected payload is skipped so other files can sync; the
failure remains visible in history and `waitUntilFinished()`. Editing it, resetting
the client, or changing settings permits a fresh attempt. Earlier attempts that
committed despite a lost reply are incorporated through normal event replay.
@ -44,10 +46,9 @@ are implicit; empty directories are not synced.
- Content is uploaded before new membership; the client chooses and persists UUIDs.
- Text uses three-way `reconcile`. First contact at a matching path adopts the
remote ID and reconciles differing text with an empty parent.
- Concurrent binary/unmergeable edits keep the server version and retain
displaced local bytes in recovery. If only one side changed, keep that side.
- Deletion follows the file manifest decision. Displaced edited bytes are retained
locally, without automatically restoring membership.
- Concurrent binary/unmergeable edits keep the server version. If only one side
changed, keep that side. No local backup archive is created.
- Deletion follows the file manifest decision, including when the file has local edits.
- Notified moves preserve IDs. Offline inference requires a unique nonempty hash
match on both sides; uncertain cases become delete/create.
- Ignored and oversized files preserve their disk paths and remote membership.
@ -65,9 +66,8 @@ writes. WebSocket events wake HTTP replay, which validates that checkpoint. A
restored database cannot silently reuse a version number: a missing or different
event incarnation returns a history mismatch before executing the request.
On mismatch the client first recovers its filesystem journal and scans local
work. It durably records a reset before clearing the old checkpoint, archives
pending content requests, and clears old receipts, cursors, bases and diff-cache
On mismatch the client scans local work and records a reset before clearing the
old checkpoint. It clears old requests, receipts, cursors, bases and diff-cache
entries. Clean files reconcile against the restored snapshot. Unsent edits with
an old base become separate local documents; path conflicts preserve them under
conflict names instead of applying a diff to an unrelated restored version.
@ -79,59 +79,37 @@ Their failures are reported, and remaining observers continue receiving updates.
## Required adapter contract
`FileSystemOperations` and `PersistenceProvider` define the adapter operations
needed by the recovery model. The Obsidian and CLI adapters have intentionally
not been updated in this rewrite.
`FileSystemOperations` and `PersistenceProvider` define the adapter operations.
The Obsidian and CLI adapters still need to adopt the current core interface.
The sync client does not coordinate separate processes accessing the same vault.
The sync client does not coordinate separate processes accessing the same local
vault.
The filesystem adapter must provide complete scans, coherent snapshots, recursive
directory creation, exclusive file creation, no-replace renames, and deletion of
individual regular files. Directory pruning uses only `rmdir`, never recursive
file removal. Reject links, special files and paths escaping the vault. An unreadable
file or directory must fail the scan rather than look like a deletion. Optional
editor cursor metadata is applied with file content.
The filesystem adapter must provide:
User-file writes have no power-loss durability requirement. There is no filesystem
journal, hidden staging area, replay, backup archive, or flush protocol. Swaps and
cycles temporarily move occupied files to visible conflict names; a fresh scan
can discover them if application stops partway through. Partial writes and edits
made before, during or after an interruption are treated as current disk content.
The next sync reconciles that content; it does not finish a saved filesystem plan.
Power loss may lose bytes or rename identity and may cause a merge to run again.
- Complete scans and coherent content snapshots. Missing and unreadable must
be distinguished; reject read races rather than return a partial snapshot.
- Vault-relative operations that reject symlinks, hardlinks, special files, and
ancestor symlink traversal. Staging and visible files share one filesystem.
- Atomic, complete, exclusive file creation (a crash leaves absent or complete
bytes, never a partial file) and atomic renames that **never replace** destinations.
Writes apply optional cursor metadata.
- `flushPaths` flushes existing files and ancestor directories, including parents
of absent paths. Recovery uses it to complete interrupted rename durability.
- Durable recursive directory creation, and directory pruning using only `rmdir`
operations (never unlink descendant files). All affected data/directory entries
must be flushed before a mutation resolves, including intermediate ancestors.
Metadata is the one atomic boundary: persistence `save` replaces a complete value
and an interrupted/failed save leaves an old or new complete value, never a torn
object. A failed save is reloaded before retrying. No ordering between metadata and
user-file durability is required. Settings and engine saves are serialized.
Keep metadata outside the user-file namespace or in `.vault-link-sync`, which is
excluded from scans. Changing a client's vault once it holds identities, offline
notifications, pending requests or bootstrap state requires a separate state store.
Keep the persistence store outside the user file namespace or inside the reserved
`.vault-link-sync` directory, so writing sync state cannot generate user edits.
Persistence `save` atomically replaces the complete state and resolves only after
its data and directory entry are durable. A failed save may leave either complete
value; the engine reloads before retrying. Settings saves are assumed to succeed
and do not reload after errors. Settings and engine saves are serialized.
Changing a client's vault once it holds identities, offline notifications,
pending requests, or bootstrap/application state requires a separate state store
and client. Lifecycle operations are serialized.
Before changing visible files, the engine persists an application journal. It
stages every affected source before installing destinations, so swaps and cycles
work. Recovery runs before scanning, including while the server is unavailable.
Prepared content is persisted before installation, preventing repeated merges
following a crash. Unexpected occupants are preserved and given conflict names.
Notifications received while staging rebind the actual source identity before
content is merged. Editor saves to a temporarily staged path continue the original
document's journal, including saves racing preparation and installation. Ignored
or oversized occupants are reserved before conflict allocation.
`.vault-link-sync/transactions` contains only active recovery artifacts. Staged
sources and prepared output are removed after every install is durably recorded;
completed journals are not archived. `.vault-link-sync/recovery` retains displaced
original bytes and their document/path metadata before staging files are deleted.
`.vault-link-sync/requests` retains rejected submission snapshots for explicit
recovery. These directories are never scanned
as user files. Pending requests and active recovery journals must not be
independently discarded.
These guarantees assume the adapters/storage honor their durability contracts.
Arbitrary external writers can change a file between observations; detected races
are preserved/reconciled, but unobserved writes and offline identity cannot be
reconstructed perfectly. There is no automated v2/v3 persisted-state migration.
Filesystem application saves metadata once when it finishes. Namespace changes detected
before mutations invalidate the plan. Changes during application are incorporated
between files using their document identities; the next scan rereads content. Content is reread before
replacement. Unexpected eligible occupants receive visible conflict names;
ignored and oversized occupants stay in place. External writers can still change
files between observations, so these checks do not promise transactional file
contents or perfect identity reconstruction. There is no automated v2/v3 state migration.

428
frontend/sync-client/src/file-operations/file-operations.recovery.test.ts
View file

@ -1,428 +0,0 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import {
Database,
emptyState,
type StoredDatabase
} from "../persistence/database";
import type { ServerConfig } from "../services/server-config";
import { toStoredSnapshot } from "../sync-operations/content";
import { isInternalPath } from "../utils/portable-path";
import { Logger } from "../tracing/logger";
import { FileOperations, type FileWrite } from "./file-operations";
import type {
FileSnapshot,
FileSystemOperations
} from "./filesystem-operations";
const bytes = (text: string) => new TextEncoder().encode(text);
const text = (snapshot: FileSnapshot) =>
new TextDecoder().decode(snapshot.content);
const parent = (path: string) => path.split("/").slice(0, -1).join("/");
/** Atomic mutations with an injectable interruption before or after durability. */
class MemoryFiles implements FileSystemOperations {
public readonly files = new Map<string, FileSnapshot>();
public readonly directories = new Set([""]);
public boundary: (label: string) => void = () => {};
public change(label: string, action: () => void): void {
this.boundary(`before ${label}`);
action();
this.boundary(`after ${label}`);
}
public put(path: string, value: string): void {
for (
let directory = parent(path);
directory;
directory = parent(directory)
)
this.directories.add(directory);
this.files.set(path, { content: bytes(value) });
}
public async listFilesRecursively(root = ""): Promise<string[]> {
assert(this.directories.has(root), `Cannot list non-directory ${root}`);
return [...this.files.keys()].filter(
(path) => !root || path.startsWith(`${root}/`)
);
}
public async readSnapshot(path: string): Promise<FileSnapshot | undefined> {
assert(!this.directories.has(path), `Cannot read directory ${path}`);
return structuredClone(this.files.get(path));
}
public async stat(path: string) {
for (let ancestor = parent(path); ancestor; ancestor = parent(ancestor))
assert(
!this.files.has(ancestor),
`Cannot traverse file ${ancestor}`
);
const snapshot = this.files.get(path);
if (snapshot)
return { kind: "file" as const, size: snapshot.content.length };
if (this.directories.has(path))
return { kind: "directory" as const, size: 0 };
return undefined;
}
public async exists(path: string): Promise<boolean> {
return this.files.has(path) || this.directories.has(path);
}
public async createDirectory(path: string): Promise<void> {
this.change(`mkdir ${path}`, () => {
for (
let directory = path;
directory;
directory = parent(directory)
) {
assert(
!this.files.has(directory),
`File blocks directory ${directory}`
);
this.directories.add(directory);
}
});
}
public async write(path: string, snapshot: FileSnapshot): Promise<void> {
this.change(`write ${path}`, () => {
assert(
!this.files.has(path) && !this.directories.has(path),
`Destination exists: ${path}`
);
assert(
this.directories.has(parent(path)),
`Parent missing: ${path}`
);
this.files.set(path, structuredClone(snapshot));
});
}
public async rename(from: string, to: string): Promise<void> {
this.change(`rename ${from} -> ${to}`, () => {
assert(this.files.has(from), `Source missing: ${from}`);
assert(
!this.files.has(to) && !this.directories.has(to),
`Destination exists: ${to}`
);
assert(this.directories.has(parent(to)), `Parent missing: ${to}`);
this.files.set(to, this.files.get(from)!);
this.files.delete(from);
});
}
public async flushPaths(paths: readonly string[]): Promise<void> {
this.change(`flush ${paths.join(",")}`, () => {});
}
public async delete(path: string): Promise<void> {
this.change(`rmdir ${path}`, () => {
assert(!this.files.has(path));
assert(
![...this.files.keys()].some((file) =>
file.startsWith(`${path}/`)
)
);
for (const directory of this.directories)
if (directory === path || directory.startsWith(`${path}/`))
this.directories.delete(directory);
});
}
public async deleteFile(path: string): Promise<void> {
this.change(`unlink ${path}`, () => {
assert(!this.directories.has(path));
this.files.delete(path);
});
}
}
async function setup(
contents: Record<string, string>,
local: Record<string, string>
) {
const fs = new MemoryFiles();
for (const [path, value] of Object.entries(contents)) fs.put(path, value);
let durable: StoredDatabase = emptyState("test");
durable.local = local;
for (const [id, path] of Object.entries(local))
durable.documents[id] = {
materialized: true,
observedHash: (await toStoredSnapshot(fs.files.get(path)!)).hash
};
const config = {
getConfig: async () => ({ mergeableFileExtensions: ["md"] })
} as ServerConfig;
const restart = () => {
const database = new Database(
new Logger(),
structuredClone(durable),
async (next) => {
fs.change("save", () => {
durable = structuredClone(next);
});
},
"test",
async () => structuredClone(durable)
);
return {
database,
operations: new FileOperations(fs, database, config)
};
};
return { fs, restart, ...restart() };
}
async function scenario() {
const fixture = await setup(
{
"a.md": "A",
"b.md": "B",
"c.md": "C",
"d.md": "Unmanaged",
"e.md": "E",
"folder/keep.md": "Keep",
"deleted.md": "Deleted"
},
{ a: "a.md", b: "b.md", c: "c.md", e: "e.md", deleted: "deleted.md" }
);
const next = structuredClone(fixture.database.state);
next.local = {
a: "b.md",
b: "a.md",
c: "d.md",
e: "folder",
added: "new.md"
};
next.documents.added = { materialized: false };
const writes: Record<string, FileWrite> = {
a: {
expected: await fixture.operations.snapshot("a.md"),
replacement: await toStoredSnapshot({ content: bytes("A updated") })
},
added: {
replacement: await toStoredSnapshot({
content: bytes("Downloaded")
})
}
};
return { ...fixture, next, writes };
}
async function verifyScenario(fs: MemoryFiles, database: Database) {
assert.equal(database.state.application, undefined);
for (const [id, expected] of Object.entries({
a: "A updated",
b: "B",
c: "C",
e: "E",
added: "Downloaded"
})) {
const snapshot = fs.files.get(database.state.local[id])!;
assert.equal(text(snapshot), expected, id);
assert.equal(database.state.documents[id].materialized, true, id);
assert.equal(
database.state.documents[id].observedHash,
(await toStoredSnapshot(snapshot)).hash,
id
);
}
assert.equal(database.state.local.a, "b.md");
assert.equal(database.state.local.b, "a.md");
assert.equal(database.state.local.c, "d.md");
assert.notEqual(database.state.local.e, "folder");
assert.equal(text(fs.files.get("folder/keep.md")!), "Keep");
assert.equal(fs.files.has("deleted.md"), false);
assert.equal(database.state.local.deleted, undefined);
const visible = [...fs.files].filter(([path]) => !isInternalPath(path));
assert.deepEqual(
visible.map(([, value]) => text(value)).sort(),
["A updated", "B", "C", "Downloaded", "E", "Keep", "Unmanaged"].sort()
);
assert.equal(
[...fs.files.keys()].some((path) =>
path.startsWith(".vault-link-sync/transactions/")
),
false
);
}
describe("filesystem journal recovery", () => {
it("stages swaps, preserves unexpected files/directories, and cleans recovery artifacts", async () => {
const { operations, database, fs, next, writes } = await scenario();
await operations.apply(next, writes);
await verifyScenario(fs, database);
fs.boundary = () => {
throw new Error("Completed recovery must not write");
};
await operations.recover();
});
it("resumes after interruption at every filesystem and metadata mutation boundary", async (t) => {
const baseline = await scenario();
const boundaries: string[] = [];
baseline.fs.boundary = (label) => boundaries.push(label);
await baseline.operations.apply(baseline.next, baseline.writes);
for (const [index, label] of boundaries.entries()) {
await t.test(`${index}: ${label}`, async () => {
const fixture = await scenario();
const interrupted = new Error("Interrupted");
let count = 0;
fixture.fs.boundary = () => {
if (count++ === index) throw interrupted;
};
await assert.rejects(
fixture.operations.apply(fixture.next, fixture.writes),
(error) => error === interrupted
);
fixture.fs.boundary = () => {};
const { database, operations } = fixture.restart();
if (
!database.state.application &&
database.state.local.a === "a.md"
)
await operations.apply(
structuredClone(fixture.next),
fixture.writes
);
else await operations.recover();
await verifyScenario(fixture.fs, database);
});
}
});
it("merges an edit made after planning exactly once across an interrupted install", async () => {
const base = "First paragraph.\n\nLast paragraph.\n";
const local = "Local first paragraph.\n\nLast paragraph.\n";
const remote = "First paragraph.\n\nRemote last paragraph.\n";
const fixture = await setup({ "note.md": base }, { note: "note.md" });
const expected = await fixture.operations.snapshot("note.md");
let edited = false;
const interrupted = new Error("Interrupted after install");
fixture.fs.boundary = (label) => {
if (!edited && label === "after save") {
fixture.fs.put("note.md", local);
edited = true;
}
if (
label.startsWith("after rename ") &&
label.endsWith(" -> note.md")
)
throw interrupted;
};
await assert.rejects(
fixture.operations.apply(structuredClone(fixture.database.state), {
note: {
expected,
replacement: await toStoredSnapshot({
content: bytes(remote)
})
}
}),
(error) => error === interrupted
);
fixture.fs.boundary = () => {};
const { operations, database } = fixture.restart();
await operations.recover();
assert.equal(
text(fixture.fs.files.get("note.md")!),
"Local first paragraph.\n\nRemote last paragraph.\n"
);
assert.equal(
database.state.documents.note.observedHash,
(await operations.snapshot("note.md"))!.hash
);
assert.equal(
[...fixture.fs.files.keys()].some((path) =>
path.startsWith(".vault-link-sync/transactions/")
),
false
);
});
it("preserves an external deletion between planning and staging", async () => {
const fixture = await setup(
{ "note.md": "Local" },
{ note: "note.md" }
);
const expected = await fixture.operations.snapshot("note.md");
fixture.fs.boundary = (label) => {
if (label === "after save") fixture.fs.files.delete("note.md");
};
await fixture.operations.apply(
structuredClone(fixture.database.state),
{
note: {
expected,
replacement: await toStoredSnapshot({
content: bytes("Remote")
})
}
}
);
assert.equal(fixture.database.state.local.note, undefined);
assert.equal(fixture.fs.files.has("note.md"), false);
});
it("preserves a file created after the destination check", async () => {
const fixture = await setup(
{ "source.md": "Incoming" },
{ doc: "source.md" }
);
const next = structuredClone(fixture.database.state);
next.local.doc = "target.md";
fixture.fs.boundary = (label) => {
if (
label.startsWith("before rename ") &&
label.endsWith(" -> target.md")
)
fixture.fs.put("target.md", "Concurrent");
};
await assert.rejects(
fixture.operations.apply(next),
/Destination exists/
);
fixture.fs.boundary = () => {};
const { operations, database } = fixture.restart();
await operations.recover();
assert.equal(text(fixture.fs.files.get("target.md")!), "Incoming");
const displaced = Object.entries(database.state.local).find(
([id]) => id !== "doc"
)!;
assert.equal(text(fixture.fs.files.get(displaced[1])!), "Concurrent");
});
it("preserves an unexpected file blocking a destination's ancestor", async () => {
const fixture = await setup(
{ "source.md": "Incoming", target: "Occupant" },
{ doc: "source.md" }
);
const next = structuredClone(fixture.database.state);
next.local.doc = "target/child.md";
await fixture.operations.apply(next);
assert.equal(
text(fixture.fs.files.get("target/child.md")!),
"Incoming"
);
const displaced = Object.entries(fixture.database.state.local).find(
([id]) => id !== "doc"
)!;
assert.equal(text(fixture.fs.files.get(displaced[1])!), "Occupant");
});
it("handles paths becoming directories and directories becoming files", async () => {
const fixture = await setup(
{ a: "A", "b/deep/child": "B" },
{ a: "a", b: "b/deep/child" }
);
const next = structuredClone(fixture.database.state);
next.local = { a: "b", b: "a/deep/child" };
await fixture.operations.apply(next);
assert.equal(text(fixture.fs.files.get("b")!), "A");
assert.equal(text(fixture.fs.files.get("a/deep/child")!), "B");
});
});

1123
frontend/sync-client/src/file-operations/file-operations.ts
View file

File diff suppressed because it is too large Load diff

24
frontend/sync-client/src/file-operations/filesystem-operations.ts
View file

@ -6,10 +6,9 @@ export interface FileSnapshot {
cursors?: CursorPosition[];
}
/** Required adapter contract for API v4; there is deliberately no weak fallback.
/** Filesystem operations act on the current user-owned namespace.
* Paths are vault-relative. Never follow symlinks (including ancestor symlinks).
* Only independent regular files are supported; reject hardlinks/special files.
* Every mutation must flush file data and affected directories before resolving.
* A successful scan is complete: unreadable directories must reject, not vanish.
*/
export interface FileSystemOperations {
@ -22,24 +21,15 @@ export interface FileSystemOperations {
) => Promise<{ kind: "file" | "directory"; size: number } | undefined>;
/** Includes directories. */
exists: (path: string) => Promise<boolean>;
/** Recursive, idempotent, durable directory creation. */
/** Recursive, idempotent directory creation. */
createDirectory: (path: string) => Promise<void>;
/** Atomically create a complete file: never replace an existing object.
* A crash leaves the path absent or complete, never a partial file.
* Flush data and directory entries before resolving. */
/** Create a file exclusively; never overwrite a raced destination.
* Writes apply optional editor cursor metadata. No durability guarantee. */
write: (path: string, snapshot: FileSnapshot) => Promise<void>;
/** Atomic, durable rename on the same filesystem; fail if destination exists.
* Flush the source's data and both affected directories before resolving. */
/** Move a file without replacing an existing destination. */
rename: (from: string, to: string) => Promise<void>;
/** Flush existing files and all affected ancestor directories, even when a
* named file is absent. Recovery uses this after a mutation became visible
* before its caller learned that it was durable. */
flushPaths: (paths: readonly string[]) => Promise<void>;
/** Idempotently unlink one regular file and flush its parent directory.
* The sync engine uses this only for its own internal recovery artifacts. */
/** Idempotently unlink one regular file. */
deleteFile: (path: string) => Promise<void>;
/** Prune an implicit directory tree using rmdir only, durably. Reject
* regular files and never unlink descendant files; reject if files remain.
* Partial removal of empty child directories on failure is harmless. */
/** Prune directories using rmdir only. Reject files and nonempty directories. */
delete: (path: string) => Promise<void>;
}

54
frontend/sync-client/src/persistence/database.ts
View file

@ -1,5 +1,4 @@
import type { CursorPosition } from "reconcile-text";
import type { Logger } from "../tracing/logger";
import type { DocumentUpdateResponse } from "../services/types/DocumentUpdateResponse";
import type { DocumentVersionWithoutContent } from "../services/types/DocumentVersionWithoutContent";
import type { FileManifest } from "../services/types/FileManifest";
@ -7,7 +6,6 @@ import type { FileManifestUpdateResponse } from "../services/types/FileManifestU
import type { PushFileManifest } from "../services/types/PushFileManifest";
import type { PutFileContent } from "../services/types/PutFileContent";
import type { VaultSnapshot } from "../services/types/VaultSnapshot";
import type { EventRecord } from "../services/types/EventRecord";
import type { FileManifestEntries } from "../types/file-manifest-entries";
export type VaultUpdateId = number;
@ -23,7 +21,6 @@ export interface StoredSnapshot {
export interface DocumentMetadata {
parentVersionId: number;
hash: string;
remoteRelativePath?: string;
}
export interface DocumentState {
@ -54,7 +51,7 @@ export type PendingRequest = { rejection?: string } & (
);
export interface EngineState {
/** Durable reset intent; clear the transport checkpoint before bootstrap. */
/** Persisted reset intent; clear the transport checkpoint before bootstrap. */
historyRecovery?: boolean;
vaultKey: string;
initialized: boolean;
@ -64,17 +61,10 @@ export interface EngineState {
lastSeenUpdateId: number;
remoteHeads: Record<string, DocumentVersionWithoutContent>;
bootstrap?: VaultSnapshot;
/** Fold pages durably without applying intermediate remote snapshots. */
eventReplay?: {
after: number;
manifest: FileManifest;
heads: Record<string, DocumentVersionWithoutContent>;
receipts: EventRecord[];
};
pending?: PendingRequest;
/** A rejected retry does not prove an earlier attempt failed to commit. */
unconfirmed?: PendingRequest[];
/** Commits consumption of the durable notification queue with the identity map. */
/** Commits consumption of the saved notification queue with the identity map. */
lastAppliedLocalChangeId?: string;
/** Physical exclusions, including files with no sync identity. */
protectedPaths?: string[];
@ -83,38 +73,13 @@ export interface EngineState {
rejectedManifest?: { entries: FileManifestEntries; message: string };
}
export interface FileStep {
documentId: string;
from?: string;
to?: string;
expected?: StoredSnapshot;
replacement?: StoredSnapshot;
staged: string;
output: string;
/** A permanent destination error has already selected a root fallback. */
replanned?: boolean;
/** Retained input of a later step for this identity; never projected as live. */
superseded?: boolean;
phase: "planned" | "staged" | "prepared" | "installing" | "installed";
}
export interface ApplicationJournal {
id: string;
extensions: string[];
next: EngineState;
steps: FileStep[];
}
export interface StoredDatabase extends EngineState {
application?: ApplicationJournal;
}
export type StoredDatabase = EngineState;
// Read-only view used by cursor tracking and status reporting.
export interface DocumentRecord {
documentId: string;
relativePath: string;
metadata?: DocumentMetadata;
isDeleted: boolean;
}
export function emptyState(vaultKey: string): StoredDatabase {
@ -134,9 +99,7 @@ function canRebindEmpty(state: Partial<StoredDatabase> | undefined): boolean {
state !== undefined &&
!state.initialized &&
!state.pending &&
!state.application &&
!state.bootstrap &&
!state.eventReplay &&
(state.unconfirmed?.length ?? 0) === 0 &&
Object.keys(state.local ?? {}).length === 0
);
@ -146,7 +109,6 @@ export class Database {
public state: StoredDatabase;
private needsReload = false;
public constructor(
private readonly logger: Logger,
initial: Partial<StoredDatabase> | undefined,
private readonly saveData: (data: StoredDatabase) => Promise<void>,
vaultKey: string,
@ -181,7 +143,7 @@ export class Database {
const snapshot = structuredClone(next);
if (this.needsReload)
throw new Error(
"Reload durable state before retrying an uncertain save"
"Reload saved state before retrying an uncertain save"
);
this.needsReload = true;
await this.saveData(snapshot);
@ -204,10 +166,6 @@ export class Database {
return Object.keys(this.state.local).length;
}
public getLastSeenUpdateId(): number {
return this.state.lastSeenUpdateId;
}
public getDocumentByDocumentId(id: string): DocumentRecord | undefined {
const relativePath = this.state.local[id];
if (relativePath === undefined) return undefined;
@ -215,11 +173,9 @@ export class Database {
return {
documentId: id,
relativePath,
isDeleted: false,
metadata: base && {
parentVersionId: base.vaultUpdateId,
hash: base.hash,
remoteRelativePath: this.state.fileManifest.entries[id]
hash: base.hash
}
};
}

6
frontend/sync-client/src/persistence/persistence.ts
View file

@ -1,6 +1,6 @@
/** save() atomically replaces the entire value and resolves only after both the
* data and its directory entry are durable across power loss. Failed saves must
* leave either complete value readable; callers retain their recovery journal.
/** Replace the complete metadata value atomically. Interrupted or failed saves
* must leave an old/new complete value (or no value for the first save), never
* partially parsed state. No ordering or durability of user files is required.
*/
export interface PersistenceProvider<T> {
load: () => Promise<T | undefined>;

30
frontend/sync-client/src/sync-client.ts
View file

@ -25,12 +25,18 @@ import { isInternalPath } from "./utils/portable-path";
import { FixedSizeDocumentCache } from "./utils/data-structures/fix-sized-cache";
import { globsToRegexes } from "./utils/globs-to-regexes";
import { Lock } from "./utils/data-structures/locks";
import type { LocalChange } from "./sync-operations/local-changes";
import {
unappliedChanges,
type LocalChange
} from "./sync-operations/local-changes";
type StoredClient = Partial<{
settings: Partial<SyncSettings>;
database: Partial<StoredDatabase>;
localChanges: LocalChange[];
localChanges: (
| LocalChange
| { type: "update"; path: string; changeId?: string }
)[];
localChangesVaultKey: string;
historyCheckpoint: { vaultKey: string; checkpoint?: string };
}>;
@ -96,7 +102,7 @@ export class SyncClient {
let stored = (await persistence.load()) ?? {};
let saving: Promise<void> = Promise.resolve();
let reloadBeforeSave = false;
// Settings and engine state share a store. Serialize complete durable
// Settings and engine state share a store. Serialize complete metadata
// replacements so one cannot overwrite a newer save from the other.
const save = async (update: StoredClient): Promise<void> => {
const operation = saving.then(async () => {
@ -122,7 +128,6 @@ export class SyncClient {
save({ settings: data })
);
const database = new Database(
logger,
stored.database,
async (data) => save({ database: data }),
vaultKey(settings.getSettings()),
@ -132,8 +137,15 @@ export class SyncClient {
return stored.database as StoredDatabase | undefined;
}
);
// Older stores also queued content updates. Trim the acknowledged prefix
// before discarding them, since its marker may belong to an update.
const localChanges = unappliedChanges(
database.state,
stored.localChanges ?? []
// eslint-disable-next-line no-restricted-syntax -- Migration needs a narrowed copy, not removal of a specific object.
).filter((change) => change.type !== "update");
if (
stored.localChanges?.length &&
localChanges.length &&
stored.localChangesVaultKey !== vaultKey(settings.getSettings())
) {
throw new Error(
@ -168,7 +180,6 @@ export class SyncClient {
);
const serverConfig = new ServerConfig(service);
const notifier = new FileChangeNotifier();
const localChanges = stored.localChanges ?? [];
let syncer: Syncer;
const files = new FileOperations(
fs,
@ -283,7 +294,7 @@ export class SyncClient {
this.check();
if (this.started)
throw new Error("SyncClient has already been started");
await this.syncer.recoverLocalState();
await this.syncer.reloadLocalState();
await this.database.bindVault(
vaultKey(this.settings.getSettings())
);
@ -310,7 +321,7 @@ export class SyncClient {
await stopped;
}
/** Restart transports; never discard pending requests or filesystem journals. */
/** Restart transports; never discard pending requests. */
public async reset(): Promise<void> {
await this.lifecycle.withLock(async () => {
this.check();
@ -335,12 +346,11 @@ export class SyncClient {
this.database.state.initialized ||
this.database.state.bootstrap ||
this.database.state.pending ||
this.database.state.application ||
this.syncer.hasLocalChanges ||
Object.keys(this.database.state.local).length)
)
throw new Error(
"Changing vaults requires a separate state store and client; pending recovery data must not be reset"
"Changing vaults requires a separate state store and client; existing sync state must not be reset"
);
try {

3
frontend/sync-client/src/sync-operations/content.ts
View file

@ -47,8 +47,7 @@ export async function mergeContent(
remote.contentBase64
].map(base64ToBytes);
// Concurrent unmergeable edits keep the server's content. The filesystem
// journal retains displaced local bytes for recovery.
// Concurrent unmergeable edits keep the server's content.
if (!isFileTypeMergable(path, extensions) || inputs.some(isBinary))
return remote;

1
frontend/sync-client/src/sync-operations/cursor-tracker.test.ts
View file

@ -18,7 +18,6 @@ it("retains future cursors and publishes them when content catches up", async ()
getDocumentByDocumentId: () => ({
documentId: "document",
relativePath: "note.md",
isDeleted: false,
metadata: { parentVersionId: 2, hash: contentHash }
})
} as unknown as Database;

8
frontend/sync-client/src/sync-operations/local-changes.ts
View file

@ -11,13 +11,12 @@ export type LocalChange = {
| { type: "move"; oldPath: string; relativePath: string }
| { type: "create"; path: string; documentId?: string }
| { type: "delete"; path: string }
| { type: "update"; path: string }
);
export function unappliedChanges(
export function unappliedChanges<T extends { changeId?: string }>(
state: EngineState,
changes: readonly LocalChange[]
): LocalChange[] {
changes: readonly T[]
): T[] {
for (const change of changes) change.changeId ??= uuid();
const applied = changes.findIndex(
(change) => change.changeId === state.lastAppliedLocalChangeId
@ -63,7 +62,6 @@ export function applyLocalChange(
(path === from || path.startsWith(from + "/"))
)
);
if (change.type === "update") return;
for (const [id, path] of Object.entries(change.identities)) {
if (change.type === "delete") {
if (ignored(path)) {

1
frontend/sync-client/src/sync-operations/outgoing-cursor.test.ts
View file

@ -17,7 +17,6 @@ for (const change of ["version", "identity", "path", "delete"] as const)
let current: DocumentRecord | undefined = {
documentId: "a",
relativePath: "a.md",
isDeleted: false,
metadata: { parentVersionId: 1, hash: digest }
};
const sent: { documentsWithCursors: DocumentWithCursors[] }[] = [];

2
frontend/sync-client/src/sync-operations/scan.ts
View file

@ -38,7 +38,7 @@ export async function scanLocalFiles(
}: ScanOptions,
initial?: VaultSnapshot
): Promise<Set<string>> {
// Leave the prefix queued until its resulting state is durably committed.
// Leave the prefix queued until its resulting state is saved.
// A persisted change ID makes replay safe after an uncertain save.
const changes = [...queuedChanges];
const unsyncablePaths = new Set<string>();

30
frontend/sync-client/src/sync-operations/syncer.test.ts
View file

@ -153,8 +153,7 @@ describe("Syncer per-file state and event delivery", () => {
path === "note.md"
? {
documentId: "document",
relativePath: path,
isDeleted: false
relativePath: path
}
: undefined
} as unknown as Database;
@ -166,10 +165,8 @@ describe("Syncer per-file state and event delivery", () => {
(batch: EventBatch) => Promise<void>
>()
} as unknown as WebSocketManager;
let httpEventsCalls = 0;
const service = {
events: async () => {
httpEventsCalls++;
return { headEventId: 0, events: [] };
}
} as unknown as SyncService;
@ -191,8 +188,7 @@ describe("Syncer per-file state and event delivery", () => {
return {
database,
websocket,
syncer,
httpEventsCalls: () => httpEventsCalls
syncer
};
}
@ -211,28 +207,6 @@ describe("Syncer per-file state and event delivery", () => {
assert.strictEqual(syncer.isDocumentUpToDate("note.md"), false);
});
it("consumes a complete WebSocket event batch without fetching it again", async () => {
const { websocket, syncer, httpEventsCalls } = fixture();
const batch: EventBatch = {
headEventId: 1,
events: [
{
eventId: 1,
requestId: "00000000-0000-0000-0000-000000000001",
type: "fileManifest",
fileManifest: { fileManifestId: 1, entries: {} }
}
]
};
await websocket.onRemoteVaultUpdateReceived.triggerAsync(batch);
const selected = await (
syncer as unknown as { nextEventBatch: () => Promise<EventBatch> }
).nextEventBatch();
assert.deepEqual(selected, batch);
assert.strictEqual(httpEventsCalls(), 0);
});
it("projects content responses to metadata before persistence", () => {
const { syncer } = fixture();
const projected = (

248
frontend/sync-client/src/sync-operations/syncer.ts
View file

@ -20,6 +20,7 @@ import type { WebSocketManager } from "../services/websocket-manager";
import type { DocumentVersion } from "../services/types/DocumentVersion";
import type { DocumentVersionWithoutContent } from "../services/types/DocumentVersionWithoutContent";
import type { EventBatch } from "../services/types/EventBatch";
import type { EventRecord } from "../services/types/EventRecord";
import type { FileManifest } from "../services/types/FileManifest";
import type { VaultSnapshot } from "../services/types/VaultSnapshot";
import type { PushContent } from "../services/types/PushContent";
@ -66,7 +67,6 @@ export class Syncer {
private notificationSaveFailed = false;
private readonly pendingPaths = new Set<string>();
private unsyncablePaths = new Set<string>();
private eventBatches: EventBatch[] = [];
private hasScanned = false;
public constructor(
@ -99,10 +99,7 @@ export class Syncer {
// including when the optional polling interval is disabled.
this.wake();
});
websocket.onRemoteVaultUpdateReceived.add(async (batch) => {
this.eventBatches.push(batch);
// Notifications are only hints; HTTP replay recovers discarded pages.
if (this.eventBatches.length > 4) this.eventBatches.shift();
websocket.onRemoteVaultUpdateReceived.add(async () => {
this.wake();
});
}
@ -120,7 +117,6 @@ export class Syncer {
!!this.running ||
this.dirty ||
!!this.database.state.pending ||
!!this.database.state.application ||
!!this.failure ||
!!this.rejectionMessage
);
@ -169,11 +165,7 @@ export class Syncer {
return false;
}
return !this.database.state.application?.steps.some(
(step) =>
step.documentId === record.documentId &&
step.phase !== "installed"
);
return true;
}
public start(): void {
@ -273,7 +265,7 @@ export class Syncer {
oldPath?: string;
relativePath: string;
}): Promise<void> {
// Content snapshots are re-read during journal staging. Only namespace
// Content snapshots are re-read before writing. Only namespace
// notifications invalidate an identity plan; keystrokes must not starve
// a complete scan of unrelated documents.
if (oldPath !== undefined && oldPath !== "") this.localGeneration++;
@ -289,23 +281,14 @@ export class Syncer {
oldPath,
relativePath
});
} else if (!isInternalPath(relativePath)) {
await this.recordLocalChange({
type: "update",
path: relativePath
});
}
this.wake();
}
/** Local recovery must run even when network sync is disabled. Caller stops
* the engine first and holds the client's lifecycle lock. */
public async recoverLocalState(): Promise<void> {
/** Reload an uncertain metadata save before starting another attempt. */
public async reloadLocalState(): Promise<void> {
await this.flushLocalChanges();
await this.database.recoverPersistence();
const { application } = this.database.state;
await this.files.recover();
if (application) this.recordRecoveredApplication(application.steps);
}
private async recordLocalChange(change: LocalChange): Promise<void> {
@ -339,7 +322,7 @@ export class Syncer {
this.onRemainingOperationsCountChanged.trigger(1);
this.failure = undefined;
try {
await this.recoverLocalState();
await this.reloadLocalState();
if (
this.database.state.historyRecovery ||
(this.database.state.initialized &&
@ -360,15 +343,13 @@ export class Syncer {
await this.finishPending();
}
await this.scan();
const batch = await this.nextEventBatch();
const batch = await this.service.events(
this.database.state.lastSeenUpdateId
);
await this.incorporateEventBatch(batch);
if (this.database.state.eventReplay) {
this.dirty = true;
continue;
}
await this.scan();
// Previously excluded content becomes eligible after a setting
// change. Its durable obligation survives advancing the cursor.
// change. Its saved obligation survives advancing the cursor.
if (Object.keys(this.database.state.excluded ?? {}).length)
await this.incorporateFileManifest(
this.database.state.fileManifest
@ -392,14 +373,6 @@ export class Syncer {
await this.finishPending();
this.dirty = true;
}
if (
this.eventBatches.some(
(batch) =>
batch.headEventId >
this.database.state.lastSeenUpdateId
)
)
this.dirty = true;
} while (this.dirty && !this.stopped);
if (!this.dirty && !this.database.state.pending)
this.pendingPaths.clear();
@ -428,24 +401,11 @@ export class Syncer {
private async recoverServerHistory(): Promise<void> {
this.onServerHistoryChanged.trigger();
this.contentCache.reset();
this.eventBatches = [];
this.hasScanned = false;
if (!this.database.state.historyRecovery) {
await this.scan();
const guard = this.localGuard();
const previous = this.database.state;
for (const pending of [
previous.pending,
...(previous.unconfirmed ?? [])
]) {
if (pending?.type === "content")
await this.files.retainSnapshot(
pending.request.requestId,
pending.documentId,
previous.local[pending.documentId],
pending.snapshot
);
}
const next = emptyState(previous.vaultKey);
next.historyRecovery = true;
next.lastAppliedLocalChangeId = previous.lastAppliedLocalChangeId;
@ -561,10 +521,7 @@ export class Syncer {
}
private next(): EngineState {
const { application: _, ...next } = structuredClone(
this.database.state
);
return next;
return structuredClone(this.database.state);
}
private async bootstrap(): Promise<void> {
@ -604,56 +561,53 @@ export class Syncer {
}
private async incorporateEventBatch(batch: EventBatch): Promise<void> {
const replay = this.database.state.eventReplay;
const after = replay?.after ?? this.database.state.lastSeenUpdateId;
let cursor = after;
let manifest = replay?.manifest ?? this.database.state.fileManifest;
const heads = { ...(replay?.heads ?? this.database.state.remoteHeads) };
const receipts = [...(replay?.receipts ?? [])];
for (const event of batch.events) {
if (event.eventId <= cursor) continue;
if (event.eventId !== cursor + 1)
throw new PermanentSyncError(
"Non-contiguous event history; refusing to skip changes"
);
cursor = event.eventId;
if (
this.database.state.unconfirmed?.some(
(pending) => pending.request.requestId === event.requestId
) === true
)
receipts.push(event);
if (event.type === "fileManifest") {
if (event.fileManifest.fileManifestId > manifest.fileManifestId)
manifest = event.fileManifest;
} else if (
!heads[event.document.documentId] ||
heads[event.document.documentId].vaultUpdateId <
event.document.vaultUpdateId
) {
heads[event.document.documentId] = this.documentHead(
event.document
);
}
}
const end = batch.endEventId ?? batch.headEventId;
if (
end !== cursor ||
end > batch.headEventId ||
(end === after && end < batch.headEventId)
)
throw new PermanentSyncError("Incomplete event batch");
if (end < batch.headEventId) {
await this.database.commit({
...this.next(),
eventReplay: {
after: end,
manifest,
heads,
receipts
let cursor = this.database.state.lastSeenUpdateId;
let manifest = this.database.state.fileManifest;
const heads = { ...this.database.state.remoteHeads };
const receipts: EventRecord[] = [];
// Fold every page before touching local files. If fetching is interrupted,
// discard this work and retry from the last committed cursor.
for (;;) {
const after = cursor;
for (const event of batch.events) {
if (event.eventId <= cursor) continue;
if (event.eventId !== cursor + 1)
throw new PermanentSyncError(
"Non-contiguous event history; refusing to skip changes"
);
cursor = event.eventId;
if (
this.database.state.unconfirmed?.some(
(pending) =>
pending.request.requestId === event.requestId
) === true
)
receipts.push(event);
if (event.type === "fileManifest") {
if (
event.fileManifest.fileManifestId >
manifest.fileManifestId
)
manifest = event.fileManifest;
} else if (
!heads[event.document.documentId] ||
heads[event.document.documentId].vaultUpdateId <
event.document.vaultUpdateId
) {
heads[event.document.documentId] = this.documentHead(
event.document
);
}
});
return;
}
const end = batch.endEventId ?? batch.headEventId;
if (
end !== cursor ||
end > batch.headEventId ||
(end === after && end < batch.headEventId)
)
throw new PermanentSyncError("Incomplete event batch");
if (end === batch.headEventId) break;
batch = await this.service.events(cursor);
}
if (cursor === this.database.state.lastSeenUpdateId) return;
// Recover our submitted bases before folding later remote edits. A
@ -707,24 +661,6 @@ export class Syncer {
);
}
private async nextEventBatch(): Promise<EventBatch> {
const after =
this.database.state.eventReplay?.after ??
this.database.state.lastSeenUpdateId;
if (this.service.verifiesHistory) {
this.eventBatches = [];
return this.service.events(after);
}
this.eventBatches = this.eventBatches.filter(
(batch) => (batch.endEventId ?? batch.headEventId) > after
);
const queued = this.eventBatches.findIndex((batch) =>
batch.events.some((event) => event.eventId === after + 1)
);
if (queued !== -1) return this.eventBatches.splice(queued, 1)[0];
return this.service.events(after);
}
private documentHead(
version: DocumentVersionWithoutContent
): DocumentVersionWithoutContent {
@ -738,42 +674,6 @@ export class Syncer {
};
}
private recordRecoveredApplication(
steps: readonly {
documentId: string;
from?: string;
to?: string;
replacement?: unknown;
}[]
): void {
for (const step of steps) {
const to = this.database.state.local[step.documentId];
if (!step.from && !to) continue;
this.history.addHistoryEntry({
status: SyncStatus.SUCCESS,
message: "Filesystem application recovered",
details:
step.from && to && step.from !== to
? {
type: SyncType.MOVE,
relativePath: to,
movedFrom: step.from
}
: !step.from && to
? { type: SyncType.CREATE, relativePath: to }
: step.from && !to
? {
type: SyncType.DELETE,
relativePath: step.from
}
: {
type: SyncType.UPDATE,
relativePath: to ?? step.from!
}
});
}
}
private recordManifestApplication(
before: Readonly<Record<string, string>>,
writes: Readonly<Record<string, FileWrite>>
@ -948,7 +848,6 @@ export class Syncer {
const next = this.next();
if (cursor !== undefined) {
next.lastSeenUpdateId = cursor;
delete next.eventReplay;
}
if (clearPending) {
delete next.pending;
@ -1037,7 +936,7 @@ export class Syncer {
}
private async preparePush(): Promise<boolean> {
if (this.stopped || this.database.state.eventReplay) {
if (this.stopped) {
return false;
}
@ -1214,18 +1113,15 @@ export class Syncer {
await this.database.recoverPersistence();
const next = structuredClone(this.database.state);
let changed = false;
for (const state of [next, next.application?.next]) {
if (!state) continue;
if (state.rejectedManifest) {
delete state.rejectedManifest;
if (next.rejectedManifest) {
delete next.rejectedManifest;
changed = true;
}
for (const doc of Object.values(next.documents)) {
if (doc.rejected) {
delete doc.rejected;
changed = true;
}
for (const doc of Object.values(state.documents)) {
if (doc.rejected) {
delete doc.rejected;
changed = true;
}
}
}
if (changed) await this.database.commit(next);
}
@ -1260,12 +1156,6 @@ export class Syncer {
if (pending.rejection && !pending.response) {
const next = this.next();
if (pending.type === "content") {
await this.files.retainSnapshot(
pending.request.requestId,
pending.documentId,
next.local[pending.documentId],
pending.snapshot
);
next.documents[pending.documentId].rejected = {
hash: pending.snapshot.hash,
message: pending.rejection
@ -1346,12 +1236,6 @@ export class Syncer {
} else {
const response = pending.response!;
if (response.type === "StaleBase") {
await this.files.retainSnapshot(
pending.request.requestId,
pending.documentId,
this.database.state.local[pending.documentId],
pending.snapshot
);
await this.incorporateContent(
response,
undefined,

13
frontend/sync-client/src/utils/assert-set-contains-exactly.ts
View file

@ -1,13 +0,0 @@
import assert from "node:assert";
export function assertSetContainsExactly<T>(set: Set<T>, ...values: T[]): void {
assert.ok(
set.size === values.length &&
Array.from(set).every((value) => values.includes(value)),
`Expected set to contain only ${values.map((v) => '"' + v + '"').join(", ")}, but it contained ${Array.from(
set
)
.map((v) => '"' + v + '"')
.join(", ")}`
);
}

76
frontend/sync-client/src/utils/data-structures/min-covered.test.ts
View file

@ -1,76 +0,0 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import { CoveredValues } from "./min-covered";
describe("CoveredValues", () => {
it("should initialize with the given min value", () => {
const covered = new CoveredValues(5);
assert.strictEqual(covered.min, 5);
});
it("should add values greater than min", () => {
const covered = new CoveredValues(0);
covered.add(3);
assert.strictEqual(covered.min, 0);
covered.add(1);
assert.strictEqual(covered.min, 1);
covered.add(4);
assert.strictEqual(covered.min, 1);
covered.add(2);
assert.strictEqual(covered.min, 4);
});
it("should ignore duplicate values", () => {
const covered = new CoveredValues(0);
covered.add(3);
covered.add(3);
covered.add(3);
assert.strictEqual(covered.min, 0);
covered.add(1);
covered.add(2);
assert.strictEqual(covered.min, 3);
});
it("should handle multiple consecutive values", () => {
const covered = new CoveredValues(132);
for (let i = 250; i > 132; i--) {
assert.strictEqual(covered.min, 132);
covered.add(i);
}
assert.strictEqual(covered.min, 250);
});
it("should handle adding values lower than current min", () => {
const covered = new CoveredValues(5);
covered.add(3);
assert.strictEqual(covered.min, 5);
covered.add(6);
assert.strictEqual(covered.min, 6);
});
it("should auto-advance when setting min value", () => {
const covered = new CoveredValues(5);
covered.add(7);
covered.add(8);
covered.add(9);
assert.strictEqual(covered.min, 5);
// Setting min to 6 should auto-advance through 7, 8, 9
covered.min = 6;
assert.strictEqual(covered.min, 9);
covered.add(10);
assert.strictEqual(covered.min, 10);
});
it("should handle setting min value with no consecutive values", () => {
const covered = new CoveredValues(5);
covered.add(10);
covered.add(15);
assert.strictEqual(covered.min, 5);
// Setting min to 8 should not auto-advance (no consecutive values)
covered.min = 8;
assert.strictEqual(covered.min, 8);
// Add 9 to trigger auto-advance to 10
covered.add(9);
assert.strictEqual(covered.min, 10);
});
});

61
frontend/sync-client/src/utils/data-structures/min-covered.ts
View file

@ -1,61 +0,0 @@
/**
* A class that tracks the minimum covered value in a sequence of numbers.
* It keeps track of a minimum value based on the seen values.
*
* It expects integers slightly out of order and makes sure that the value of `min` is
* always the minimum of the seen values. This is done with bounded memory usage.
*
* @example
* ```typescript
* const covered = new CoveredValues(0);
* covered.add(2); // seenValues = [2], min = 0
* covered.add(1); // seenValues = [], min = 2
* covered.min; // returns 2
* ```
*/
export class CoveredValues {
private seenValues: number[] = [];
public constructor(private minValue: number) {}
public get min(): number {
return this.minValue;
}
public set min(value: number) {
this.minValue = Math.max(value, this.minValue);
this.seenValues = this.seenValues.filter((v) => v > this.minValue);
this.advanceMinWhilePossible();
}
public add(value: number | undefined): void {
if (value === undefined || value < this.minValue) {
return;
}
let i = 0;
while (i < this.seenValues.length && this.seenValues[i] < value) {
i++;
}
if (i === this.seenValues.length) {
this.seenValues.push(value);
} else if (this.seenValues[i] === value) {
return;
} else {
this.seenValues.splice(i, 0, value);
}
this.advanceMinWhilePossible();
}
private advanceMinWhilePossible(): void {
while (
this.seenValues.length > 0 &&
this.seenValues[0] === this.minValue + 1
) {
this.seenValues.shift();
this.minValue++;
}
}
}