power loss
This commit is contained in:
parent
0f6b618c0b
commit
d56084e4bc
27 changed files with 18667 additions and 3902 deletions
163
frontend/test-support/README.md
Normal file
163
frontend/test-support/README.md
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
# Shared sync harness support
|
||||
|
||||
This is test-only code, shared by `deterministic-tests` and `test-client` rather
|
||||
than exported as debugging machinery by the production sync client.
|
||||
|
||||
The journal suites also cover repeated saves replacing earlier unsynced text,
|
||||
tracked file/directory moves into ignored paths, and saves crossing the size
|
||||
limit. They check exact bytes and identities through recovery, including real
|
||||
server reads. Client and Rust unit suites bound deep-path validation resources.
|
||||
|
||||
| Layer | What it establishes |
|
||||
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `harness.test.ts` | Divergence/self-comparison detection, binary-safe comparisons, marker loss/duplication, cleanup failure propagation, deep persistence ownership, exclusive operations, exercised faults, seed reproducibility |
|
||||
| `oracle-regressions.test.ts` | Negative checks for regenerated retries, wrong fault kinds, duplicate clients, corrupt hashes, BOMs, path byte limits, idempotent unlink, directory durability, and complete identity/content expectations |
|
||||
| `recovery.test.ts` | Real engine journal recovery at every enumerated apply and replay mutation/save boundary; two interruptions followed by another power loss; identity, retained bytes and merge-policy regressions |
|
||||
| `engine-regressions.test.ts` | Recovery archives, offline notifications, rejection/retry, exclusions, reverted remote updates, lifecycle races and editor changes during staging/install |
|
||||
| `event-delivery.test.ts` | Real sync loop with checkpoint-controlled WS bursts before an iteration and during event application; no polling or extra wakeups to hide queued events |
|
||||
| `polling.test.ts` | Real sync loop with fake clocks: missed local/remote notifications, disabled controls, repeated polls, shutdown and network retries independent of polling |
|
||||
| `protocol.test.ts` | Terminated worker runtimes after accepted creates, existing-document Snapshot/Diff updates, and manifest requests, exact durable retries after newer remote heads and real server SIGKILL; concurrent CAS and ordered WS catchup |
|
||||
| `real-disk.test.ts` | Host filesystem exclusive create/rename, symlink/hardlink rejection and a real engine rename-cycle application |
|
||||
| Scripted suite | Existing multi-client regression scenarios plus v4 binary, identity, path-conflict, retry, and server-restart cases |
|
||||
| Seeded workloads | Concrete replay traces, offline and rapid online moves/edits, concurrent creates, exact deletion/overwrite exemptions, nested Unicode paths, delayed notifications and before/after network failures |
|
||||
|
||||
`scripts/e2e-runner.test.mjs` also guards clean-install workspace/lockfile
|
||||
consistency, CI dependency installation and artifact upload wiring, subprocess
|
||||
exit codes, log failures, spawn failures, and timeouts.
|
||||
|
||||
`MemoryDisk` distinguishes visible namespace/data from durable entries. Each
|
||||
mutation has `before:`, `visible:`, and `durable:` boundaries. `crash(false)`
|
||||
models process death with dirty kernel state retained; `crash(true)` discards
|
||||
unflushed state. Operations that return successfully have flushed their affected
|
||||
entries. Idempotent unlink flushes absent entries too, and durable directory
|
||||
removal makes all former descendants unreachable. This is a deliberately small atomic-operation model, not an emulation
|
||||
of APFS/ext4 writeback. A crashed session is fenced against subsequent writes.
|
||||
|
||||
`MemoryPersistence` atomically saves complete, cloned objects. Failure before a
|
||||
save leaves the old object; failure after durable replacement leaves the new one.
|
||||
The journal sweep recreates the production database/file-operations objects after
|
||||
each fault. From each interrupted state it traces recovery and injects a second
|
||||
process death or power loss at every replay mutation/save boundary. Final recovery
|
||||
is replayed again and followed by another power loss. Cases cover swaps with new
|
||||
output, cross-directory moves with a surviving sibling, concurrent text merges,
|
||||
binary replacement, deletion and recovery archive cleanup. Exact visible bytes,
|
||||
UUID/path mappings, retained bytes and absence of transaction files are checked. An intent is resubmitted only if nothing was durably saved;
|
||||
resubmitting a partially recovered operation would hide journal defects.
|
||||
|
||||
`crash-worker.ts` runs a complete SyncClient in a separate worker. The parent
|
||||
terminates it at an explicit accepted-response checkpoint without `destroy()`;
|
||||
no old timers, websocket handlers, or promises survive. Update cases bootstrap an existing UUID and assert the non-null parent and exact
|
||||
Snapshot/Diff request type before termination. Its saved memory-disk image and persisted database seed a fresh runtime. The separate real-disk fixture
|
||||
uses a test-only native exclusive-rename primitive. It is **not** a
|
||||
production adapter or an adversarial `openat` sandbox: concurrent hostile changes
|
||||
to ancestor directories and actual machine power cuts are outside its coverage.
|
||||
|
||||
## Regression coverage
|
||||
|
||||
The previously exposed WS queue, notified-rename UUID, uncertain-save replay,
|
||||
and cross-directory durability failures are fixed and remain asserted by the
|
||||
default harness. Scripted, seeded, event-delivery and engine regression clients
|
||||
explicitly disable polling so it cannot hide a missing wakeup. Crash workers also
|
||||
assert that this setting survives restart. Polling is enabled only in separate
|
||||
tests that intentionally omit notifications and advance a fake clock. Identity
|
||||
assertions distinguish documents even when bytes
|
||||
are equal. The engine regressions additionally exercise retained recovery bytes,
|
||||
durable offline notifications, permanently rejected requests, ignored namespace
|
||||
conflicts, reverted remote updates, and concurrent lifecycle changes.
|
||||
|
||||
Tests assert intended invariants, including bytes and UUID mappings, rather than
|
||||
accepting the observed output as an oracle. The memory fault model and real-disk
|
||||
checks complement one another; neither proves behavior under every real power cut.
|
||||
|
||||
## September 2026 audit regressions
|
||||
|
||||
`scripts/e2e.sh` now runs the Rust and client unit suites as well as the harness,
|
||||
real-server protocol checks, scripted scenarios, and seeded workloads. A failing
|
||||
unit suite makes the complete run fail. The audit tests are separate, individually
|
||||
quoted arguments in the default scripts, with a runner regression guarding that wiring.
|
||||
|
||||
The audit cases were first run with assertions for the required behavior and
|
||||
observed failing before the corresponding fixes. Coverage is organized by the
|
||||
original finding numbers:
|
||||
|
||||
| Findings | Permanent coverage and resulting guarantee |
|
||||
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 1 | Engine and real-server lost-reply/rejected-retry tests preserve later content and rename undos; interrupted receipt saves and paged replay are tested after process replacement. |
|
||||
| 2, 6 | Ignored/oversized unmanaged occupants keep their paths, including arrivals after scanning; submitted manifests reserve excluded server paths and remain valid. |
|
||||
| 3, 12 | Oversized remote heads defer uploads; oversized local files are rejected before snapshot reads; scans retain hashes rather than encoded vault contents; content notifications cannot starve namespace scans. |
|
||||
| 4 | A subprocess deadline detects synchronous conflict-allocation hangs caused by long extensions. |
|
||||
| 5 | Persistent destination errors select a durable root fallback. The recovery sweep keeps that error active through every first and second crash boundary, checking exact bytes, identities and retained originals. |
|
||||
| 7–9 | Bootstrap notifications adopt remote identities; settings updates resume enabled engines, disabling sync persists across reset, and initial vault bindings survive restart; disabled startup recovers local journals. |
|
||||
| 10–11 | Fake-clock silent-socket recovery and real-server connection admission, slot release and handshake expiry. |
|
||||
| 13 | Rust tests bound event count and aggregate bytes per page; client tests fold remote reversions before application and preserve the replay position across restart. |
|
||||
| 14–15 | Metadata-only manifest reference validation and active-pool cleanup tests ensure unrelated payload decoding and checked-out transactions cannot block these operations. |
|
||||
| 16 | Dirty cursor updates invalidate precise positions; disconnect/reset clear presented cursors. |
|
||||
| 17 | Read-only configuration preservation and concurrent initialization verify that existing credentials are never rewritten or replaced. |
|
||||
|
||||
The follow-up audit added these independently reproduced failures. Each behavior
|
||||
was asserted before its fix; the default test commands include every regression.
|
||||
|
||||
| Additional failure | Regression coverage |
|
||||
| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Delayed content receipts crash after local deletion | `sync-audit.test.ts` verifies receipt consumption, preserved deletion, and cursor advancement without a local path. |
|
||||
| Cursor reads restore presence after disconnect/reset | Cursor unit tests hold an active read and queue another update, retire the connection, then release both. |
|
||||
| Failed WebSocket upgrades strand clients with polling disabled | `polling.test.ts` requires exact remote content and rename catchup over HTTP after a disconnect signal. |
|
||||
| WebSocket construction errors escape startup/retry callbacks | Transport unit tests fail the first or second construction and require a successful subsequent connection. |
|
||||
| Opening a locked vault blocks unrelated vaults | Rust tests hold a real SQLite writer lock during lazy opening; another vault must remain readable. Concurrent openings must share the configured pool, including after cancellation. |
|
||||
| Failure clearing rejected requests leaves an enabled client stopped | Settings tests fail the second save, before and after commit, and require later notifications to wake synchronization. |
|
||||
| Caller mutations bypass the checked settings snapshot | A settings test changes the caller's vault name and exclusion array while transports pause; persisted settings must retain the original input. |
|
||||
| Cursor freshness uses metadata captured before an asynchronous read | A cursor test advances the document while the read is suspended and requires the old cursor to be marked outdated. |
|
||||
| Throwing/silent socket closes leave live handlers after shutdown | Transport tests require disconnected status and no message delivery after a failed or timed-out close. |
|
||||
| Handshake send errors escape the socket-open callback | A transport test drops the send during opening and requires safe retirement and retry. |
|
||||
| Old connection cleanup deletes its replacement's cursors | A real-server overlapping reconnect test preserves the replacement; Rust tests also reject delayed updates from the old session. |
|
||||
|
||||
These cases exercise transitions the earlier tests did not combine: a deletion
|
||||
between submission and acknowledgement, disconnects during reads, failures after
|
||||
the settings save, and two connections sharing a device identity. Happy-path
|
||||
reconnect and single-save fault tests do not establish those guarantees.
|
||||
|
||||
Replay pages contain at most 64 events and approximately 1 MiB of aggregate event
|
||||
JSON. One atomic manifest can exceed that budget; it is sent alone so replay still
|
||||
advances. `endEventId` identifies the page boundary while `headEventId` identifies
|
||||
the vault head. Deploy the updated client and server together: older clients
|
||||
require each batch to reach the vault head. Local partial replay folds retain only
|
||||
latest document heads, the latest manifest, and matching outstanding receipts;
|
||||
local files are reconciled only when catchup reaches the head.
|
||||
|
||||
Permanent destination failures use `Recovered <document ID><extension>` in the
|
||||
vault root, resolving any occupied name. The journal retains the original bytes
|
||||
when content is replaced. Permission or storage failures that also prevent that
|
||||
fallback remain explicit recovery failures with the journal intact.
|
||||
|
||||
A rejected retry retains an unconfirmed receipt until its request appears in the
|
||||
event log. Absence from one catchup is not proof that an earlier in-flight attempt
|
||||
cannot still commit. Receipt/artifact pruning remains an explicit operational concern; these tests do not claim
|
||||
formal completeness or simulate arbitrary physical-media corruption.
|
||||
|
||||
Reproduce the complete validation run from the repository root:
|
||||
|
||||
```sh
|
||||
E2E_ITERATIONS=100 E2E_SEED=8201 E2E_WORKERS=2 scripts/e2e.sh
|
||||
```
|
||||
|
||||
## Sync regression coverage added after the September audit
|
||||
|
||||
The normal test commands include these regressions:
|
||||
|
||||
| Failure | Maintained coverage |
|
||||
| ------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ignored occupant renamed into scope; incoming download discarded by unrelated notifications | `journal-protocol.test.ts` checks visible files, original IDs and actual server content. |
|
||||
| Source path reused or moved during staging; editor save becomes a duplicate | `journal-interleavings.test.ts` injects moves, delete/recreate, move/recreate and content saves at every enumerated read, mutation and state-save boundary. Assertions specify bytes and document identities independently. |
|
||||
| Journal/editor interaction followed by a crash | `journal-protocol.test.ts` crosses three editor operations with failures before/after state persistence, process replacement and simulated power loss, then checks another reset. |
|
||||
| Restored server history falsely appears current | `server-restore.test.ts` restores actual SQLite backups and restarts Rust, covering equal/lower counters, dirty files, lost acknowledgements, interrupted recovery saves and old client state without checkpoints. |
|
||||
| Oversized remote fallback and stale responses | `content-limits.test.ts` measures payload downloads, checks metadata-only responses and verifies highly fragmented local edits fall back to snapshots. |
|
||||
| Adversarial diff work | Rust endpoint tests reject excessive operations without appending events and exercise Unicode reconstruction. |
|
||||
| Observer exception kills sync | `observer.test.ts` runs throwing/rejecting subscribers in a child process and verifies a second sync; listener unit tests verify continued delivery. |
|
||||
| Outgoing cursor version race | `outgoing-cursor.test.ts` changes version, identity, path and existence during a read, deliberately including identical bytes. |
|
||||
| Unpersisted credentials | Rust config tests reject both missing users and missing user lists while preserving existing file bytes; initialization tests verify stable saved defaults. |
|
||||
| Extensionless filenames enable merging | Extension unit tests include root/nested `md` and `txt` and dots in ancestor directories. |
|
||||
| Corrupt vault blocks startup | Rust database regression opens a healthy vault alongside an invalid SQLite file and verifies the invalid file remains untouched. |
|
||||
|
||||
The filesystem sweep reports the number of boundaries and injected interleavings.
|
||||
A passing sweep covers this adapter model and these operations; arbitrary physical
|
||||
media corruption and every possible editor/OS behavior remain outside its scope.
|
||||
157
frontend/test-support/canonical.ts
Normal file
157
frontend/test-support/canonical.ts
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import type { StoredDatabase } from "sync-client";
|
||||
import { assertQuiescent, assertSameFiles } from "./oracles";
|
||||
|
||||
export interface CanonicalSnapshot {
|
||||
headEventId: number;
|
||||
fileManifest: { fileManifestId: number; entries: Record<string, string> };
|
||||
documents: { documentId: string; vaultUpdateId: number }[];
|
||||
}
|
||||
export interface InspectableClient {
|
||||
files(): Map<string, Uint8Array>;
|
||||
database(): Partial<StoredDatabase> | undefined;
|
||||
}
|
||||
|
||||
export async function getJson<T>(url: string, token: string): Promise<T> {
|
||||
const response = await fetch(url, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Device-Id": "harness-observer"
|
||||
},
|
||||
signal: AbortSignal.timeout(5_000)
|
||||
});
|
||||
assert(response.ok, `Observer GET ${url}: HTTP ${response.status}`);
|
||||
return (await response.json()) as T;
|
||||
}
|
||||
|
||||
/** Equality alone can pass when every replica loses the same bytes. Check the
|
||||
* canonical UUIDs, content heads, receipts, journals and event cursor as well.
|
||||
* Scenario-specific content/identity oracles still run separately.
|
||||
*/
|
||||
export async function assertCanonical(
|
||||
clients: InspectableClient[],
|
||||
vaultUrl: string,
|
||||
token: string
|
||||
): Promise<CanonicalSnapshot> {
|
||||
assert.equal(
|
||||
new Set(clients).size,
|
||||
clients.length,
|
||||
"Cannot compare a client to itself"
|
||||
);
|
||||
const files = clients.map((client) => client.files());
|
||||
const states = clients.map((client) => client.database());
|
||||
assertSameFiles(files);
|
||||
const snapshot = await getJson<CanonicalSnapshot>(
|
||||
`${vaultUrl}/vault-snapshot`,
|
||||
token
|
||||
);
|
||||
const entries = snapshot.fileManifest.entries;
|
||||
for (const id of Object.keys(entries))
|
||||
assert(
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(
|
||||
id
|
||||
),
|
||||
`Non-UUID document identity: ${id}`
|
||||
);
|
||||
const heads = Object.fromEntries(
|
||||
snapshot.documents.map((doc) => [doc.documentId, doc])
|
||||
);
|
||||
assert.equal(
|
||||
Object.keys(heads).length,
|
||||
snapshot.documents.length,
|
||||
"Duplicate document heads"
|
||||
);
|
||||
assert.deepEqual(
|
||||
Object.keys(heads).sort(),
|
||||
Object.keys(entries).sort(),
|
||||
"Snapshot heads differ from membership"
|
||||
);
|
||||
for (const [i, state] of states.entries()) {
|
||||
assert(state?.initialized, `Client ${i} never initialized`);
|
||||
assertQuiescent(state, entries, snapshot.headEventId);
|
||||
assert.equal(
|
||||
state.fileManifest?.fileManifestId,
|
||||
snapshot.fileManifest.fileManifestId,
|
||||
"Wrong manifest CAS parent"
|
||||
);
|
||||
assert.deepEqual(
|
||||
[...files[i].keys()].sort(),
|
||||
Object.values(entries).sort(),
|
||||
"Untracked or missing disk files"
|
||||
);
|
||||
for (const id of Object.keys(entries)) {
|
||||
assert(
|
||||
state.documents?.[id]?.materialized,
|
||||
`Unmaterialized document: ${id}`
|
||||
);
|
||||
assert.equal(
|
||||
state.documents?.[id]?.base?.vaultUpdateId,
|
||||
heads[id]?.vaultUpdateId,
|
||||
`Wrong content CAS parent: ${id}`
|
||||
);
|
||||
}
|
||||
}
|
||||
for (const [id, path] of Object.entries(entries)) {
|
||||
const doc = await getJson<{ contentBase64: string }>(
|
||||
`${vaultUrl}/documents/${id}`,
|
||||
token
|
||||
);
|
||||
const serverBytes = new Uint8Array(
|
||||
Buffer.from(doc.contentBase64, "base64")
|
||||
);
|
||||
assert.deepEqual(
|
||||
files[0].get(path),
|
||||
serverBytes,
|
||||
`Server bytes differ for ${id} at ${path}`
|
||||
);
|
||||
const hash = createHash("sha256").update(serverBytes).digest("hex");
|
||||
for (const [i, state] of states.entries()) {
|
||||
assert.equal(
|
||||
state!.documents?.[id]?.base?.hash,
|
||||
hash,
|
||||
`Wrong base hash: client ${i}, ${id}`
|
||||
);
|
||||
assert.equal(
|
||||
state!.documents?.[id]?.observedHash,
|
||||
hash,
|
||||
`Wrong observed hash: client ${i}, ${id}`
|
||||
);
|
||||
}
|
||||
}
|
||||
const events: { eventId: number; requestId: string }[] = [];
|
||||
let after = 0;
|
||||
do {
|
||||
const batch = await getJson<{
|
||||
headEventId: number;
|
||||
endEventId?: number;
|
||||
events: { eventId: number; requestId: string }[];
|
||||
}>(`${vaultUrl}/events-since?after=${after}`, token);
|
||||
assert.equal(
|
||||
batch.headEventId,
|
||||
snapshot.headEventId,
|
||||
"Server changed while checking convergence"
|
||||
);
|
||||
const end = batch.endEventId ?? batch.headEventId;
|
||||
assert.equal(
|
||||
end,
|
||||
batch.events.at(-1)?.eventId ?? after,
|
||||
"Event page cursor differs from its contents"
|
||||
);
|
||||
if (after < snapshot.headEventId)
|
||||
assert(end > after, "Event page made no progress");
|
||||
events.push(...batch.events);
|
||||
after = end;
|
||||
} while (after < snapshot.headEventId);
|
||||
assert.deepEqual(
|
||||
events.map((e) => e.eventId),
|
||||
Array.from({ length: snapshot.headEventId }, (_, i) => i + 1),
|
||||
"Server event log is not contiguous and totally ordered"
|
||||
);
|
||||
assert.equal(
|
||||
new Set(events.map((e) => e.requestId)).size,
|
||||
events.length,
|
||||
"An idempotent request emitted multiple events"
|
||||
);
|
||||
return snapshot;
|
||||
}
|
||||
103
frontend/test-support/content-limits.test.ts
Normal file
103
frontend/test-support/content-limits.test.ts
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { fixture, bytes, randomUUID } from "./sync-server-fixture";
|
||||
test("stale manifest reconciliation must not fetch excluded remote content", async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const id = randomUUID(),
|
||||
big = randomUUID();
|
||||
await f.put("/documents/" + id, f.content("A"));
|
||||
const m = await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
await f.client.setSetting("maxFileSizeMB", 100 / (1024 * 1024));
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.disk.userRename("a.md", "renamed.md");
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "renamed.md"
|
||||
});
|
||||
let injected = false,
|
||||
downloaded = 0;
|
||||
f.setFetchHook(async (input, init) => {
|
||||
const path = String(input);
|
||||
if (
|
||||
!injected &&
|
||||
init?.method === "PUT" &&
|
||||
path.endsWith("/file-manifest")
|
||||
) {
|
||||
injected = true;
|
||||
await f.put("/documents/" + big, f.content("B".repeat(4096)));
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: m.fileManifestId,
|
||||
entries: { [id]: "a.md", [big]: "big.bin" }
|
||||
});
|
||||
}
|
||||
const response = await fetch(input, init);
|
||||
if (
|
||||
path.includes("/documents/" + big) &&
|
||||
!path.endsWith("/metadata") &&
|
||||
(!init?.method || init.method === "GET")
|
||||
)
|
||||
downloaded += (await response.clone().arrayBuffer()).byteLength;
|
||||
return response;
|
||||
});
|
||||
await f.client.setSetting("isSyncEnabled", true);
|
||||
|
||||
assert(injected);
|
||||
assert(!f.disk.userFiles().has("big.bin"));
|
||||
assert.equal(downloaded, 0);
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
test("stale content acknowledgements contain metadata without the excluded payload", async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const id = randomUUID();
|
||||
await f.put("/documents/" + id, f.content("B".repeat(4096)));
|
||||
const stale = await f.put("/documents/" + id, f.content("LOCAL"));
|
||||
assert.equal(stale.type, "StaleBase");
|
||||
assert.equal(stale.contentSize, 4096);
|
||||
assert.equal(stale.contentBase64, undefined);
|
||||
const metadata = await f.get("/documents/" + id + "/metadata");
|
||||
assert.equal(metadata.contentSize, 4096);
|
||||
assert.equal(metadata.contentBase64, undefined);
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
test("highly fragmented local edits use a snapshot within the server operation budget", async () => {
|
||||
const { fixture } = await import("./sync-fixture");
|
||||
const { toStoredSnapshot } = await import(
|
||||
"../sync-client/src/sync-operations/content"
|
||||
);
|
||||
const f = await fixture({});
|
||||
f.contentCache.resize(1024 * 1024);
|
||||
const original = Array.from({ length: 12_000 }, (_, i) => `word${i}`).join(
|
||||
" "
|
||||
);
|
||||
const edited = Array.from({ length: 12_000 }, (_, i) =>
|
||||
i % 2 ? `changed${i}` : `word${i}`
|
||||
).join(" ");
|
||||
f.contentCache.put(1, bytes(original));
|
||||
const syncer = f.syncer as unknown as {
|
||||
pushContent(
|
||||
path: string,
|
||||
version: number,
|
||||
snapshot: unknown
|
||||
): Promise<{ type: string }>;
|
||||
};
|
||||
const request = await syncer.pushContent(
|
||||
"a.md",
|
||||
1,
|
||||
await toStoredSnapshot({ content: bytes(edited) })
|
||||
);
|
||||
assert.equal(request.type, "Snapshot");
|
||||
});
|
||||
910
frontend/test-support/engine-regressions.test.ts
Normal file
910
frontend/test-support/engine-regressions.test.ts
Normal file
|
|
@ -0,0 +1,910 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { PermanentSyncError } from "../sync-client/src/errors/errors";
|
||||
import { test } from "node:test";
|
||||
import {
|
||||
Database,
|
||||
emptyState,
|
||||
type StoredDatabase
|
||||
} from "../sync-client/src/persistence/database";
|
||||
import { Settings } from "../sync-client/src/persistence/settings";
|
||||
import { FileOperations } from "../sync-client/src/file-operations/file-operations";
|
||||
import { Syncer } from "../sync-client/src/sync-operations/syncer";
|
||||
import { SyncClient } from "../sync-client/src/sync-client";
|
||||
import { toStoredSnapshot } from "../sync-client/src/sync-operations/content";
|
||||
import { Logger } from "../sync-client/src/tracing/logger";
|
||||
import { SyncHistory } from "../sync-client/src/tracing/sync-history";
|
||||
import { FixedSizeDocumentCache } from "../sync-client/src/utils/data-structures/fix-sized-cache";
|
||||
import { EventListeners } from "../sync-client/src/utils/data-structures/event-listeners";
|
||||
import { allocatePortablePath } from "../sync-client/src/utils/portable-path";
|
||||
import type { ServerConfig } from "../sync-client/src/services/server-config";
|
||||
import type { SyncService } from "../sync-client/src/services/sync-service";
|
||||
import type { WebSocketManager } from "../sync-client/src/services/websocket-manager";
|
||||
import { MemoryDisk, MemoryPersistence } from "./storage";
|
||||
import type { LocalChange } from "../sync-client/src/sync-operations/local-changes";
|
||||
|
||||
const bytes = (text: string) => new TextEncoder().encode(text);
|
||||
const head = (id: string, version: number, content: string) => ({
|
||||
documentId: id,
|
||||
vaultUpdateId: version,
|
||||
contentSize: bytes(content).length,
|
||||
updatedDate: "2026-09-19T00:00:00Z",
|
||||
userId: "user",
|
||||
deviceId: "device"
|
||||
});
|
||||
|
||||
async function fixture(
|
||||
contents: Record<string, string> = { "a.md": "local" },
|
||||
service: Partial<SyncService> = {}
|
||||
) {
|
||||
const disk = new MemoryDisk();
|
||||
const initial = emptyState("test");
|
||||
initial.initialized = true;
|
||||
for (const [path, content] of Object.entries(contents)) {
|
||||
await disk.userWrite(path, bytes(content));
|
||||
const id = path.split(".")[0];
|
||||
initial.local[id] = path;
|
||||
initial.documents[id] = {
|
||||
materialized: true,
|
||||
observedHash: (await toStoredSnapshot({ content: bytes(content) }))
|
||||
.hash
|
||||
};
|
||||
}
|
||||
initial.fileManifest = { fileManifestId: 1, entries: { ...initial.local } };
|
||||
const persistence = new MemoryPersistence({ database: initial });
|
||||
const logger = new Logger();
|
||||
const database = new Database(
|
||||
logger,
|
||||
initial,
|
||||
async (next) => persistence.save({ database: next }),
|
||||
"test",
|
||||
async () => (await persistence.load()).database as StoredDatabase
|
||||
);
|
||||
const config = {
|
||||
initialize: async () => {},
|
||||
getConfig: async () => ({ mergeableFileExtensions: ["md"] }),
|
||||
reset: () => {}
|
||||
} as unknown as ServerConfig;
|
||||
const settings = new Settings(
|
||||
logger,
|
||||
{ syncIntervalMs: 0 },
|
||||
async () => {}
|
||||
);
|
||||
const changes: LocalChange[] = [];
|
||||
const files = new FileOperations(
|
||||
disk.session(),
|
||||
database,
|
||||
config,
|
||||
undefined,
|
||||
{ entries: () => changes, flush: async () => {} }
|
||||
);
|
||||
const websocket = {
|
||||
onWebSocketStatusChanged: new EventListeners(),
|
||||
onRemoteVaultUpdateReceived: new EventListeners()
|
||||
} as unknown as WebSocketManager;
|
||||
const syncer = new Syncer(
|
||||
"device",
|
||||
logger,
|
||||
database,
|
||||
settings,
|
||||
service as SyncService,
|
||||
websocket,
|
||||
files,
|
||||
config,
|
||||
new SyncHistory(logger),
|
||||
new FixedSizeDocumentCache(1024),
|
||||
{ entries: changes, save: async () => {} }
|
||||
);
|
||||
const internals = syncer as unknown as {
|
||||
finishPending(): Promise<void>;
|
||||
preparePush(): Promise<boolean>;
|
||||
scan(): Promise<void>;
|
||||
incorporateEventBatch(
|
||||
batch: import("../sync-client/src/services/types/EventBatch").EventBatch
|
||||
): Promise<void>;
|
||||
stopped: boolean;
|
||||
incorporateFileManifest(remote: {
|
||||
fileManifestId: number;
|
||||
entries: Record<string, string>;
|
||||
}): Promise<void>;
|
||||
incorporateContent(remote: ReturnType<typeof head>): Promise<void>;
|
||||
};
|
||||
return {
|
||||
disk,
|
||||
database,
|
||||
files,
|
||||
settings,
|
||||
syncer,
|
||||
internals,
|
||||
persistence,
|
||||
changes
|
||||
};
|
||||
}
|
||||
|
||||
test("remote binary replacement retains displaced bytes outside the visible namespace", async () => {
|
||||
const f = await fixture({ "a.bin": "private edit" });
|
||||
const next = structuredClone(f.database.state);
|
||||
await f.files.apply(next, {
|
||||
a: {
|
||||
expected: await f.files.snapshot("a.bin"),
|
||||
replacement: await toStoredSnapshot({ content: bytes("remote") })
|
||||
}
|
||||
});
|
||||
assert.deepEqual(f.disk.userFiles(), new Map([["a.bin", bytes("remote")]]));
|
||||
const retained = (await f.disk.listFilesRecursively()).filter((path) =>
|
||||
path.startsWith(".vault-link-sync/recovery/")
|
||||
);
|
||||
assert(
|
||||
(
|
||||
await Promise.all(retained.map((path) => f.disk.readSnapshot(path)))
|
||||
).some(
|
||||
(value) =>
|
||||
value &&
|
||||
Buffer.from(value.content).toString() === "private edit"
|
||||
)
|
||||
);
|
||||
await f.files.recover();
|
||||
assert.deepEqual(
|
||||
await f.disk.listFilesRecursively(),
|
||||
["a.bin", ...retained].sort()
|
||||
);
|
||||
});
|
||||
|
||||
test("remote deletion retains unsent bytes for recovery", async () => {
|
||||
const f = await fixture();
|
||||
await f.internals.incorporateFileManifest({
|
||||
fileManifestId: 2,
|
||||
entries: {}
|
||||
});
|
||||
assert.equal(f.disk.userFiles().size, 0);
|
||||
const retained = await f.disk.listFilesRecursively();
|
||||
assert(
|
||||
(
|
||||
await Promise.all(retained.map((path) => f.disk.readSnapshot(path)))
|
||||
).some(
|
||||
(value) =>
|
||||
value && Buffer.from(value.content).toString() === "local"
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
test("rejected content recovery remains idempotent across a rename and network failure", async () => {
|
||||
let offline = true;
|
||||
const f = await fixture(undefined, {
|
||||
getDocumentVersionContent: async () => {
|
||||
if (offline) throw new Error("offline");
|
||||
return bytes("base");
|
||||
}
|
||||
});
|
||||
const snapshot = (await f.files.snapshot("a.md"))!;
|
||||
f.database.state.documents.a.base = {
|
||||
...head("a", 1, "base"),
|
||||
hash: (await toStoredSnapshot({ content: bytes("base") })).hash
|
||||
};
|
||||
f.database.state.pending = {
|
||||
type: "content",
|
||||
documentId: "a",
|
||||
snapshot,
|
||||
request: {
|
||||
requestId: "request",
|
||||
parentVersionId: 1,
|
||||
content: { type: "Snapshot", value: snapshot.contentBase64 }
|
||||
},
|
||||
response: {
|
||||
type: "StaleBase",
|
||||
...head("a", 2, "remote")
|
||||
}
|
||||
};
|
||||
await f.database.save();
|
||||
await f.disk.userRename("a.md", "renamed.md");
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "renamed.md"
|
||||
});
|
||||
await assert.rejects(f.internals.finishPending(), /offline/);
|
||||
offline = false;
|
||||
await f.internals.finishPending();
|
||||
assert.equal(f.database.state.pending, undefined);
|
||||
assert.equal(f.database.state.local.a, "renamed.md");
|
||||
});
|
||||
|
||||
test("conflict names respect component limits without imposing a total path limit", () => {
|
||||
const wanted = "a".repeat(235) + ".md";
|
||||
const allocated = allocatePortablePath(
|
||||
wanted,
|
||||
"00000000-0000-4000-8000-000000000001",
|
||||
{ other: wanted }
|
||||
);
|
||||
assert(Buffer.byteLength(allocated) <= 255);
|
||||
assert(allocated.endsWith(".md"));
|
||||
assert.notEqual(allocated, wanted);
|
||||
const longPath = "folder/".repeat(100) + "note.md";
|
||||
assert.equal(allocatePortablePath(longPath, "id", {}), longPath);
|
||||
});
|
||||
|
||||
test("an unmaterialized file with a directory at its path is deconflicted", async () => {
|
||||
const f = await fixture(
|
||||
{},
|
||||
{ getDocumentVersionContent: async () => bytes("remote") }
|
||||
);
|
||||
await f.disk.userWrite("folder/note.md", bytes("local"));
|
||||
f.database.state.local.a = "folder";
|
||||
f.database.state.documents.a = { materialized: false };
|
||||
f.database.state.fileManifest.entries.a = "folder";
|
||||
await f.database.save();
|
||||
await f.internals.incorporateContent(head("a", 2, "remote"));
|
||||
assert(
|
||||
[...f.disk.userFiles().values()].some(
|
||||
(value) => Buffer.from(value).toString() === "remote"
|
||||
)
|
||||
);
|
||||
assert(
|
||||
[...f.disk.userFiles().values()].some(
|
||||
(value) => Buffer.from(value).toString() === "local"
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
test("ignored materialized content survives remote deletion", async () => {
|
||||
const f = await fixture();
|
||||
await f.settings.setSettings({ ignorePatterns: ["a.md"] });
|
||||
await f.internals.incorporateFileManifest({
|
||||
fileManifestId: 2,
|
||||
entries: {}
|
||||
});
|
||||
assert.deepEqual(f.disk.userFiles(), new Map([["a.md", bytes("local")]]));
|
||||
assert.deepEqual(f.database.state.fileManifest.entries, {});
|
||||
});
|
||||
|
||||
test("unfinished bootstrap cannot be rebound to another vault", async () => {
|
||||
const database = emptyState(JSON.stringify(["http://example.test", "old"]));
|
||||
database.bootstrap = {
|
||||
headEventId: 1,
|
||||
fileManifest: { fileManifestId: 1, entries: {} },
|
||||
documents: []
|
||||
};
|
||||
const persistence = new MemoryPersistence({
|
||||
settings: { remoteUri: "http://example.test", vaultName: "old" },
|
||||
database
|
||||
});
|
||||
const client = await SyncClient.create({
|
||||
fs: new MemoryDisk(),
|
||||
persistence
|
||||
});
|
||||
try {
|
||||
await assert.rejects(
|
||||
client.setSettings({ vaultName: "new" }),
|
||||
/separate state store/
|
||||
);
|
||||
assert.equal(persistence.snapshot().settings!.vaultName, "old");
|
||||
} finally {
|
||||
await client.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
test("local swap arriving after a remote journal is saved preserves both identities and contents", async () => {
|
||||
const f = await fixture({ "a.md": "A", "b.md": "B" });
|
||||
let fired = false;
|
||||
f.persistence.boundary = async (label) => {
|
||||
if (
|
||||
fired ||
|
||||
label !== "durable:save" ||
|
||||
!f.persistence.snapshot().database?.application
|
||||
)
|
||||
return;
|
||||
fired = true;
|
||||
for (const [oldPath, relativePath] of [
|
||||
["a.md", "tmp.md"],
|
||||
["b.md", "a.md"],
|
||||
["tmp.md", "b.md"]
|
||||
]) {
|
||||
await f.disk.userRename(oldPath, relativePath);
|
||||
await f.syncer.syncLocallyUpdatedFile({ oldPath, relativePath });
|
||||
}
|
||||
};
|
||||
const next = structuredClone(f.database.state);
|
||||
next.local = { a: "remote-a.md", b: "remote-b.md" };
|
||||
await f.files.apply(next);
|
||||
assert(fired);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
f.disk.userFiles().get(f.database.state.local.a)!
|
||||
).toString(),
|
||||
"A"
|
||||
);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
f.disk.userFiles().get(f.database.state.local.b)!
|
||||
).toString(),
|
||||
"B"
|
||||
);
|
||||
assert.deepEqual(f.database.state.local, { a: "b.md", b: "a.md" });
|
||||
});
|
||||
|
||||
test("a rejected upload is retained locally and does not block another file or a corrected snapshot", async () => {
|
||||
let reject = true;
|
||||
const sent: string[] = [];
|
||||
const f = await fixture(
|
||||
{ "a.md": "too large", "b.md": "other" },
|
||||
{
|
||||
putFileContent: async (id) => {
|
||||
sent.push(id);
|
||||
if (id === "a" && reject)
|
||||
throw new PermanentSyncError("HTTP 413");
|
||||
return {
|
||||
type: "Accepted",
|
||||
...head(id, sent.length + 2, "small")
|
||||
};
|
||||
}
|
||||
}
|
||||
);
|
||||
f.internals.stopped = false;
|
||||
assert(await f.internals.preparePush());
|
||||
await f.internals.finishPending();
|
||||
assert.equal(f.database.state.pending, undefined);
|
||||
assert.match(f.database.state.documents.a.rejected!.message, /413/);
|
||||
assert(await f.internals.preparePush());
|
||||
await f.internals.finishPending();
|
||||
assert.deepEqual(sent, ["a", "b"]);
|
||||
assert.equal(await f.internals.preparePush(), false);
|
||||
await f.disk.userWrite("a.md", bytes("small"));
|
||||
reject = false;
|
||||
assert(await f.internals.preparePush());
|
||||
await f.internals.finishPending();
|
||||
assert.deepEqual(sent, ["a", "b", "a"]);
|
||||
assert.equal(f.database.state.documents.a.rejected, undefined);
|
||||
});
|
||||
|
||||
test("a transient upload failure retries the exact request and recovers its acknowledgement", async () => {
|
||||
const requests: unknown[] = [];
|
||||
const f = await fixture(undefined, {
|
||||
putFileContent: async (_id, request) => {
|
||||
requests.push(structuredClone(request));
|
||||
if (requests.length === 1) throw new Error("lost upload reply");
|
||||
return { type: "Accepted", ...head("a", 2, "local") };
|
||||
}
|
||||
});
|
||||
f.internals.stopped = false;
|
||||
await f.internals.preparePush();
|
||||
await assert.rejects(f.internals.finishPending(), /lost upload reply/);
|
||||
assert(f.database.state.pending);
|
||||
await f.internals.finishPending();
|
||||
assert.equal(requests.length, 2);
|
||||
assert.deepEqual(requests[1], requests[0]);
|
||||
assert.equal(f.database.state.documents.a.base!.vaultUpdateId, 2);
|
||||
assert.equal(f.database.state.documents.a.rejected, undefined);
|
||||
assert.equal(f.database.state.pending, undefined);
|
||||
});
|
||||
|
||||
test("a persisted upload rejection finishes locally after a recovery write fails", async () => {
|
||||
let uploads = 0;
|
||||
const f = await fixture(undefined, {
|
||||
putFileContent: async () => {
|
||||
uploads++;
|
||||
throw new PermanentSyncError("HTTP 413");
|
||||
}
|
||||
});
|
||||
f.internals.stopped = false;
|
||||
await f.internals.preparePush();
|
||||
const { pending } = f.database.state;
|
||||
assert(pending);
|
||||
const { requestId } = pending.request;
|
||||
const retained = `.vault-link-sync/requests/${requestId}.content`;
|
||||
const { boundary } = f.disk;
|
||||
f.disk.boundary = (label): void => {
|
||||
if (label === `before:write:${retained}`)
|
||||
throw new Error("recovery write failed");
|
||||
};
|
||||
await assert.rejects(f.internals.finishPending(), /recovery write failed/);
|
||||
assert.equal(
|
||||
f.persistence.snapshot().database?.pending?.rejection,
|
||||
"HTTP 413"
|
||||
);
|
||||
f.disk.boundary = boundary;
|
||||
await f.internals.finishPending();
|
||||
assert.equal(uploads, 1);
|
||||
assert.equal(f.database.state.pending, undefined);
|
||||
assert.equal(f.database.state.documents.a.rejected?.message, "HTTP 413");
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot(retained))?.content,
|
||||
bytes("local")
|
||||
);
|
||||
assert.deepEqual(f.disk.userFiles(), new Map([["a.md", bytes("local")]]));
|
||||
});
|
||||
|
||||
test("event replay incorporates an earlier accepted upload after a permanent retry rejection", async () => {
|
||||
const f = await fixture(undefined, {
|
||||
putFileContent: async () => {
|
||||
throw new PermanentSyncError(
|
||||
"HTTP 413 after proxy reconfiguration"
|
||||
);
|
||||
},
|
||||
getDocumentVersionContent: async () => bytes("local")
|
||||
});
|
||||
f.internals.stopped = false;
|
||||
await f.internals.preparePush();
|
||||
const { pending } = f.database.state;
|
||||
assert(pending);
|
||||
const { requestId } = pending.request;
|
||||
await f.internals.finishPending();
|
||||
await assert.rejects(f.syncer.waitUntilFinished(), /413/);
|
||||
f.database.state.lastSeenUpdateId = 1;
|
||||
await f.internals.incorporateEventBatch({
|
||||
headEventId: 2,
|
||||
events: [
|
||||
{
|
||||
eventId: 2,
|
||||
requestId,
|
||||
type: "content",
|
||||
document: head("a", 2, "local")
|
||||
}
|
||||
]
|
||||
});
|
||||
assert.equal(f.database.state.documents.a.base?.vaultUpdateId, 2);
|
||||
assert.equal(f.database.state.pending, undefined);
|
||||
assert.equal(await f.internals.preparePush(), false);
|
||||
await f.syncer.waitUntilFinished();
|
||||
assert.deepEqual(f.disk.userFiles(), new Map([["a.md", bytes("local")]]));
|
||||
});
|
||||
|
||||
test("rejected manifests can be replaced after editing the local namespace", async () => {
|
||||
let reject = true;
|
||||
const f = await fixture(
|
||||
{},
|
||||
{
|
||||
pushFileManifest: async () => {
|
||||
if (reject) throw new PermanentSyncError("HTTP 413");
|
||||
return { type: "Accepted", fileManifestId: 3 };
|
||||
}
|
||||
}
|
||||
);
|
||||
f.database.state.fileManifest.entries = { a: "old.md" };
|
||||
f.database.state.pending = {
|
||||
type: "fileManifest",
|
||||
request: {
|
||||
requestId: "request",
|
||||
parentFileManifestId: 1,
|
||||
entries: { a: "new.md" }
|
||||
}
|
||||
};
|
||||
f.database.state.local = { a: "new.md" };
|
||||
f.database.state.documents.a = {
|
||||
materialized: false,
|
||||
base: { ...head("a", 1, "x"), hash: "hash" }
|
||||
};
|
||||
await f.database.save();
|
||||
await f.internals.finishPending();
|
||||
f.internals.stopped = false;
|
||||
assert.equal(await f.internals.preparePush(), false);
|
||||
f.database.state.local = {};
|
||||
reject = false;
|
||||
assert(await f.internals.preparePush());
|
||||
await f.internals.finishPending();
|
||||
assert.deepEqual(f.database.state.fileManifest.entries, {});
|
||||
});
|
||||
|
||||
test("offline delete/recreate identities survive restarting before any network connection", async () => {
|
||||
const key = JSON.stringify(["http://offline.test", "test"]);
|
||||
const initial = emptyState(key);
|
||||
initial.initialized = true;
|
||||
initial.local.a = "a.md";
|
||||
initial.documents.a = { materialized: true };
|
||||
initial.fileManifest = { fileManifestId: 1, entries: { a: "a.md" } };
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite("a.md", bytes("same"));
|
||||
const persistence = new MemoryPersistence({
|
||||
database: initial,
|
||||
settings: { remoteUri: "http://offline.test", vaultName: "test" }
|
||||
});
|
||||
const first = await SyncClient.create({ fs: disk, persistence });
|
||||
await disk.userDelete("a.md");
|
||||
await first.syncLocallyDeletedFile("a.md");
|
||||
await disk.userWrite("a.md", bytes("same"));
|
||||
await first.syncLocallyCreatedFile("a.md");
|
||||
await first.destroy();
|
||||
const second = await SyncClient.create({ fs: disk, persistence });
|
||||
try {
|
||||
const internals = second as unknown as {
|
||||
syncer: { scan(): Promise<void> };
|
||||
database: Database;
|
||||
};
|
||||
await internals.syncer.scan();
|
||||
assert.equal(internals.database.state.local.a, undefined);
|
||||
assert.equal(
|
||||
Object.values(internals.database.state.local).filter(
|
||||
(path) => path === "a.md"
|
||||
).length,
|
||||
1
|
||||
);
|
||||
} finally {
|
||||
await second.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
test("offline notifications bind an otherwise empty client to its vault", async () => {
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite("a.md", bytes("local"));
|
||||
const persistence = new MemoryPersistence({
|
||||
settings: { remoteUri: "http://offline.test", vaultName: "test" }
|
||||
});
|
||||
const client = await SyncClient.create({ fs: disk, persistence });
|
||||
await client.syncLocallyCreatedFile("a.md");
|
||||
await assert.rejects(
|
||||
client.setSettings({ vaultName: "other" }),
|
||||
/separate state store/
|
||||
);
|
||||
await client.destroy();
|
||||
const stored = persistence.snapshot();
|
||||
stored.settings!.vaultName = "other";
|
||||
await persistence.save(stored);
|
||||
await assert.rejects(
|
||||
SyncClient.create({ fs: disk, persistence }),
|
||||
/another vault/
|
||||
);
|
||||
});
|
||||
|
||||
test("unignoring a deleted file resumes the deferred remote deletion without resurrection", async () => {
|
||||
const f = await fixture();
|
||||
await f.settings.setSettings({ ignorePatterns: ["a.md"] });
|
||||
await f.internals.incorporateFileManifest({
|
||||
fileManifestId: 2,
|
||||
entries: {}
|
||||
});
|
||||
await f.settings.setSettings({ ignorePatterns: [] });
|
||||
await f.internals.incorporateFileManifest(f.database.state.fileManifest);
|
||||
assert.equal(f.disk.userFiles().size, 0);
|
||||
assert.deepEqual(f.database.state.local, {});
|
||||
});
|
||||
|
||||
test("an ignored file does not move when a remote rename targets its path", async () => {
|
||||
const f = await fixture({ "a.md": "A", "b.md": "B" });
|
||||
await f.settings.setSettings({ ignorePatterns: ["a.md"] });
|
||||
await f.internals.incorporateFileManifest({
|
||||
fileManifestId: 2,
|
||||
entries: { a: "c.md", b: "a.md" }
|
||||
});
|
||||
assert.deepEqual(
|
||||
f.disk.userFiles(),
|
||||
new Map([
|
||||
["a.md", bytes("A")],
|
||||
["b.md", bytes("B")]
|
||||
])
|
||||
);
|
||||
f.internals.stopped = false;
|
||||
assert.equal(await f.internals.preparePush(), false);
|
||||
assert.deepEqual(
|
||||
f.disk.userFiles(),
|
||||
new Map([
|
||||
["a.md", bytes("A")],
|
||||
["b.md", bytes("B")]
|
||||
])
|
||||
);
|
||||
});
|
||||
|
||||
for (const kind of ["path", "content"] as const) {
|
||||
test(`a remote ${kind} change that was reverted before observation preserves the local edit`, async () => {
|
||||
const f = await fixture(
|
||||
{ "a.md": "local" },
|
||||
{
|
||||
getDocumentVersionContent: async ({ vaultUpdateId }) =>
|
||||
bytes(vaultUpdateId === 2 ? "temporary" : "base")
|
||||
}
|
||||
);
|
||||
f.database.state.lastSeenUpdateId = 1;
|
||||
f.database.state.documents.a.base = {
|
||||
...head("a", 1, "base"),
|
||||
hash: (await toStoredSnapshot({ content: bytes("base") })).hash
|
||||
};
|
||||
if (kind === "path") {
|
||||
await f.disk.userRename("a.md", "c.md");
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "c.md"
|
||||
});
|
||||
}
|
||||
await f.internals.incorporateEventBatch({
|
||||
headEventId: 3,
|
||||
events: [2, 3].map((id) =>
|
||||
kind === "path"
|
||||
? {
|
||||
eventId: id,
|
||||
requestId: String(id),
|
||||
type: "fileManifest",
|
||||
fileManifest: {
|
||||
fileManifestId: id,
|
||||
entries: { a: id === 2 ? "b.md" : "a.md" }
|
||||
}
|
||||
}
|
||||
: {
|
||||
eventId: id,
|
||||
requestId: String(id),
|
||||
type: "content",
|
||||
document: head(
|
||||
"a",
|
||||
id,
|
||||
id === 2 ? "temporary" : "base"
|
||||
)
|
||||
}
|
||||
)
|
||||
});
|
||||
assert.equal(
|
||||
f.database.state.local.a,
|
||||
kind === "path" ? "c.md" : "a.md"
|
||||
);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.disk.readSnapshot(f.database.state.local.a))!.content
|
||||
).toString(),
|
||||
"local"
|
||||
);
|
||||
assert.equal(f.database.state.lastSeenUpdateId, 3);
|
||||
});
|
||||
}
|
||||
|
||||
test("concurrent settings updates cannot bind the engine to a different vault than its settings", async () => {
|
||||
const disk = new MemoryDisk();
|
||||
const persistence = new MemoryPersistence();
|
||||
const client = await SyncClient.create({ fs: disk, persistence });
|
||||
try {
|
||||
await Promise.all([
|
||||
client.setSettings({ vaultName: "changed" }),
|
||||
client.setSettings({ maxFileSizeMB: 20 })
|
||||
]);
|
||||
const stored = persistence.snapshot();
|
||||
assert.equal(
|
||||
stored.database!.vaultKey,
|
||||
JSON.stringify([
|
||||
stored.settings!.remoteUri,
|
||||
stored.settings!.vaultName
|
||||
])
|
||||
);
|
||||
assert.equal(stored.settings!.maxFileSizeMB, 20);
|
||||
} finally {
|
||||
await client.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
test("destroy waits for an in-progress offline notification to become durable", async () => {
|
||||
const disk = new MemoryDisk();
|
||||
const persistence = new MemoryPersistence();
|
||||
const client = await SyncClient.create({ fs: disk, persistence });
|
||||
await disk.userWrite("a.md", bytes("local"));
|
||||
const reached = Promise.withResolvers<void>();
|
||||
const release = Promise.withResolvers<void>();
|
||||
persistence.boundary = async (label) => {
|
||||
if (label === "before:save") {
|
||||
reached.resolve();
|
||||
await release.promise;
|
||||
}
|
||||
};
|
||||
const notification = client.syncLocallyCreatedFile("a.md");
|
||||
await reached.promise;
|
||||
let destroyed = false;
|
||||
const destroying = client.destroy().then(() => {
|
||||
destroyed = true;
|
||||
});
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
assert.equal(destroyed, false);
|
||||
release.resolve();
|
||||
await Promise.all([notification, destroying]);
|
||||
});
|
||||
|
||||
test("ignoring a locally renamed file defers its move without forgetting the local rename", async () => {
|
||||
const f = await fixture();
|
||||
f.database.state.documents.a.base = {
|
||||
...head("a", 1, "local"),
|
||||
hash: f.database.state.documents.a.observedHash!
|
||||
};
|
||||
await f.disk.userRename("a.md", "ignored.md");
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "ignored.md"
|
||||
});
|
||||
await f.settings.setSettings({ ignorePatterns: ["ignored.md"] });
|
||||
await f.internals.scan();
|
||||
f.internals.stopped = false;
|
||||
assert.equal(await f.internals.preparePush(), false);
|
||||
await f.internals.incorporateFileManifest({
|
||||
fileManifestId: 2,
|
||||
entries: { a: "a.md" }
|
||||
});
|
||||
await f.settings.setSettings({ ignorePatterns: [] });
|
||||
await f.internals.incorporateFileManifest(f.database.state.fileManifest);
|
||||
assert.equal(f.database.state.local.a, "ignored.md");
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.disk.readSnapshot("ignored.md"))!.content
|
||||
).toString(),
|
||||
"local"
|
||||
);
|
||||
});
|
||||
|
||||
test("a notified deletion of an ignored file does not delete its server membership", async () => {
|
||||
const f = await fixture();
|
||||
await f.settings.setSettings({ ignorePatterns: ["a.md"] });
|
||||
await f.disk.userDelete("a.md");
|
||||
await f.syncer.syncLocallyDeletedFile("a.md");
|
||||
await f.internals.scan();
|
||||
f.internals.stopped = false;
|
||||
assert.equal(await f.internals.preparePush(), false);
|
||||
assert.equal(f.database.state.local.a, undefined);
|
||||
await f.settings.setSettings({ ignorePatterns: [] });
|
||||
await f.internals.incorporateFileManifest(f.database.state.fileManifest);
|
||||
assert(await f.internals.preparePush());
|
||||
assert.equal(f.database.state.pending?.type, "fileManifest");
|
||||
});
|
||||
|
||||
test("an incoming ignored document cannot force an existing ignored file to move", async () => {
|
||||
const f = await fixture();
|
||||
await f.settings.setSettings({ ignorePatterns: ["a.md", "*conflict*"] });
|
||||
await f.internals.incorporateFileManifest({
|
||||
fileManifestId: 2,
|
||||
entries: { a: "b.md", other: "a.md" }
|
||||
});
|
||||
f.internals.stopped = false;
|
||||
await f.internals.preparePush();
|
||||
assert.deepEqual(f.disk.userFiles(), new Map([["a.md", bytes("local")]]));
|
||||
assert.equal(
|
||||
new Set(Object.values(f.database.state.local)).size,
|
||||
Object.keys(f.database.state.local).length
|
||||
);
|
||||
});
|
||||
|
||||
for (const boundary of ["visible", "durable"] as const) {
|
||||
for (const action of [
|
||||
"move-installed",
|
||||
"recreate-staged-source"
|
||||
] as const) {
|
||||
test(`${action} at a ${boundary} filesystem boundary keeps original identities and all bytes`, async () => {
|
||||
const f = await fixture({ "a.md": "A", "b.md": "B" });
|
||||
let changed = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (changed || !label.startsWith(`${boundary}:rename:`)) return;
|
||||
if (action === "move-installed" && label.endsWith("->b.md")) {
|
||||
changed = true;
|
||||
await f.disk.userRename("b.md", "later.md");
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "b.md",
|
||||
relativePath: "later.md"
|
||||
});
|
||||
} else if (
|
||||
action === "recreate-staged-source" &&
|
||||
label.includes("rename:a.md->")
|
||||
) {
|
||||
changed = true;
|
||||
await f.disk.userWrite("a.md", bytes("new"));
|
||||
await f.syncer.syncLocallyCreatedFile("a.md");
|
||||
}
|
||||
};
|
||||
const next = structuredClone(f.database.state);
|
||||
next.local = { a: "b.md", b: "a.md" };
|
||||
next.fileManifest = {
|
||||
fileManifestId: 2,
|
||||
entries: { ...next.local }
|
||||
};
|
||||
await f.files.apply(next);
|
||||
assert(changed);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.disk.readSnapshot(f.database.state.local.a))!
|
||||
.content
|
||||
).toString(),
|
||||
"A"
|
||||
);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.disk.readSnapshot(f.database.state.local.b))!
|
||||
.content
|
||||
).toString(),
|
||||
"B"
|
||||
);
|
||||
assert.deepEqual(
|
||||
[...f.disk.userFiles().values()]
|
||||
.map((value) => Buffer.from(value).toString())
|
||||
.sort(),
|
||||
action === "move-installed" ? ["A", "B"] : ["A", "B", "new"]
|
||||
);
|
||||
f.disk.crash(true);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.disk.readSnapshot(f.database.state.local.a))!
|
||||
.content
|
||||
).toString(),
|
||||
"A"
|
||||
);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.disk.readSnapshot(f.database.state.local.b))!
|
||||
.content
|
||||
).toString(),
|
||||
"B"
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
test("rejected content is reported to callers and an explicit retry clears the rejection", async () => {
|
||||
const f = await fixture(undefined, {
|
||||
putFileContent: async () => {
|
||||
throw new PermanentSyncError("HTTP 413");
|
||||
}
|
||||
});
|
||||
f.internals.stopped = false;
|
||||
await f.internals.preparePush();
|
||||
await f.internals.finishPending();
|
||||
await assert.rejects(f.syncer.waitUntilFinished(), /413/);
|
||||
assert(f.syncer.isBusy);
|
||||
await f.syncer.retryRejectedRequests();
|
||||
assert(await f.internals.preparePush());
|
||||
});
|
||||
|
||||
test("notification saves cannot roll back an uncertain engine commit, even if the first reload fails", async () => {
|
||||
const key = JSON.stringify(["http://offline.test", "test"]);
|
||||
const initial = emptyState(key);
|
||||
initial.initialized = true;
|
||||
const disk = new MemoryDisk();
|
||||
const store = new MemoryPersistence({
|
||||
database: initial,
|
||||
settings: { remoteUri: "http://offline.test", vaultName: "test" }
|
||||
});
|
||||
let failLoad = false;
|
||||
const persistence = {
|
||||
load: async () => {
|
||||
if (failLoad) {
|
||||
failLoad = false;
|
||||
throw new Error("reload failed");
|
||||
}
|
||||
return store.load();
|
||||
},
|
||||
save: async (state: Parameters<typeof store.save>[0]) =>
|
||||
store.save(state)
|
||||
};
|
||||
const client = await SyncClient.create({ fs: disk, persistence });
|
||||
try {
|
||||
const internals = client as unknown as {
|
||||
database: Database;
|
||||
syncer: Syncer;
|
||||
};
|
||||
const next = structuredClone(initial);
|
||||
next.lastSeenUpdateId = 2;
|
||||
store.boundary = (label) => {
|
||||
if (label === "durable:save") {
|
||||
store.boundary = () => {};
|
||||
throw new Error("durable save failed");
|
||||
}
|
||||
};
|
||||
await assert.rejects(
|
||||
internals.database.commit(next),
|
||||
/durable save failed/
|
||||
);
|
||||
await disk.userWrite("a.md", bytes("new"));
|
||||
failLoad = true;
|
||||
await assert.rejects(
|
||||
client.syncLocallyCreatedFile("a.md"),
|
||||
/reload failed/
|
||||
);
|
||||
await internals.syncer.flushLocalChanges();
|
||||
assert.equal(store.snapshot().database!.lastSeenUpdateId, 2);
|
||||
await internals.database.recoverPersistence();
|
||||
assert.equal(internals.database.state.lastSeenUpdateId, 2);
|
||||
} finally {
|
||||
await client.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
test(
|
||||
"a settings listener can await client shutdown without deadlocking the lifecycle lock",
|
||||
{ timeout: 1_000 },
|
||||
async () => {
|
||||
const client = await SyncClient.create({
|
||||
fs: new MemoryDisk(),
|
||||
persistence: new MemoryPersistence()
|
||||
});
|
||||
let notified = false;
|
||||
client.onSettingsChanged.add(async () => {
|
||||
await client.destroy();
|
||||
notified = true;
|
||||
});
|
||||
await client.setSettings({ maxFileSizeMB: 20 });
|
||||
assert(notified);
|
||||
}
|
||||
);
|
||||
157
frontend/test-support/event-delivery.test.ts
Normal file
157
frontend/test-support/event-delivery.test.ts
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import {
|
||||
Database,
|
||||
emptyState,
|
||||
type StoredDatabase
|
||||
} from "../sync-client/src/persistence/database";
|
||||
import { Settings } from "../sync-client/src/persistence/settings";
|
||||
import { FileOperations } from "../sync-client/src/file-operations/file-operations";
|
||||
import type { ServerConfig } from "../sync-client/src/services/server-config";
|
||||
import type { SyncService } from "../sync-client/src/services/sync-service";
|
||||
import type { WebSocketManager } from "../sync-client/src/services/websocket-manager";
|
||||
import type { EventBatch } from "../sync-client/src/services/types/EventBatch";
|
||||
import type { EventRecord } from "../sync-client/src/services/types/EventRecord";
|
||||
import { Syncer } from "../sync-client/src/sync-operations/syncer";
|
||||
import { Logger } from "../sync-client/src/tracing/logger";
|
||||
import { SyncHistory } from "../sync-client/src/tracing/sync-history";
|
||||
import { EventListeners } from "../sync-client/src/utils/data-structures/event-listeners";
|
||||
import { FixedSizeDocumentCache } from "../sync-client/src/utils/data-structures/fix-sized-cache";
|
||||
import { MemoryDisk, MemoryPersistence } from "./storage";
|
||||
|
||||
for (const [count, delivery] of [
|
||||
[1, "before the loop"],
|
||||
[3, "before the loop"],
|
||||
[3, "during the first batch"]
|
||||
] as const) {
|
||||
test(
|
||||
`${count} WS batches delivered ${delivery} drain without polling or another notification`,
|
||||
{ timeout: 5_000 },
|
||||
async () => {
|
||||
const logger = new Logger();
|
||||
const initial = emptyState("event-delivery");
|
||||
initial.initialized = true;
|
||||
const persistence = new MemoryPersistence({ database: initial });
|
||||
const database = new Database(
|
||||
logger,
|
||||
initial,
|
||||
async (next) => persistence.save({ database: next }),
|
||||
initial.vaultKey,
|
||||
async () =>
|
||||
(await persistence.load()).database as StoredDatabase
|
||||
);
|
||||
const settings = new Settings(
|
||||
logger,
|
||||
{ syncIntervalMs: 0 },
|
||||
async () => {}
|
||||
);
|
||||
assert.equal(settings.getSettings().syncIntervalMs, 0);
|
||||
const disk = new MemoryDisk();
|
||||
const initializing = Promise.withResolvers<void>();
|
||||
const initialized = Promise.withResolvers<void>();
|
||||
const firstBatch = Promise.withResolvers<void>();
|
||||
const continueBatch = Promise.withResolvers<void>();
|
||||
let heldBatch = false;
|
||||
const applied: number[] = [];
|
||||
persistence.boundary = async (label) => {
|
||||
if (label !== "durable:save") return;
|
||||
const cursor =
|
||||
persistence.snapshot().database!.lastSeenUpdateId;
|
||||
if (cursor && applied.at(-1) !== cursor) applied.push(cursor);
|
||||
if (
|
||||
delivery === "during the first batch" &&
|
||||
cursor === 1 &&
|
||||
!heldBatch
|
||||
) {
|
||||
heldBatch = true;
|
||||
firstBatch.resolve();
|
||||
await continueBatch.promise;
|
||||
}
|
||||
};
|
||||
const config = {
|
||||
initialize: async () => {
|
||||
initializing.resolve();
|
||||
await initialized.promise;
|
||||
},
|
||||
getConfig: async () => ({ mergeableFileExtensions: ["md"] }),
|
||||
reset: () => {}
|
||||
} as unknown as ServerConfig;
|
||||
const events: EventRecord[] = Array.from(
|
||||
{ length: count },
|
||||
(_, index) => ({
|
||||
eventId: index + 1,
|
||||
requestId: `manifest-${index + 1}`,
|
||||
type: "fileManifest",
|
||||
fileManifest: { fileManifestId: index + 1, entries: {} }
|
||||
})
|
||||
);
|
||||
const service = {
|
||||
// A correct client may drain the queue or catch up over HTTP. Either
|
||||
// must reach the same head without the test manually waking it.
|
||||
events: async (after: number): Promise<EventBatch> => ({
|
||||
headEventId: count,
|
||||
events: events.filter((event) => event.eventId > after)
|
||||
})
|
||||
} as unknown as SyncService;
|
||||
const websocket = {
|
||||
onWebSocketStatusChanged: new EventListeners<
|
||||
(connected: boolean) => unknown
|
||||
>(),
|
||||
onRemoteVaultUpdateReceived: new EventListeners<
|
||||
(batch: EventBatch) => unknown
|
||||
>()
|
||||
};
|
||||
const syncer = new Syncer(
|
||||
"device",
|
||||
logger,
|
||||
database,
|
||||
settings,
|
||||
service,
|
||||
websocket as unknown as WebSocketManager,
|
||||
new FileOperations(disk.session(), database, config),
|
||||
config,
|
||||
new SyncHistory(logger),
|
||||
new FixedSizeDocumentCache(1024)
|
||||
);
|
||||
const deliver = async (event: EventRecord) => {
|
||||
await websocket.onRemoteVaultUpdateReceived.triggerAsync({
|
||||
headEventId: event.eventId,
|
||||
events: [event]
|
||||
});
|
||||
};
|
||||
try {
|
||||
syncer.start();
|
||||
await initializing.promise;
|
||||
if (delivery === "before the loop") {
|
||||
for (const event of events) await deliver(event);
|
||||
initialized.resolve();
|
||||
} else {
|
||||
await deliver(events[0]);
|
||||
initialized.resolve();
|
||||
await firstBatch.promise;
|
||||
for (const event of events.slice(1)) await deliver(event);
|
||||
continueBatch.resolve();
|
||||
}
|
||||
await syncer.waitUntilFinished();
|
||||
assert.equal(
|
||||
database.state.lastSeenUpdateId,
|
||||
count,
|
||||
"Client became idle with already-delivered events unapplied"
|
||||
);
|
||||
assert.deepEqual(
|
||||
applied,
|
||||
events.map((event) => event.eventId),
|
||||
"Every event must be committed in order"
|
||||
);
|
||||
assert.equal(database.state.fileManifest.fileManifestId, count);
|
||||
assert.equal(database.state.application, undefined);
|
||||
assert.equal(database.state.pending, undefined);
|
||||
assert.equal(syncer.isBusy, false);
|
||||
} finally {
|
||||
initialized.resolve();
|
||||
continueBatch.resolve();
|
||||
await syncer.stop();
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
316
frontend/test-support/harness.test.ts
Normal file
316
frontend/test-support/harness.test.ts
Normal file
|
|
@ -0,0 +1,316 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { MemoryDisk, MemoryPersistence } from "./storage";
|
||||
import { NetworkFaults, classifyRequest } from "./network";
|
||||
import {
|
||||
ContentLedger,
|
||||
assertManifest,
|
||||
assertSameFiles,
|
||||
pathDecision
|
||||
} from "./oracles";
|
||||
import { Random } from "./random";
|
||||
import { TestRunner } from "../deterministic-tests/src/test-runner";
|
||||
import type { ServerControl } from "../deterministic-tests/src/server-control";
|
||||
import { Logger } from "sync-client";
|
||||
import { ManagedWebSocketFactory } from "../deterministic-tests/src/managed-websocket";
|
||||
|
||||
const bytes = (text: string) => new TextEncoder().encode(text);
|
||||
|
||||
test("consistency oracle rejects self comparison, divergent paths and invalid UTF-8 bytes", () => {
|
||||
const first = new Map([["a.bin", new Uint8Array([0xff])]]);
|
||||
assert.throws(() => assertSameFiles([first, first]), /itself/);
|
||||
assert.throws(() =>
|
||||
assertSameFiles([first, new Map([["a.bin", new Uint8Array([0xfe])]])])
|
||||
);
|
||||
assert.throws(() => assertSameFiles([first, new Map()]));
|
||||
assertSameFiles([first, structuredClone(first)]);
|
||||
});
|
||||
|
||||
test("ledger catches shared loss, duplicated bytes, and unrelated loss in delete workloads", () => {
|
||||
const ledger = new ContentLedger();
|
||||
ledger.add("first-marker");
|
||||
ledger.add("second-marker");
|
||||
ledger.removedBy("first-marker", "explicit delete of doc A");
|
||||
assert.throws(
|
||||
() => ledger.assertPreserved(new Map()),
|
||||
/second-marker.*lost/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
ledger.assertPreserved(
|
||||
new Map([["a", bytes("second-marker second-marker")]])
|
||||
),
|
||||
/duplicated/
|
||||
);
|
||||
ledger.assertPreserved(new Map([["a", bytes("second-marker")]]));
|
||||
});
|
||||
|
||||
test("memory adapter enforces v4 operations and snapshot ownership", async () => {
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite("folder/a", bytes("A"));
|
||||
const snap = (await disk.readSnapshot("folder/a"))!;
|
||||
snap.content[0] = 0;
|
||||
assert.equal(
|
||||
Buffer.from((await disk.readSnapshot("folder/a"))!.content).toString(),
|
||||
"A"
|
||||
);
|
||||
await assert.rejects(
|
||||
disk.write("folder/a", { content: bytes("B") }),
|
||||
/exists/
|
||||
);
|
||||
await disk.userWrite("folder/b", bytes("B"));
|
||||
await assert.rejects(disk.rename("folder/a", "folder/b"), /exists/);
|
||||
await assert.rejects(disk.delete("folder/a"), /regular file/);
|
||||
await assert.rejects(disk.delete("folder"), /regular file/);
|
||||
await assert.rejects(disk.readSnapshot("../outside"), /Unsafe/);
|
||||
await assert.rejects(disk.stat("folder/a/child"), /ancestor/);
|
||||
await assert.rejects(disk.readSnapshot("folder"), /directory/);
|
||||
});
|
||||
|
||||
for (const phase of ["before", "visible", "durable"] as const) {
|
||||
for (const powerLoss of [false, true]) {
|
||||
test(`${powerLoss ? "power loss" : "process death"} at ${phase} rename boundary`, async () => {
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite("a", bytes("A"));
|
||||
const session = disk.session();
|
||||
let fired = false;
|
||||
disk.boundary = (label) => {
|
||||
if (label === `${phase}:rename:a->b`) {
|
||||
fired = true;
|
||||
disk.crash(powerLoss);
|
||||
throw new Error("crash");
|
||||
}
|
||||
};
|
||||
await assert.rejects(session.rename("a", "b"), /crash/);
|
||||
assert(fired);
|
||||
disk.boundary = () => {};
|
||||
const moved =
|
||||
phase === "durable" || (phase === "visible" && !powerLoss);
|
||||
assert.deepEqual([...disk.userFiles().keys()], [moved ? "b" : "a"]);
|
||||
assert.throws(
|
||||
() => session.write("stale", { content: bytes("bad") }),
|
||||
/crashed/
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
test("persistence has deep ownership and both old/new uncertain save outcomes", async () => {
|
||||
for (const phase of ["before", "durable"]) {
|
||||
const store = new MemoryPersistence({ settings: { vaultName: "old" } });
|
||||
const loaded = await store.load();
|
||||
loaded.settings!.vaultName = "mutated";
|
||||
assert.equal(store.snapshot().settings!.vaultName, "old");
|
||||
store.boundary = (label) => {
|
||||
if (label === `${phase}:save`) throw new Error("save failed");
|
||||
};
|
||||
await assert.rejects(
|
||||
store.save({ settings: { vaultName: "new" } }),
|
||||
/save failed/
|
||||
);
|
||||
assert.equal(
|
||||
store.snapshot().settings!.vaultName,
|
||||
phase === "before" ? "old" : "new"
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("CAS hooks match v4 create, update and manifest and preserve retry payload", async () => {
|
||||
for (const kind of ["create", "content", "manifest"] as const) {
|
||||
const net = new NetworkFaults();
|
||||
const url = `http://test/vaults/test/${kind === "manifest" ? "file-manifest" : "documents/uuid"}`;
|
||||
const body = {
|
||||
requestId: "same",
|
||||
parentVersionId: kind === "create" ? null : 3
|
||||
};
|
||||
const init = { method: "PUT", body: JSON.stringify(body) };
|
||||
assert.equal(classifyRequest(url, init)?.kind, kind);
|
||||
let committed = 0;
|
||||
const wrapped = net.wrap(async () => {
|
||||
committed++;
|
||||
return new Response('{"type":"Accepted"}');
|
||||
});
|
||||
net.arm(kind);
|
||||
await assert.rejects(wrapped(url, init), /Injected after-commit/);
|
||||
await net.wait();
|
||||
assert.throws(() => net.assertConsumed(), /never retried/);
|
||||
assert.equal(committed, 1);
|
||||
await wrapped(url, init);
|
||||
net.assertConsumed();
|
||||
await assert.rejects(
|
||||
wrapped(url, {
|
||||
...init,
|
||||
body: JSON.stringify({ ...body, changed: true })
|
||||
}),
|
||||
/Retry changed/
|
||||
);
|
||||
}
|
||||
assert.equal(
|
||||
classifyRequest("http://test/documents", { method: "POST" }),
|
||||
undefined
|
||||
);
|
||||
});
|
||||
|
||||
test("unconsumed and before-send faults cannot silently pass", async () => {
|
||||
const net = new NetworkFaults();
|
||||
net.arm("manifest", "before");
|
||||
assert.throws(() => net.assertConsumed(), /never fired/);
|
||||
let called = false;
|
||||
await assert.rejects(
|
||||
net.wrap(async () => {
|
||||
called = true;
|
||||
return new Response();
|
||||
})("http://test/file-manifest", {
|
||||
method: "PUT",
|
||||
body: '{"requestId":"r"}'
|
||||
})
|
||||
);
|
||||
assert(!called);
|
||||
assert.throws(() => net.assertConsumed(), /never retried/);
|
||||
await net.wrap(async () => new Response())("http://test/file-manifest", {
|
||||
method: "PUT",
|
||||
body: '{"requestId":"r"}'
|
||||
});
|
||||
net.assertConsumed();
|
||||
});
|
||||
|
||||
test("HTTP observation checkpoint blocks catchup, resumes, and respects abort", async () => {
|
||||
const network = new NetworkFaults();
|
||||
const wrapped = network.wrap(async () => new Response("{}"));
|
||||
network.pauseObservation();
|
||||
let delivered = false;
|
||||
const request = wrapped("http://test/events-since?after=0").then(() => {
|
||||
delivered = true;
|
||||
});
|
||||
await network.waitForObservation();
|
||||
assert(!delivered, "Catchup bypassed the observation checkpoint");
|
||||
network.resumeObservation();
|
||||
await request;
|
||||
network.assertConsumed();
|
||||
network.pauseObservation();
|
||||
const controller = new AbortController();
|
||||
const interrupted = wrapped("http://test/vault-snapshot", {
|
||||
signal: controller.signal
|
||||
});
|
||||
const rejected = assert.rejects(interrupted, /abort/i);
|
||||
await network.waitForObservation();
|
||||
controller.abort();
|
||||
await rejected;
|
||||
network.resumeObservation();
|
||||
network.assertConsumed();
|
||||
});
|
||||
|
||||
test("independent path decision truth table covers create, delete and rename", () => {
|
||||
for (const base of [undefined, "a", "b"])
|
||||
for (const local of [undefined, "a", "b"])
|
||||
for (const remote of [undefined, "a", "b"]) {
|
||||
const result = pathDecision(base, local, remote);
|
||||
assert.equal(result, remote !== base ? remote : local);
|
||||
}
|
||||
assertManifest({ a: "folder/é.md", b: "other.bin" });
|
||||
for (const entries of [
|
||||
{ a: "A", b: "a" },
|
||||
{ a: "é", b: "e\u0301" },
|
||||
{ a: "x", b: "x/y" }
|
||||
])
|
||||
assert.throws(() => assertManifest(entries));
|
||||
});
|
||||
|
||||
test("seeded actions are repeatable, including seed zero", () => {
|
||||
for (const seed of [0, 1, 12345, 0xffffffff]) {
|
||||
const first = new Random(seed);
|
||||
const second = new Random(seed);
|
||||
assert.deepEqual(
|
||||
Array.from({ length: 100 }, () => first.int(100)),
|
||||
Array.from({ length: 100 }, () => second.int(100))
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("runner cleanup errors fail an otherwise successful test", async () => {
|
||||
const server = {
|
||||
isRunning: () => true,
|
||||
resume: () => {}
|
||||
} as unknown as ServerControl;
|
||||
const runner = new TestRunner(
|
||||
server,
|
||||
new Logger(),
|
||||
"unused",
|
||||
"http://unused"
|
||||
);
|
||||
let cleaned = 0;
|
||||
Object.assign(runner, {
|
||||
initializeAgents: async () => {},
|
||||
agents: [
|
||||
{
|
||||
clientId: 0,
|
||||
database: () => ({ evidence: "before cleanup" }),
|
||||
disk: { image: () => ({ bytes: [0, 255] }) },
|
||||
network: { requests: [{ body: { requestId: "original" } }] },
|
||||
cleanup: async () => {
|
||||
cleaned++;
|
||||
throw new Error("late background error");
|
||||
}
|
||||
}
|
||||
]
|
||||
});
|
||||
const result = await runner.runTest("cleanup-probe", {
|
||||
clients: 2,
|
||||
steps: []
|
||||
});
|
||||
assert.equal(result.success, false);
|
||||
assert.match(result.error!, /late background error/);
|
||||
assert.equal(cleaned, 1, "Failed cleanup was run twice");
|
||||
assert.deepEqual(result.diagnostics, [
|
||||
{
|
||||
client: 0,
|
||||
database: { evidence: "before cleanup" },
|
||||
disk: { bytes: [0, 255] },
|
||||
requests: [{ body: { requestId: "original" } }]
|
||||
}
|
||||
]);
|
||||
});
|
||||
|
||||
test("disposed harness rejects late websocket reconnects", async () => {
|
||||
const factory = new ManagedWebSocketFactory();
|
||||
const Socket = factory.constructorFn;
|
||||
await factory.finish();
|
||||
assert.throws(() => new Socket("ws://unused"), /after harness cleanup/);
|
||||
});
|
||||
|
||||
test("runner preserves primary failure and cleanup diagnostics", async () => {
|
||||
const server = {
|
||||
isRunning: () => true,
|
||||
resume: () => {}
|
||||
} as unknown as ServerControl;
|
||||
const runner = new TestRunner(
|
||||
server,
|
||||
new Logger(),
|
||||
"unused",
|
||||
"http://unused"
|
||||
);
|
||||
Object.assign(runner, {
|
||||
initializeAgents: async () => {},
|
||||
executeStep: async () => {
|
||||
throw new Error("primary assertion failed");
|
||||
},
|
||||
agents: [
|
||||
{
|
||||
clientId: 0,
|
||||
database: () => ({}),
|
||||
disk: { image: () => ({}) },
|
||||
network: { requests: [] },
|
||||
cleanup: async () => {
|
||||
throw new Error("cleanup failed too");
|
||||
}
|
||||
}
|
||||
]
|
||||
});
|
||||
const result = await runner.runTest("cleanup-probe", {
|
||||
clients: 2,
|
||||
steps: [{ type: "barrier" }]
|
||||
});
|
||||
assert.equal(result.success, false);
|
||||
assert.match(result.error!, /primary assertion failed/);
|
||||
assert.match(result.error!, /cleanup failed too/);
|
||||
});
|
||||
407
frontend/test-support/journal-interleavings.test.ts
Normal file
407
frontend/test-support/journal-interleavings.test.ts
Normal file
|
|
@ -0,0 +1,407 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { fixture, bytes, head } from "./sync-fixture";
|
||||
import { toStoredSnapshot } from "../sync-client/src/sync-operations/content";
|
||||
|
||||
for (const origin of ["bootstrap", "history recovery"])
|
||||
test(`cancelling a replacement retains the ${origin} merge context`, async () => {
|
||||
const f = await fixture(
|
||||
{ "a.md": "BASE\n" },
|
||||
{ getDocumentVersionContent: async () => bytes("REMOTE\nBASE\n") }
|
||||
);
|
||||
const recoveryBase = await toStoredSnapshot({
|
||||
content: bytes("BASE\n")
|
||||
});
|
||||
if (origin === "bootstrap")
|
||||
f.database.state.documents.a.bootstrap = true;
|
||||
else f.database.state.documents.a.recoveryBase = recoveryBase;
|
||||
await f.database.save();
|
||||
await f.settings.setSetting("ignorePatterns", ["private.md"]);
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith("before:mkdir:.vault-link-sync/transactions/")
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userRename("a.md", "private.md");
|
||||
await f.disk.userWrite("private.md", bytes("BASE\nLOCAL\n"));
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "private.md"
|
||||
});
|
||||
};
|
||||
await f.internals.incorporateContent(head("a", 2, "REMOTE\nBASE\n"));
|
||||
assert(injected);
|
||||
const deferred = f.database.state.documents.a;
|
||||
assert.equal(deferred.base, undefined);
|
||||
if (origin === "bootstrap") assert.equal(deferred.bootstrap, true);
|
||||
else assert.deepEqual(deferred.recoveryBase, recoveryBase);
|
||||
f.disk.boundary = () => {};
|
||||
await f.settings.setSetting("ignorePatterns", []);
|
||||
await f.internals.incorporateFileManifest(
|
||||
f.database.state.fileManifest
|
||||
);
|
||||
const content = Buffer.from(
|
||||
f.disk.userFiles().get("private.md")!
|
||||
).toString();
|
||||
assert.equal(content.match(/REMOTE/g)?.length, 1);
|
||||
assert.equal(content.match(/LOCAL/g)?.length, 1);
|
||||
assert.equal(f.database.state.documents.a.base?.vaultUpdateId, 2);
|
||||
assert.equal(f.database.state.documents.a.recoveryBase, undefined);
|
||||
});
|
||||
|
||||
for (const timing of ["before", "durable"])
|
||||
test(`a save that exceeds the size limit ${timing} staging keeps its exact bytes`, async () => {
|
||||
const f = await fixture({ "a.bin": "PUBLIC" });
|
||||
await f.settings.setSetting("maxFileSizeMB", 100 / (1024 * 1024));
|
||||
const large = "PRIVATE".repeat(100);
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith(
|
||||
`${timing}:rename:a.bin->.vault-link-sync/transactions/`
|
||||
)
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userWrite("a.bin", bytes(large));
|
||||
await f.syncer.syncLocallyUpdatedFile({ relativePath: "a.bin" });
|
||||
};
|
||||
await f.files.apply(structuredClone(f.database.state), {
|
||||
a: {
|
||||
expected: await toStoredSnapshot({ content: bytes("PUBLIC") }),
|
||||
replacement: await toStoredSnapshot({
|
||||
content: bytes("REMOTE")
|
||||
})
|
||||
}
|
||||
});
|
||||
assert(injected);
|
||||
assert.equal(
|
||||
Buffer.from(f.disk.userFiles().get("a.bin")!).toString(),
|
||||
large
|
||||
);
|
||||
assert.equal(f.database.state.excluded?.a, "a.bin");
|
||||
assert.deepEqual(f.database.state.local, { a: "a.bin" });
|
||||
});
|
||||
|
||||
test("an excluded move after installation survives an uncertain journal acknowledgement", async () => {
|
||||
const f = await fixture({ "a.md": "PUBLIC" });
|
||||
await f.settings.setSetting("ignorePatterns", ["private.md"]);
|
||||
let moved = false,
|
||||
crashed = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
moved ||
|
||||
!label.startsWith("durable:rename:") ||
|
||||
!label.endsWith(".output->a.md")
|
||||
)
|
||||
return;
|
||||
moved = true;
|
||||
await f.disk.userRename("a.md", "private.md");
|
||||
await f.disk.userWrite("private.md", bytes("SECRET"));
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "private.md"
|
||||
});
|
||||
};
|
||||
f.persistence.boundary = (label) => {
|
||||
if (!moved || crashed || label !== "durable:save") return;
|
||||
crashed = true;
|
||||
throw new Error("lost journal acknowledgement");
|
||||
};
|
||||
await assert.rejects(
|
||||
f.files.apply(structuredClone(f.database.state), {
|
||||
a: {
|
||||
expected: await toStoredSnapshot({ content: bytes("PUBLIC") }),
|
||||
replacement: await toStoredSnapshot({
|
||||
content: bytes("PUBLIC REMOTE")
|
||||
})
|
||||
}
|
||||
}),
|
||||
/lost journal acknowledgement/
|
||||
);
|
||||
assert(moved && crashed);
|
||||
f.disk.boundary = () => {};
|
||||
f.persistence.boundary = () => {};
|
||||
await f.database.recoverPersistence();
|
||||
await f.files.recover();
|
||||
assert.deepEqual(f.database.state.local, { a: "private.md" });
|
||||
assert.equal(
|
||||
Buffer.from(f.disk.userFiles().get("private.md")!).toString(),
|
||||
"SECRET"
|
||||
);
|
||||
});
|
||||
|
||||
test("moving a tracked file into an ignored path during a journal leaves it protected", async () => {
|
||||
const f = await fixture({ "a.md": "PUBLIC", "b.md": "OTHER" });
|
||||
await f.settings.setSetting("ignorePatterns", ["private.md"]);
|
||||
const next = structuredClone(f.database.state);
|
||||
next.local.b = "remote-b.md";
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith("before:mkdir:.vault-link-sync/transactions/")
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userRename("a.md", "private.md");
|
||||
await f.disk.userWrite("private.md", bytes("SECRET"));
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "private.md"
|
||||
});
|
||||
};
|
||||
await f.files.apply(next);
|
||||
assert(injected);
|
||||
assert.equal(
|
||||
Buffer.from(f.disk.userFiles().get("private.md") ?? []).toString(),
|
||||
"SECRET"
|
||||
);
|
||||
assert.equal(f.database.state.local.a, "private.md");
|
||||
assert.equal(f.database.state.excluded?.a, "a.md");
|
||||
});
|
||||
|
||||
test("a directory move protects each ignored descendant during a journal", async () => {
|
||||
const f = await fixture({ "folder/a.md": "PUBLIC", "b.md": "OTHER" });
|
||||
await f.settings.setSetting("ignorePatterns", ["private/**"]);
|
||||
const next = structuredClone(f.database.state);
|
||||
next.local.b = "remote-b.md";
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith("before:mkdir:.vault-link-sync/transactions/")
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userRename("folder/a.md", "private/a.md");
|
||||
await f.disk.userWrite("private/a.md", bytes("SECRET"));
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: "folder",
|
||||
relativePath: "private"
|
||||
});
|
||||
};
|
||||
await f.files.apply(next);
|
||||
assert(injected);
|
||||
assert.equal(
|
||||
Buffer.from(f.disk.userFiles().get("private/a.md") ?? []).toString(),
|
||||
"SECRET"
|
||||
);
|
||||
assert.equal(f.database.state.local["folder/a"], "private/a.md");
|
||||
assert.equal(f.database.state.excluded?.["folder/a"], "folder/a.md");
|
||||
});
|
||||
|
||||
test("a later editor save replaces earlier unsynced text after a merge was prepared", async () => {
|
||||
const f = await fixture({ "a.md": "base" });
|
||||
const next = structuredClone(f.database.state);
|
||||
let saves = 0;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (saves === 0 && label.startsWith("before:rename:a.md->")) {
|
||||
saves++;
|
||||
await f.disk.userWrite("a.md", bytes("base ONE"));
|
||||
await f.syncer.syncLocallyUpdatedFile({ relativePath: "a.md" });
|
||||
} else if (
|
||||
saves === 1 &&
|
||||
label.startsWith("before:write:") &&
|
||||
label.endsWith(".output")
|
||||
) {
|
||||
saves++;
|
||||
await f.disk.userWrite("a.md", bytes("base TWO"));
|
||||
await f.syncer.syncLocallyUpdatedFile({ relativePath: "a.md" });
|
||||
}
|
||||
};
|
||||
await f.files.apply(next, {
|
||||
a: {
|
||||
expected: await toStoredSnapshot({ content: bytes("base") }),
|
||||
replacement: await toStoredSnapshot({
|
||||
content: bytes("base REMOTE")
|
||||
})
|
||||
}
|
||||
});
|
||||
assert.equal(saves, 2);
|
||||
assert.equal(
|
||||
Buffer.from(f.disk.userFiles().get("a.md")!).toString(),
|
||||
"base TWO REMOTE"
|
||||
);
|
||||
assert.deepEqual(f.database.state.local, { a: "a.md" });
|
||||
});
|
||||
|
||||
test("a content save during remote application remains on the same document", async () => {
|
||||
const f = await fixture({ "a.md": "base" });
|
||||
f.database.state.documents.a.base = {
|
||||
...head("a", 1, "base"),
|
||||
hash: f.database.state.documents.a.observedHash!
|
||||
};
|
||||
const next = structuredClone(f.database.state);
|
||||
next.documents.a.base = {
|
||||
...head("a", 2, "base REMOTE"),
|
||||
hash: (await toStoredSnapshot({ content: bytes("base REMOTE") })).hash
|
||||
};
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith(
|
||||
"durable:rename:a.md->.vault-link-sync/transactions/"
|
||||
)
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userWrite("a.md", bytes("base LOCAL"));
|
||||
await f.syncer.syncLocallyUpdatedFile({ relativePath: "a.md" });
|
||||
};
|
||||
await f.files.apply(next, {
|
||||
a: {
|
||||
expected: await toStoredSnapshot({ content: bytes("base") }),
|
||||
replacement: await toStoredSnapshot({
|
||||
content: bytes("base REMOTE")
|
||||
})
|
||||
}
|
||||
});
|
||||
|
||||
assert.equal(f.disk.userFiles().size, 1);
|
||||
const result = Buffer.from(
|
||||
(await f.disk.readSnapshot("a.md"))!.content
|
||||
).toString();
|
||||
assert(result.includes("LOCAL") && result.includes("REMOTE"));
|
||||
});
|
||||
|
||||
async function setup() {
|
||||
const f = await fixture({ "a.md": "ORIGINAL", "b.md": "OTHER" });
|
||||
f.database.state.documents.a.base = {
|
||||
...head("a", 1, "ORIGINAL"),
|
||||
hash: f.database.state.documents.a.observedHash!
|
||||
};
|
||||
const next = structuredClone(f.database.state);
|
||||
next.local = { a: "remote.md", b: "remote-b.md" };
|
||||
next.fileManifest = { fileManifestId: 3, entries: { ...next.local } };
|
||||
const expected = await toStoredSnapshot({ content: bytes("ORIGINAL") });
|
||||
const replacement = await toStoredSnapshot({
|
||||
content: bytes("ORIGINAL REMOTE")
|
||||
});
|
||||
next.documents.a.base = {
|
||||
...head("a", 2, "ORIGINAL REMOTE"),
|
||||
hash: replacement.hash
|
||||
};
|
||||
return { f, next, writes: { a: { expected, replacement } } };
|
||||
}
|
||||
test("editor changes preserve exact bytes and identities at every filesystem read, mutation and persistence boundary", async (t) => {
|
||||
const { f, next, writes } = await setup();
|
||||
const labels: string[] = [];
|
||||
const trace = (l: string) => {
|
||||
labels.push(l);
|
||||
};
|
||||
f.disk.boundary = trace;
|
||||
f.persistence.boundary = trace;
|
||||
await f.files.apply(next, writes);
|
||||
const failures = [];
|
||||
let exercised = 0;
|
||||
for (const kind of ["move", "move-recreate", "delete-recreate", "save"])
|
||||
for (let target = 0; target < labels.length; target++) {
|
||||
const { f, next, writes } = await setup();
|
||||
let count = 0,
|
||||
injected = false,
|
||||
from: string | undefined;
|
||||
const boundary = async (l: string) => {
|
||||
if (injected) return;
|
||||
if (count++ !== target) return;
|
||||
const files = f.disk.userFiles();
|
||||
from = files.has("a.md")
|
||||
? "a.md"
|
||||
: files.has("remote.md")
|
||||
? "remote.md"
|
||||
: undefined;
|
||||
if (kind === "save" && !from) from = "a.md";
|
||||
if (!from) return;
|
||||
injected = true;
|
||||
exercised++;
|
||||
if (kind === "save") {
|
||||
await f.disk.userWrite(
|
||||
from,
|
||||
bytes(
|
||||
Buffer.from(
|
||||
files.get(from) ?? bytes("ORIGINAL")
|
||||
).toString() + " LOCAL"
|
||||
)
|
||||
);
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
relativePath: from
|
||||
});
|
||||
} else if (kind === "delete-recreate") {
|
||||
await f.disk.userDelete(from);
|
||||
await f.syncer.syncLocallyDeletedFile(from);
|
||||
} else {
|
||||
await f.disk.userRename(from, "mine.md");
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
oldPath: from,
|
||||
relativePath: "mine.md"
|
||||
});
|
||||
}
|
||||
if (kind !== "move" && kind !== "save") {
|
||||
await f.disk.userWrite(from, bytes("REPLACEMENT"));
|
||||
await f.syncer.syncLocallyCreatedFile(from);
|
||||
}
|
||||
};
|
||||
f.disk.boundary = boundary;
|
||||
f.persistence.boundary = boundary;
|
||||
let error;
|
||||
try {
|
||||
await f.files.apply(next, writes);
|
||||
} catch (e) {
|
||||
error = String(e);
|
||||
}
|
||||
f.disk.boundary = () => {};
|
||||
f.persistence.boundary = () => {};
|
||||
try {
|
||||
await f.database.recoverPersistence();
|
||||
await f.files.recover();
|
||||
await f.internals.scan();
|
||||
} catch (e) {
|
||||
error += ";retry=" + String(e);
|
||||
}
|
||||
if (!injected) continue;
|
||||
const fs = Object.fromEntries(
|
||||
[...f.disk.userFiles()].map(([p, b]) => [
|
||||
p,
|
||||
Buffer.from(b).toString()
|
||||
])
|
||||
);
|
||||
const bad =
|
||||
kind === "save"
|
||||
? ![
|
||||
"ORIGINAL LOCAL REMOTE",
|
||||
"ORIGINAL REMOTE LOCAL"
|
||||
].includes(fs[f.database.state.local.a]) ||
|
||||
Object.keys(f.database.state.local).length !== 2
|
||||
: kind === "delete-recreate"
|
||||
? f.database.state.local.a !== undefined
|
||||
: f.database.state.local.a !== "mine.md" ||
|
||||
fs["mine.md"] !== "ORIGINAL REMOTE";
|
||||
if (
|
||||
bad ||
|
||||
(kind !== "move" &&
|
||||
kind !== "save" &&
|
||||
fs[from!] !== "REPLACEMENT")
|
||||
)
|
||||
failures.push({
|
||||
kind,
|
||||
target,
|
||||
label: labels[target],
|
||||
error,
|
||||
ids: f.database.state.local,
|
||||
files: fs
|
||||
});
|
||||
}
|
||||
assert(
|
||||
exercised > 50,
|
||||
"The sweep must actually exercise the injected actions"
|
||||
);
|
||||
t.diagnostic(
|
||||
`${labels.length} boundaries; ${exercised} editor interleavings`
|
||||
);
|
||||
assert.deepEqual(failures, []);
|
||||
});
|
||||
612
frontend/test-support/journal-protocol.test.ts
Normal file
612
frontend/test-support/journal-protocol.test.ts
Normal file
|
|
@ -0,0 +1,612 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { fixture, bytes, randomUUID } from "./sync-server-fixture";
|
||||
|
||||
for (const exclusion of ["size", "ignore"] as const)
|
||||
for (const phase of [
|
||||
"planned",
|
||||
...(exclusion === "size" ? ["staged", "prepared"] : []),
|
||||
"installed"
|
||||
])
|
||||
for (const recovery of [
|
||||
"none",
|
||||
"restart",
|
||||
"before:save",
|
||||
"durable:save"
|
||||
] as const)
|
||||
test(`reenabling ${exclusion} exclusion at ${phase} preserves the content base through ${recovery}`, async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const installed = phase === "installed";
|
||||
const id = randomUUID();
|
||||
const base = await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("BASE\n")
|
||||
);
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.client.setSettings({
|
||||
maxFileSizeMB:
|
||||
exclusion === "size" ? 100 / (1024 * 1024) : 1,
|
||||
ignorePatterns: ["private.md"]
|
||||
});
|
||||
const remote = await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("REMOTE\nBASE\n", base.vaultUpdateId)
|
||||
);
|
||||
const local = "BASE\n" + "LOCAL\n".repeat(30);
|
||||
const path = exclusion === "size" ? "a.md" : "private.md";
|
||||
let injected = false,
|
||||
armed = false,
|
||||
crashed = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
const atBoundary =
|
||||
phase === "installed"
|
||||
? label.startsWith("durable:rename:") &&
|
||||
label.endsWith(".output->a.md")
|
||||
: phase === "staged"
|
||||
? label.startsWith("durable:rename:a.md->")
|
||||
: phase === "prepared"
|
||||
? label.startsWith("before:write:") &&
|
||||
label.endsWith(".output")
|
||||
: label.startsWith(
|
||||
"before:mkdir:.vault-link-sync/transactions/"
|
||||
);
|
||||
if (injected || !atBoundary) return;
|
||||
injected = true;
|
||||
if (exclusion === "ignore")
|
||||
await f.disk.userRename("a.md", path);
|
||||
// When installation already completed, deleting REMOTE
|
||||
// is a subsequent local edit, which must not be undone.
|
||||
await f.disk.userWrite(path, bytes(local));
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
relativePath: path,
|
||||
...(exclusion === "ignore"
|
||||
? { oldPath: "a.md" }
|
||||
: {})
|
||||
});
|
||||
armed = true;
|
||||
};
|
||||
f.store.boundary = (label) => {
|
||||
if (!armed || crashed || label !== recovery) return;
|
||||
crashed = true;
|
||||
f.disk.crash(true);
|
||||
throw new Error("interrupted exclusion save");
|
||||
};
|
||||
if (recovery.endsWith(":save")) {
|
||||
await assert.rejects(
|
||||
f.client.setSetting("isSyncEnabled", true),
|
||||
/interrupted exclusion save/
|
||||
);
|
||||
assert(crashed);
|
||||
} else await f.client.setSetting("isSyncEnabled", true);
|
||||
assert(injected);
|
||||
f.disk.boundary = f.store.boundary = () => {};
|
||||
if (recovery !== "none") {
|
||||
await f.reopenClient();
|
||||
await f.client.start();
|
||||
}
|
||||
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot(path))?.content,
|
||||
bytes(local)
|
||||
);
|
||||
const stored = f.store.snapshot().database!;
|
||||
assert.equal(
|
||||
stored.documents![id].base?.vaultUpdateId,
|
||||
installed ? remote.vaultUpdateId : base.vaultUpdateId
|
||||
);
|
||||
assert.equal(
|
||||
stored.remoteHeads![id].vaultUpdateId,
|
||||
remote.vaultUpdateId
|
||||
);
|
||||
assert.equal(
|
||||
(await f.get("/documents/" + id)).contentBase64,
|
||||
Buffer.from("REMOTE\nBASE\n").toString("base64")
|
||||
);
|
||||
await f.client.setSettings({
|
||||
maxFileSizeMB: 1,
|
||||
ignorePatterns: []
|
||||
});
|
||||
const expected = installed ? local : "REMOTE\n" + local;
|
||||
const verify = async () => {
|
||||
assert.deepEqual(
|
||||
(await f.get("/file-manifest")).entries,
|
||||
{ [id]: path }
|
||||
);
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot(path))?.content,
|
||||
bytes(expected)
|
||||
);
|
||||
assert.equal(
|
||||
(await f.get("/documents/" + id)).contentBase64,
|
||||
Buffer.from(expected).toString("base64")
|
||||
);
|
||||
};
|
||||
await verify();
|
||||
await f.client.reset();
|
||||
await verify();
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
for (const at of [undefined, "before:save", "durable:save"])
|
||||
for (const powerLoss of at ? [false, true] : [false])
|
||||
test(`repeated saves retain only the latest local edit through ${at ?? "completion"} and powerLoss=${powerLoss}`, async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const id = randomUUID();
|
||||
const base = await f.put("/documents/" + id, f.content("base"));
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("base REMOTE", base.vaultUpdateId)
|
||||
);
|
||||
let saves = 0,
|
||||
armed = false,
|
||||
crashed = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
saves === 0 &&
|
||||
label.startsWith("before:rename:a.md->")
|
||||
) {
|
||||
saves++;
|
||||
await f.disk.userWrite("a.md", bytes("base ONE"));
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
relativePath: "a.md"
|
||||
});
|
||||
} else if (
|
||||
saves === 1 &&
|
||||
label.startsWith("before:write:") &&
|
||||
label.endsWith(".output")
|
||||
) {
|
||||
saves++;
|
||||
await f.disk.userWrite("a.md", bytes("base TWO"));
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
relativePath: "a.md"
|
||||
});
|
||||
armed = true;
|
||||
}
|
||||
};
|
||||
f.store.boundary = (label) => {
|
||||
if (!armed || crashed || label !== at) return;
|
||||
crashed = true;
|
||||
f.disk.crash(powerLoss);
|
||||
throw new Error("injected repeated-save crash");
|
||||
};
|
||||
if (at) {
|
||||
await assert.rejects(
|
||||
f.client.setSetting("isSyncEnabled", true),
|
||||
/injected repeated-save crash/
|
||||
);
|
||||
assert(crashed);
|
||||
f.disk.boundary = () => {};
|
||||
f.store.boundary = () => {};
|
||||
await f.reopenClient();
|
||||
await f.client.start();
|
||||
} else await f.client.setSetting("isSyncEnabled", true);
|
||||
assert.equal(saves, 2);
|
||||
const verify = async () => {
|
||||
assert.deepEqual((await f.get("/file-manifest")).entries, {
|
||||
[id]: "a.md"
|
||||
});
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot("a.md"))?.content,
|
||||
bytes("base TWO REMOTE")
|
||||
);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.get("/documents/" + id)).contentBase64,
|
||||
"base64"
|
||||
).toString(),
|
||||
"base TWO REMOTE"
|
||||
);
|
||||
};
|
||||
await verify();
|
||||
await f.client.reset();
|
||||
await verify();
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
for (const timing of ["before", "durable"])
|
||||
for (const crash of [false, true])
|
||||
test(`oversized save ${timing} staging stays local across crash=${crash}`, async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const id = randomUUID(),
|
||||
large = "PRIVATE".repeat(100);
|
||||
const base = await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("PUBLIC")
|
||||
);
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.bin" }
|
||||
});
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.client.setSetting("maxFileSizeMB", 100 / (1024 * 1024));
|
||||
await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("REMOTE", base.vaultUpdateId)
|
||||
);
|
||||
let injected = false,
|
||||
armed = false,
|
||||
crashed = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith(
|
||||
`${timing}:rename:a.bin->.vault-link-sync/transactions/`
|
||||
)
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userWrite("a.bin", bytes(large));
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
relativePath: "a.bin"
|
||||
});
|
||||
armed = true;
|
||||
};
|
||||
f.store.boundary = (label) => {
|
||||
if (!crash || !armed || crashed || label !== "durable:save")
|
||||
return;
|
||||
crashed = true;
|
||||
f.disk.crash(true);
|
||||
throw new Error("injected oversized-save crash");
|
||||
};
|
||||
if (crash) {
|
||||
await assert.rejects(
|
||||
f.client.setSetting("isSyncEnabled", true),
|
||||
/injected oversized-save crash/
|
||||
);
|
||||
assert(crashed);
|
||||
f.disk.boundary = () => {};
|
||||
f.store.boundary = () => {};
|
||||
await f.reopenClient();
|
||||
await f.client.start();
|
||||
} else await f.client.setSetting("isSyncEnabled", true);
|
||||
assert(injected);
|
||||
const verify = async () => {
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot("a.bin"))?.content,
|
||||
bytes(large)
|
||||
);
|
||||
assert.deepEqual((await f.get("/file-manifest")).entries, {
|
||||
[id]: "a.bin"
|
||||
});
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await f.get("/documents/" + id)).contentBase64,
|
||||
"base64"
|
||||
).toString(),
|
||||
"REMOTE"
|
||||
);
|
||||
};
|
||||
await verify();
|
||||
await f.client.reset();
|
||||
await verify();
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
test("a tracked file moved into an ignored path during remote application never uploads its new bytes", async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const a = randomUUID(),
|
||||
b = randomUUID();
|
||||
await f.put("/documents/" + a, f.content("PUBLIC"));
|
||||
await f.put("/documents/" + b, f.content("OTHER"));
|
||||
const m = await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [a]: "a.md", [b]: "b.md" }
|
||||
});
|
||||
await f.client.setSetting("ignorePatterns", ["private.md"]);
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: m.fileManifestId,
|
||||
entries: { [a]: "a.md", [b]: "remote-b.md" }
|
||||
});
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith("before:mkdir:.vault-link-sync/transactions/")
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userRename("a.md", "private.md");
|
||||
await f.disk.userWrite("private.md", bytes("SECRET"));
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "private.md"
|
||||
});
|
||||
};
|
||||
await f.client.setSetting("isSyncEnabled", true);
|
||||
assert(injected);
|
||||
const manifest = await f.get("/file-manifest");
|
||||
const contents = await Promise.all(
|
||||
Object.keys(manifest.entries).map((id) => f.get("/documents/" + id))
|
||||
);
|
||||
assert(
|
||||
!contents.some(
|
||||
(c) =>
|
||||
Buffer.from(c.contentBase64, "base64").toString() ===
|
||||
"SECRET"
|
||||
)
|
||||
);
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot("private.md"))?.content,
|
||||
bytes("SECRET")
|
||||
);
|
||||
assert.equal(manifest.entries[a], "a.md");
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
test("remote creation plus unrelated notification must not delete remote membership on retry", async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
const id = randomUUID();
|
||||
await f.put("/documents/" + id, f.content("REMOTE"));
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "remote.md" }
|
||||
});
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith("before:mkdir:.vault-link-sync/transactions/")
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userWrite("local.md", bytes("LOCAL"));
|
||||
await f.client.syncLocallyCreatedFile("local.md");
|
||||
};
|
||||
await f.client.setSetting("isSyncEnabled", true);
|
||||
assert(injected);
|
||||
f.disk.boundary = () => {};
|
||||
await f.client.reset();
|
||||
const manifest = await f.get("/file-manifest");
|
||||
|
||||
assert.equal(manifest.entries[id], "remote.md");
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
test("notified ignored occupant during remote rename must never upload its bytes", async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const id = randomUUID();
|
||||
await f.put("/documents/" + id, f.content("REMOTE"));
|
||||
const m = await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
await f.client.setSetting("ignorePatterns", ["private"]);
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: m.fileManifestId,
|
||||
entries: { [id]: "private/note.md" }
|
||||
});
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith("before:mkdir:.vault-link-sync/transactions/")
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userWrite("private", bytes("SECRET"));
|
||||
await f.client.syncLocallyCreatedFile("private");
|
||||
};
|
||||
await f.client.setSetting("isSyncEnabled", true);
|
||||
f.disk.boundary = () => {};
|
||||
const manifest = await f.get("/file-manifest");
|
||||
const contents = await Promise.all(
|
||||
Object.keys(manifest.entries).map((id) => f.get("/documents/" + id))
|
||||
);
|
||||
|
||||
assert(injected);
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot("private"))?.content,
|
||||
bytes("SECRET")
|
||||
);
|
||||
assert(
|
||||
!contents.some(
|
||||
(c) =>
|
||||
Buffer.from(c.contentBase64, "base64").toString() ===
|
||||
"SECRET"
|
||||
)
|
||||
);
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
test("notified delete/recreate before staging retains replacement file", async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const id = randomUUID();
|
||||
const h = await f.put("/documents/" + id, f.content("ORIGINAL"));
|
||||
const m = await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("ORIGINAL REMOTE", h.vaultUpdateId)
|
||||
);
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: m.fileManifestId,
|
||||
entries: { [id]: "remote.md" }
|
||||
});
|
||||
let injected = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith(
|
||||
"before:rename:a.md->.vault-link-sync/transactions/"
|
||||
)
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
await f.disk.userDelete("a.md");
|
||||
await f.client.syncLocallyDeletedFile("a.md");
|
||||
await f.disk.userWrite("a.md", bytes("REPLACEMENT"));
|
||||
await f.client.syncLocallyCreatedFile("a.md");
|
||||
};
|
||||
await f.client.setSetting("isSyncEnabled", true);
|
||||
f.disk.boundary = () => {};
|
||||
const manifest = await f.get("/file-manifest");
|
||||
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot("a.md"))?.content,
|
||||
bytes("REPLACEMENT")
|
||||
);
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
for (const mode of ["delete", "move", "save"] as const)
|
||||
for (const powerLoss of [false, true])
|
||||
for (const at of ["before:save", "durable:save"])
|
||||
test(`${mode} during staging survives ${at} and ${powerLoss ? "power loss" : "process replacement"}`, async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const id = randomUUID();
|
||||
const h = await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("ORIGINAL")
|
||||
);
|
||||
await f.put("/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
await f.client.start();
|
||||
await f.client.setSetting("isSyncEnabled", false);
|
||||
await f.put(
|
||||
"/documents/" + id,
|
||||
f.content("ORIGINAL REMOTE", h.vaultUpdateId)
|
||||
);
|
||||
let injected = false,
|
||||
armed = false,
|
||||
crashed = false;
|
||||
f.disk.boundary = async (label) => {
|
||||
const point = mode === "save" ? "durable" : "before";
|
||||
if (
|
||||
injected ||
|
||||
!label.startsWith(
|
||||
`${point}:rename:a.md->.vault-link-sync/transactions/`
|
||||
)
|
||||
)
|
||||
return;
|
||||
injected = true;
|
||||
if (mode === "save") {
|
||||
await f.disk.userWrite(
|
||||
"a.md",
|
||||
bytes("ORIGINAL LOCAL")
|
||||
);
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
relativePath: "a.md"
|
||||
});
|
||||
} else {
|
||||
if (mode === "delete") {
|
||||
await f.disk.userDelete("a.md");
|
||||
await f.client.syncLocallyDeletedFile("a.md");
|
||||
} else {
|
||||
await f.disk.userRename("a.md", "mine.md");
|
||||
await f.client.syncLocallyUpdatedFile({
|
||||
oldPath: "a.md",
|
||||
relativePath: "mine.md"
|
||||
});
|
||||
}
|
||||
await f.disk.userWrite(
|
||||
"a.md",
|
||||
bytes("REPLACEMENT")
|
||||
);
|
||||
await f.client.syncLocallyCreatedFile("a.md");
|
||||
}
|
||||
armed = true;
|
||||
};
|
||||
f.store.boundary = (label) => {
|
||||
if (!armed || crashed || label !== at) return;
|
||||
crashed = true;
|
||||
f.disk.crash(powerLoss);
|
||||
throw new Error("injected journal crash");
|
||||
};
|
||||
await assert.rejects(
|
||||
f.client.setSetting("isSyncEnabled", true),
|
||||
/injected journal crash/
|
||||
);
|
||||
assert(injected && crashed);
|
||||
f.disk.boundary = () => {};
|
||||
f.store.boundary = () => {};
|
||||
await f.reopenClient();
|
||||
await f.client.start();
|
||||
const manifest = await f.get("/file-manifest");
|
||||
if (mode === "save") {
|
||||
assert.deepEqual(manifest.entries, { [id]: "a.md" });
|
||||
const actual = Buffer.from(
|
||||
(await f.disk.readSnapshot("a.md"))!.content
|
||||
).toString();
|
||||
assert.equal(actual, "ORIGINAL LOCAL REMOTE");
|
||||
} else {
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot("a.md"))?.content,
|
||||
bytes("REPLACEMENT")
|
||||
);
|
||||
assert.notEqual(
|
||||
Object.keys(manifest.entries).find(
|
||||
(key) => manifest.entries[key] === "a.md"
|
||||
),
|
||||
id
|
||||
);
|
||||
if (mode === "move") {
|
||||
assert.equal(manifest.entries[id], "mine.md");
|
||||
assert.deepEqual(
|
||||
(await f.disk.readSnapshot("mine.md"))?.content,
|
||||
bytes("ORIGINAL REMOTE")
|
||||
);
|
||||
} else assert.equal(manifest.entries[id], undefined);
|
||||
}
|
||||
await f.client.reset();
|
||||
assert.deepEqual(
|
||||
(await f.get("/file-manifest")).entries,
|
||||
manifest.entries
|
||||
);
|
||||
} finally {
|
||||
await f.dispose();
|
||||
}
|
||||
});
|
||||
30
frontend/test-support/listener-crash-child.ts
Normal file
30
frontend/test-support/listener-crash-child.ts
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { fixture } from "./sync-fixture";
|
||||
(async () => {
|
||||
const f = await fixture(
|
||||
{},
|
||||
{
|
||||
events: async () => ({ headEventId: 0, events: [] })
|
||||
}
|
||||
);
|
||||
let delivered = 0;
|
||||
f.syncer.onRemainingOperationsCountChanged.add(() => {
|
||||
throw new Error("observer failure");
|
||||
});
|
||||
f.syncer.onRemainingOperationsCountChanged.add(async () => {
|
||||
throw new Error("async observer failure");
|
||||
});
|
||||
f.syncer.onRemainingOperationsCountChanged.add(() => {
|
||||
delivered++;
|
||||
});
|
||||
try {
|
||||
f.syncer.start();
|
||||
await f.syncer.waitUntilFinished();
|
||||
const before = delivered;
|
||||
f.syncer.wake();
|
||||
await f.syncer.waitUntilFinished();
|
||||
assert(delivered > before, "Background sync must remain usable");
|
||||
} finally {
|
||||
await f.syncer.stop();
|
||||
}
|
||||
})();
|
||||
27
frontend/test-support/native-fs.c
Normal file
27
frontend/test-support/native-fs.c
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
/* Test-only POSIX primitives missing from Node's filesystem API. A preflight
|
||||
* exists() followed by fs.rename() is not an exclusive rename. */
|
||||
#define _GNU_SOURCE
|
||||
#include <fcntl.h>
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <unistd.h>
|
||||
#if defined(__linux__)
|
||||
#include <sys/syscall.h>
|
||||
#endif
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
if (argc == 4 && strcmp(argv[1], "rename") == 0) {
|
||||
int result;
|
||||
#if defined(__APPLE__)
|
||||
result = renamex_np(argv[2], argv[3], RENAME_EXCL);
|
||||
#elif defined(__linux__)
|
||||
result = syscall(SYS_renameat2, AT_FDCWD, argv[2], AT_FDCWD, argv[3], 1 /* RENAME_NOREPLACE */);
|
||||
#else
|
||||
#error "The real filesystem tests require macOS or Linux"
|
||||
#endif
|
||||
if (result != 0) { perror("exclusive rename"); return 1; }
|
||||
return 0;
|
||||
}
|
||||
fprintf(stderr, "usage: native-fs rename FROM TO\n");
|
||||
return 2;
|
||||
}
|
||||
243
frontend/test-support/network.ts
Normal file
243
frontend/test-support/network.ts
Normal file
|
|
@ -0,0 +1,243 @@
|
|||
import assert from "node:assert/strict";
|
||||
|
||||
export type RequestKind = "create" | "content" | "manifest";
|
||||
export interface RequestRecord {
|
||||
kind: RequestKind;
|
||||
method: string;
|
||||
url: string;
|
||||
body: Record<string, unknown>;
|
||||
status?: number;
|
||||
}
|
||||
|
||||
interface InjectedFault {
|
||||
kind: RequestKind;
|
||||
point: "before" | "after";
|
||||
fired: boolean;
|
||||
interrupted?: RequestRecord;
|
||||
retried: boolean;
|
||||
reached: Promise<void>;
|
||||
resolve: () => void;
|
||||
}
|
||||
|
||||
export function classifyRequest(
|
||||
input: RequestInfo | URL,
|
||||
init?: RequestInit
|
||||
): RequestRecord | undefined {
|
||||
const url = new URL(input instanceof Request ? input.url : String(input));
|
||||
const method = (
|
||||
init?.method ?? (input instanceof Request ? input.method : "GET")
|
||||
).toUpperCase();
|
||||
if (method !== "PUT") return undefined;
|
||||
// The engine passes JSON strings. Fail loudly if that contract changes.
|
||||
if (!/\/(documents\/[^/]+|file-manifest)\/?$/.test(url.pathname))
|
||||
return undefined;
|
||||
assert.equal(
|
||||
typeof init?.body,
|
||||
"string",
|
||||
"CAS request body must be observable JSON"
|
||||
);
|
||||
const body = JSON.parse(init!.body as string) as Record<string, unknown>;
|
||||
const kind = /\/file-manifest\/?$/.test(url.pathname)
|
||||
? "manifest"
|
||||
: body.parentVersionId === null
|
||||
? "create"
|
||||
: "content";
|
||||
assert.equal(
|
||||
typeof body.requestId,
|
||||
"string",
|
||||
"CAS request missing requestId"
|
||||
);
|
||||
return { kind, method, url: url.href, body };
|
||||
}
|
||||
|
||||
export class NetworkFaults {
|
||||
public readonly requests: RequestRecord[] = [];
|
||||
private observation?: {
|
||||
released: boolean;
|
||||
fired: boolean;
|
||||
held: Promise<void>;
|
||||
release: () => void;
|
||||
reached: Promise<void>;
|
||||
reach: () => void;
|
||||
};
|
||||
public pauseObservation(): void {
|
||||
assert(
|
||||
!this.observation || this.observation.released,
|
||||
"Observation already paused"
|
||||
);
|
||||
let release!: () => void, reach!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
const reached = new Promise<void>((resolve) => {
|
||||
reach = resolve;
|
||||
});
|
||||
this.observation = {
|
||||
released: false,
|
||||
fired: false,
|
||||
held,
|
||||
release,
|
||||
reached,
|
||||
reach
|
||||
};
|
||||
}
|
||||
public async waitForObservation(): Promise<void> {
|
||||
assert(this.observation, "No observation checkpoint armed");
|
||||
await this.observation.reached;
|
||||
}
|
||||
public resumeObservation(): void {
|
||||
if (this.observation) {
|
||||
this.observation.released = true;
|
||||
this.observation.release();
|
||||
}
|
||||
}
|
||||
private async observationCheckpoint(
|
||||
signal?: AbortSignal | null
|
||||
): Promise<void> {
|
||||
const gate = this.observation;
|
||||
if (!gate || gate.released) return;
|
||||
gate.fired = true;
|
||||
gate.reach();
|
||||
signal?.throwIfAborted();
|
||||
let abort = () => {};
|
||||
try {
|
||||
await Promise.race([
|
||||
gate.held,
|
||||
new Promise<never>((_, reject) => {
|
||||
abort = () =>
|
||||
reject(
|
||||
signal?.reason ?? new Error("Observation aborted")
|
||||
);
|
||||
signal?.addEventListener("abort", abort, { once: true });
|
||||
})
|
||||
]);
|
||||
} finally {
|
||||
signal?.removeEventListener("abort", abort);
|
||||
}
|
||||
}
|
||||
private armed?: InjectedFault;
|
||||
private readonly faults: InjectedFault[] = [];
|
||||
public isExpectedFailure(message: string): boolean {
|
||||
return this.faults.some(
|
||||
(fault) =>
|
||||
fault.fired &&
|
||||
message.includes(
|
||||
`Injected ${fault.point}-commit ${fault.kind} network failure`
|
||||
)
|
||||
);
|
||||
}
|
||||
public arm(kind: RequestKind, point: "before" | "after" = "after"): void {
|
||||
assert(
|
||||
!this.armed || (this.armed.fired && this.armed.retried),
|
||||
"Previous network fault was not exercised and retried"
|
||||
);
|
||||
let resolve!: () => void;
|
||||
const reached = new Promise<void>((r) => {
|
||||
resolve = r;
|
||||
});
|
||||
this.armed = {
|
||||
kind,
|
||||
point,
|
||||
fired: false,
|
||||
retried: false,
|
||||
reached,
|
||||
resolve
|
||||
};
|
||||
this.faults.push(this.armed);
|
||||
}
|
||||
public async wait(): Promise<void> {
|
||||
assert(this.armed, "No network fault armed");
|
||||
await this.armed.reached;
|
||||
}
|
||||
public assertConsumed(): void {
|
||||
assert(
|
||||
!this.observation ||
|
||||
(this.observation.fired && this.observation.released),
|
||||
"Observation checkpoint was not exercised and released"
|
||||
);
|
||||
for (const fault of this.faults) {
|
||||
assert(fault.fired, "Armed network fault never fired");
|
||||
assert(fault.retried, "Interrupted CAS request was never retried");
|
||||
}
|
||||
}
|
||||
public wrap(fetch: typeof globalThis.fetch): typeof globalThis.fetch {
|
||||
return async (input, init) => {
|
||||
const url = new URL(
|
||||
input instanceof Request ? input.url : String(input)
|
||||
);
|
||||
const observing =
|
||||
(
|
||||
init?.method ??
|
||||
(input instanceof Request ? input.method : "GET")
|
||||
).toUpperCase() === "GET" && !url.pathname.endsWith("/ping");
|
||||
const signal =
|
||||
init?.signal ??
|
||||
(input instanceof Request ? input.signal : undefined);
|
||||
if (observing) await this.observationCheckpoint(signal);
|
||||
const record = classifyRequest(input, init);
|
||||
if (record) {
|
||||
// A client serializes its persisted CAS requests. Its next CAS
|
||||
// after a lost response must replay the interrupted operation,
|
||||
// even if it has since observed newer remote state.
|
||||
const interrupted = this.armed?.interrupted;
|
||||
if (interrupted && !this.armed!.retried) {
|
||||
assert.equal(
|
||||
record.url,
|
||||
interrupted.url,
|
||||
"Retry changed target"
|
||||
);
|
||||
assert.deepEqual(
|
||||
record.body,
|
||||
interrupted.body,
|
||||
"Retry changed payload/request identity"
|
||||
);
|
||||
this.armed!.retried = true;
|
||||
}
|
||||
const previous = this.requests.find(
|
||||
(r) => r.body.requestId === record.body.requestId
|
||||
);
|
||||
if (previous) {
|
||||
assert.equal(
|
||||
record.url,
|
||||
previous.url,
|
||||
"Retry changed target"
|
||||
);
|
||||
assert.deepEqual(
|
||||
record.body,
|
||||
previous.body,
|
||||
"Retry changed payload/request identity"
|
||||
);
|
||||
}
|
||||
this.requests.push(record);
|
||||
}
|
||||
const fault = this.armed;
|
||||
const matches =
|
||||
record && fault && !fault.fired && fault.kind === record.kind;
|
||||
const fail = () => {
|
||||
fault!.interrupted = structuredClone(record!);
|
||||
fault!.fired = true;
|
||||
fault!.resolve();
|
||||
throw new TypeError(
|
||||
`Injected ${fault!.point}-commit ${fault!.kind} network failure`
|
||||
);
|
||||
};
|
||||
if (matches && fault.point === "before") fail();
|
||||
const response = await fetch(input, init);
|
||||
if (observing) await this.observationCheckpoint(signal);
|
||||
if (record) record.status = response.status;
|
||||
// Only drop accepted replies, never confuse a rejected CAS with commit.
|
||||
if (
|
||||
matches &&
|
||||
!fault.fired &&
|
||||
fault.point === "after" &&
|
||||
response.ok &&
|
||||
((await response.clone().json()) as { type?: string }).type ===
|
||||
"Accepted"
|
||||
) {
|
||||
await response.body?.cancel();
|
||||
fail();
|
||||
}
|
||||
return response;
|
||||
};
|
||||
}
|
||||
}
|
||||
15
frontend/test-support/observer.test.ts
Normal file
15
frontend/test-support/observer.test.ts
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { resolve } from "node:path";
|
||||
test("a status observer exception must not terminate a background sync process", () => {
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
resolve(__dirname, "../node_modules/tsx/dist/cli.mjs"),
|
||||
resolve(__dirname, "listener-crash-child.ts")
|
||||
],
|
||||
{ encoding: "utf8", timeout: 10000 }
|
||||
);
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
});
|
||||
351
frontend/test-support/oracle-regressions.test.ts
Normal file
351
frontend/test-support/oracle-regressions.test.ts
Normal file
|
|
@ -0,0 +1,351 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { test } from "node:test";
|
||||
import type { StoredDatabase } from "sync-client";
|
||||
import { assertCanonical } from "./canonical";
|
||||
import { assertManifest } from "./oracles";
|
||||
import { NetworkFaults } from "./network";
|
||||
import { MemoryDisk } from "./storage";
|
||||
import { DeterministicAgent } from "../deterministic-tests/src/deterministic-agent";
|
||||
import { AssertableState } from "../deterministic-tests/src/utils/assertable-state";
|
||||
import type { ExpectedDocument } from "../deterministic-tests/src/test-definition";
|
||||
|
||||
const bytes = (text: string) => new TextEncoder().encode(text);
|
||||
const id = "00000000-0000-4000-8000-000000000001";
|
||||
|
||||
test("canonical checks reject repeated clients before reading snapshots", async () => {
|
||||
const client = { files: () => new Map(), database: () => ({}) };
|
||||
await assert.rejects(
|
||||
assertCanonical([client, client], "http://unused", "unused"),
|
||||
/itself/
|
||||
);
|
||||
});
|
||||
|
||||
test("canonical checks independently verify both persisted content hashes", async (context) => {
|
||||
const content = bytes("\ufeffexact bytes\r\n");
|
||||
const hash = createHash("sha256").update(content).digest("hex");
|
||||
const document = {
|
||||
documentId: id,
|
||||
vaultUpdateId: 1,
|
||||
contentSize: content.length,
|
||||
userId: "user",
|
||||
deviceId: "device",
|
||||
updatedDate: "2026-09-20T00:00:00Z"
|
||||
};
|
||||
const manifest = { fileManifestId: 2, entries: { [id]: "a.md" } };
|
||||
const state: Partial<StoredDatabase> = {
|
||||
initialized: true,
|
||||
local: manifest.entries,
|
||||
fileManifest: manifest,
|
||||
lastSeenUpdateId: 2,
|
||||
documents: {
|
||||
[id]: {
|
||||
materialized: true,
|
||||
observedHash: hash,
|
||||
base: { ...document, hash }
|
||||
}
|
||||
}
|
||||
};
|
||||
context.mock.method(
|
||||
globalThis,
|
||||
"fetch",
|
||||
async (url: string) =>
|
||||
new Response(
|
||||
JSON.stringify(
|
||||
url.endsWith("/vault-snapshot")
|
||||
? {
|
||||
headEventId: 2,
|
||||
fileManifest: manifest,
|
||||
documents: [document]
|
||||
}
|
||||
: url.includes("/events-since")
|
||||
? {
|
||||
headEventId: 2,
|
||||
events: [
|
||||
{ eventId: 1, requestId: "create" },
|
||||
{ eventId: 2, requestId: "manifest" }
|
||||
]
|
||||
}
|
||||
: {
|
||||
contentBase64:
|
||||
Buffer.from(content).toString("base64")
|
||||
}
|
||||
)
|
||||
)
|
||||
);
|
||||
const client = (database = state) => ({
|
||||
files: () => new Map([["a.md", content.slice()]]),
|
||||
database: () => structuredClone(database)
|
||||
});
|
||||
await assertCanonical([client(), client()], "http://test", "token");
|
||||
for (const field of ["base", "observed"] as const) {
|
||||
const corrupt = structuredClone(state);
|
||||
if (field === "base") corrupt.documents![id].base!.hash = "wrong";
|
||||
else corrupt.documents![id].observedHash = "wrong";
|
||||
await assert.rejects(
|
||||
assertCanonical(
|
||||
[client(), client(corrupt)],
|
||||
"http://test",
|
||||
"token"
|
||||
),
|
||||
new RegExp(`Wrong ${field} hash`)
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("lost responses require an identical retry, including request identity and target", async () => {
|
||||
for (const point of ["before", "after"] as const) {
|
||||
const faults = new NetworkFaults();
|
||||
let sent = 0;
|
||||
const send = faults.wrap(async () => {
|
||||
sent++;
|
||||
return new Response('{"type":"Accepted"}');
|
||||
});
|
||||
const url = "http://test/documents/a";
|
||||
const body = {
|
||||
requestId: "original",
|
||||
parentVersionId: 3,
|
||||
content: { type: "Snapshot", value: "YQ==" }
|
||||
};
|
||||
const init = { method: "PUT", body: JSON.stringify(body) };
|
||||
faults.arm("content", point);
|
||||
await assert.rejects(send(url, init), /Injected/);
|
||||
assert.throws(() => faults.assertConsumed(), /never retried/);
|
||||
assert.throws(
|
||||
() => faults.arm("manifest"),
|
||||
/not exercised and retried/
|
||||
);
|
||||
const before = sent;
|
||||
await assert.rejects(
|
||||
send(url, {
|
||||
...init,
|
||||
body: JSON.stringify({ ...body, requestId: "regenerated" })
|
||||
}),
|
||||
/Retry changed/
|
||||
);
|
||||
await assert.rejects(
|
||||
send(`${url}-different`, init),
|
||||
/Retry changed target/
|
||||
);
|
||||
await assert.rejects(
|
||||
send(url, {
|
||||
...init,
|
||||
body: JSON.stringify({ ...body, parentVersionId: 4 })
|
||||
}),
|
||||
/Retry changed/
|
||||
);
|
||||
assert.equal(sent, before, "Invalid retries must not reach the server");
|
||||
assert.throws(() => faults.assertConsumed(), /never retried/);
|
||||
await send(url, init);
|
||||
faults.assertConsumed();
|
||||
}
|
||||
});
|
||||
|
||||
test("an update fault cannot be consumed by creating a new UUID", async () => {
|
||||
const faults = new NetworkFaults();
|
||||
const send = faults.wrap(async () => new Response('{"type":"Accepted"}'));
|
||||
faults.arm("content");
|
||||
await send("http://test/documents/new", {
|
||||
method: "PUT",
|
||||
body: '{"requestId":"new","parentVersionId":null}'
|
||||
});
|
||||
assert.throws(() => faults.assertConsumed(), /never fired/);
|
||||
const update = {
|
||||
method: "PUT",
|
||||
body: '{"requestId":"update","parentVersionId":1}'
|
||||
};
|
||||
await assert.rejects(
|
||||
send("http://test/documents/existing", update),
|
||||
/Injected/
|
||||
);
|
||||
await send("http://test/documents/existing", update);
|
||||
faults.assertConsumed();
|
||||
});
|
||||
|
||||
test("unlink is idempotent even after an uncertain visible deletion", async () => {
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite("a", bytes("A"));
|
||||
disk.boundary = (label) => {
|
||||
if (label === "visible:unlink:a") throw new Error("uncertain unlink");
|
||||
};
|
||||
await assert.rejects(disk.deleteFile("a"), /uncertain/);
|
||||
disk.boundary = () => {};
|
||||
await disk.deleteFile("a");
|
||||
disk.crash(true);
|
||||
assert.equal(await disk.exists("a"), false);
|
||||
await disk.deleteFile("a");
|
||||
await disk.createDirectory("directory");
|
||||
await assert.rejects(disk.deleteFile("directory"), /directory/);
|
||||
});
|
||||
|
||||
test("text assertions preserve BOM, CRLF, empty content and missing final newline", async () => {
|
||||
const agent = new DeterministicAgent(0, {}, () => {});
|
||||
for (const content of [
|
||||
"",
|
||||
"\ufeffhello",
|
||||
"\ufeffa\r\nb\r\n",
|
||||
"last line"
|
||||
]) {
|
||||
await agent.disk.userWrite("a.md", bytes(content));
|
||||
assert.equal(await agent.getFileContent("a.md"), content);
|
||||
}
|
||||
});
|
||||
|
||||
test("durably pruning a directory also removes its already-unlinked durable children", async () => {
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite("transaction/source", bytes("staged bytes"));
|
||||
disk.boundary = (label) => {
|
||||
if (label === "visible:unlink:transaction/source") {
|
||||
disk.crash(false);
|
||||
throw new Error("process died after unlink");
|
||||
}
|
||||
};
|
||||
await assert.rejects(disk.deleteFile("transaction/source"), /process died/);
|
||||
disk.boundary = () => {};
|
||||
await disk.delete("transaction");
|
||||
disk.crash(true);
|
||||
assert.deepEqual(disk.userFiles(), new Map());
|
||||
assert.equal(await disk.exists("transaction"), false);
|
||||
});
|
||||
|
||||
test("portable components are limited by UTF-8 bytes, not characters or total path length", () => {
|
||||
for (const component of ["a".repeat(255), "é".repeat(127) + "a"])
|
||||
assertManifest({ [id]: `${component}/${component}/${component}` });
|
||||
for (const component of ["a".repeat(256), "é".repeat(128)])
|
||||
assert.throws(
|
||||
() => assertManifest({ [id]: `nested/${component}` }),
|
||||
/Oversized component/
|
||||
);
|
||||
});
|
||||
|
||||
function state(documents: { path: string; content: string; id: string }[]) {
|
||||
return new AssertableState({
|
||||
files: new Map(documents.map((doc) => [doc.path, doc.content])),
|
||||
clientFiles: [],
|
||||
manifests: [],
|
||||
canonical: Object.fromEntries(
|
||||
documents.map((doc) => [doc.id, doc.path])
|
||||
),
|
||||
bytes: new Map(documents.map((doc) => [doc.path, bytes(doc.content)]))
|
||||
});
|
||||
}
|
||||
|
||||
test("complete scenario assertions reject collapsed creates, forgotten edits, reverted moves and recycled UUIDs", () => {
|
||||
const identities = new Map<string, string>();
|
||||
const expected: ExpectedDocument[] = [
|
||||
{ key: "a", path: "a.md", content: "A\nimportant edit" },
|
||||
{ key: "b", path: "b.md", content: "B" }
|
||||
];
|
||||
const original = [
|
||||
{ id: "id-a", path: "a.md", content: "A\nimportant edit" },
|
||||
{ id: "id-b", path: "b.md", content: "B" }
|
||||
];
|
||||
state(original).assertDocuments(expected, identities);
|
||||
assert.throws(
|
||||
() =>
|
||||
state([
|
||||
{ ...original[0], content: "A\nimportant edit\nB" }
|
||||
]).assertDocuments(expected, identities),
|
||||
/Expected 2 file/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
state([
|
||||
{ ...original[0], content: "A" },
|
||||
original[1]
|
||||
]).assertDocuments(expected, identities),
|
||||
/Byte mismatch/
|
||||
);
|
||||
const moved = [
|
||||
{ ...expected[0], path: "b.md" },
|
||||
{ ...expected[1], path: "a.md" }
|
||||
];
|
||||
assert.throws(
|
||||
() => state(original).assertDocuments(moved, identities),
|
||||
/Byte mismatch/
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
state([
|
||||
{ ...original[0], id: "id-b" },
|
||||
{ ...original[1], id: "id-a" }
|
||||
]).assertDocuments(expected, identities),
|
||||
/identity changed/
|
||||
);
|
||||
state([original[1]]).assertDocuments([expected[1]], identities);
|
||||
assert.throws(
|
||||
() =>
|
||||
state(original).assertDocuments(
|
||||
[{ ...expected[0], key: "recreated" }, expected[1]],
|
||||
identities
|
||||
),
|
||||
/reused.*deleted UUID/
|
||||
);
|
||||
assert.equal(identities.has("recreated"), false);
|
||||
const conflict = [
|
||||
{ key: "one", path: "same.md", content: "one" },
|
||||
{ key: "two", path: "same.md", conflict: true, content: "two" }
|
||||
];
|
||||
state([
|
||||
{ id: "one", path: "same.md", content: "one" },
|
||||
{ id: "two", path: "same (conflict two).md", content: "two" }
|
||||
]).assertDocuments(conflict, new Map());
|
||||
});
|
||||
|
||||
test("canonical checks follow pages without weakening gap or duplicate-request detection", async (context) => {
|
||||
const manifest = { fileManifestId: 3, entries: {} };
|
||||
const client = () => ({
|
||||
files: () => new Map<string, Uint8Array>(),
|
||||
database: () => ({
|
||||
initialized: true,
|
||||
local: {},
|
||||
documents: {},
|
||||
fileManifest: manifest,
|
||||
lastSeenUpdateId: 3
|
||||
})
|
||||
});
|
||||
let fault = "none";
|
||||
context.mock.method(globalThis, "fetch", async (input: string) => {
|
||||
if (input.endsWith("/vault-snapshot"))
|
||||
return Response.json({
|
||||
headEventId: 3,
|
||||
fileManifest: manifest,
|
||||
documents: []
|
||||
});
|
||||
const after = Number(new URL(input).searchParams.get("after"));
|
||||
if (after === 0)
|
||||
return Response.json({
|
||||
headEventId: 3,
|
||||
endEventId: 1,
|
||||
events: [{ eventId: 1, requestId: "first" }]
|
||||
});
|
||||
assert.equal(after, 1);
|
||||
return Response.json({
|
||||
headEventId: 3,
|
||||
endEventId: fault === "empty" ? 1 : 3,
|
||||
events:
|
||||
fault === "empty"
|
||||
? []
|
||||
: [
|
||||
...(fault === "gap"
|
||||
? []
|
||||
: [
|
||||
{
|
||||
eventId: 2,
|
||||
requestId:
|
||||
fault === "duplicate"
|
||||
? "first"
|
||||
: "second"
|
||||
}
|
||||
]),
|
||||
{ eventId: 3, requestId: "third" }
|
||||
]
|
||||
});
|
||||
});
|
||||
await assertCanonical([client(), client()], "http://test", "token");
|
||||
for (fault of ["gap", "duplicate", "empty"])
|
||||
await assert.rejects(
|
||||
assertCanonical([client(), client()], "http://test", "token"),
|
||||
/contiguous|multiple events|progress/
|
||||
);
|
||||
});
|
||||
173
frontend/test-support/oracles.ts
Normal file
173
frontend/test-support/oracles.ts
Normal file
|
|
@ -0,0 +1,173 @@
|
|||
import assert from "node:assert/strict";
|
||||
import type { StoredDatabase } from "sync-client";
|
||||
|
||||
export function assertSameFiles(
|
||||
clients: readonly Map<string, Uint8Array>[]
|
||||
): void {
|
||||
assert(
|
||||
clients.length >= 2,
|
||||
"Consistency requires at least two distinct clients"
|
||||
);
|
||||
assert.equal(
|
||||
new Set(clients).size,
|
||||
clients.length,
|
||||
"Cannot compare a client to itself"
|
||||
);
|
||||
const sorted = (files: Map<string, Uint8Array>) =>
|
||||
[...files].sort(([a], [b]) => a.localeCompare(b));
|
||||
for (let i = 1; i < clients.length; i++)
|
||||
assert.deepEqual(
|
||||
sorted(clients[i]),
|
||||
sorted(clients[0]),
|
||||
`Client ${i} differs from client 0 (paths or bytes)`
|
||||
);
|
||||
}
|
||||
|
||||
export function assertManifest(entries: Record<string, string>): void {
|
||||
const paths = Object.values(entries);
|
||||
const folded = paths.map((p) =>
|
||||
p.normalize("NFC").toUpperCase().normalize("NFC")
|
||||
);
|
||||
const directorySpellings = new Map<string, string>();
|
||||
for (const path of paths) {
|
||||
assert.equal(
|
||||
path,
|
||||
path.normalize("NFC"),
|
||||
`Noncanonical Unicode path: ${path}`
|
||||
);
|
||||
const parts = path.split("/");
|
||||
parts.forEach((part, i) => {
|
||||
assert(
|
||||
Buffer.byteLength(part, "utf8") <= 255,
|
||||
`Oversized component: ${part}`
|
||||
);
|
||||
assert(
|
||||
!/[<>:"\\|?*\p{Cc}]|[. ]$/u.test(part),
|
||||
`Nonportable component: ${part}`
|
||||
);
|
||||
assert(
|
||||
!/^(CON|PRN|AUX|NUL|CONIN\$|CONOUT\$|COM[1-9¹²³]|LPT[1-9¹²³])$/u.test(
|
||||
part.split(".")[0].trimEnd().toUpperCase()
|
||||
),
|
||||
`Reserved component: ${part}`
|
||||
);
|
||||
if (i < parts.length - 1) {
|
||||
const prefix = parts.slice(0, i + 1).join("/");
|
||||
const key = prefix.toUpperCase().normalize("NFC");
|
||||
const previous = directorySpellings.get(key);
|
||||
assert(
|
||||
previous === undefined || previous === prefix,
|
||||
`Aliased directory spellings: ${prefix} and ${previous}`
|
||||
);
|
||||
directorySpellings.set(key, prefix);
|
||||
}
|
||||
});
|
||||
}
|
||||
assert.equal(
|
||||
new Set(folded).size,
|
||||
paths.length,
|
||||
"Manifest contains aliased/duplicate paths"
|
||||
);
|
||||
for (const path of folded) {
|
||||
assert(
|
||||
path && !path.startsWith("/") && !path.includes("\\"),
|
||||
`Invalid path ${path}`
|
||||
);
|
||||
assert(
|
||||
path.split("/").every((p) => p && p !== "." && p !== ".."),
|
||||
`Invalid component ${path}`
|
||||
);
|
||||
assert(
|
||||
path.split("/")[0] !== ".VAULT-LINK-SYNC",
|
||||
"Internal path in manifest"
|
||||
);
|
||||
assert(
|
||||
!folded.some(
|
||||
(other) => other !== path && other.startsWith(`${path}/`)
|
||||
),
|
||||
`File/ancestor conflict: ${path}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** Independent per-ID policy oracle, intentionally not importing merge code. */
|
||||
export function pathDecision(
|
||||
base: string | undefined,
|
||||
local: string | undefined,
|
||||
remote: string | undefined
|
||||
): string | undefined {
|
||||
if (local === base) return remote;
|
||||
if (remote === base) return local;
|
||||
return remote;
|
||||
}
|
||||
|
||||
export function assertQuiescent(
|
||||
state: Partial<StoredDatabase>,
|
||||
entries: Record<string, string>,
|
||||
head: number
|
||||
): void {
|
||||
assert.equal(
|
||||
state.pending,
|
||||
undefined,
|
||||
"Pending CAS request at convergence"
|
||||
);
|
||||
assert.equal(
|
||||
state.eventReplay,
|
||||
undefined,
|
||||
"Unfinished event replay at convergence"
|
||||
);
|
||||
assert.equal(
|
||||
state.application,
|
||||
undefined,
|
||||
"Unfinished filesystem journal at convergence"
|
||||
);
|
||||
assert.deepEqual(
|
||||
state.local,
|
||||
entries,
|
||||
"Local UUID/path map differs from server"
|
||||
);
|
||||
assert.deepEqual(
|
||||
state.fileManifest?.entries,
|
||||
entries,
|
||||
"Client manifest differs from server"
|
||||
);
|
||||
assert.equal(
|
||||
state.lastSeenUpdateId,
|
||||
head,
|
||||
"Client event watermark differs from server"
|
||||
);
|
||||
assertManifest(entries);
|
||||
}
|
||||
|
||||
/** Removal exemptions name exact markers and the explicit destructive action.
|
||||
* Deletion mode never disables preservation for unrelated documents. */
|
||||
export class ContentLedger {
|
||||
private readonly markers = new Map<string, string | undefined>();
|
||||
public add(marker: string): void {
|
||||
assert(!this.markers.has(marker), "Duplicate marker");
|
||||
this.markers.set(marker, undefined);
|
||||
}
|
||||
public removedBy(content: string, action: string): void {
|
||||
assert(action.length > 0);
|
||||
for (const marker of this.markers.keys())
|
||||
if (content.includes(marker)) this.markers.set(marker, action);
|
||||
}
|
||||
public assertPreserved(files: Map<string, Uint8Array>): void {
|
||||
const texts = [...files.values()].map((bytes) =>
|
||||
Buffer.from(bytes).toString("utf8")
|
||||
);
|
||||
for (const [marker, deletion] of this.markers) {
|
||||
const count = texts.reduce(
|
||||
(n, text) => n + text.split(marker).length - 1,
|
||||
0
|
||||
);
|
||||
assert(count <= 1, `Marker ${marker} duplicated ${count} times`);
|
||||
if (!deletion)
|
||||
assert.equal(
|
||||
count,
|
||||
1,
|
||||
`Marker ${marker} lost without an explicit delete/overwrite`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
317
frontend/test-support/polling.test.ts
Normal file
317
frontend/test-support/polling.test.ts
Normal file
|
|
@ -0,0 +1,317 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import {
|
||||
Database,
|
||||
emptyState,
|
||||
type StoredDatabase
|
||||
} from "../sync-client/src/persistence/database";
|
||||
import { Settings } from "../sync-client/src/persistence/settings";
|
||||
import { FileOperations } from "../sync-client/src/file-operations/file-operations";
|
||||
import { Syncer } from "../sync-client/src/sync-operations/syncer";
|
||||
import { toStoredSnapshot } from "../sync-client/src/sync-operations/content";
|
||||
import { Logger } from "../sync-client/src/tracing/logger";
|
||||
import { SyncHistory } from "../sync-client/src/tracing/sync-history";
|
||||
import { FixedSizeDocumentCache } from "../sync-client/src/utils/data-structures/fix-sized-cache";
|
||||
import { EventListeners } from "../sync-client/src/utils/data-structures/event-listeners";
|
||||
import type { SyncService } from "../sync-client/src/services/sync-service";
|
||||
import type { ServerConfig } from "../sync-client/src/services/server-config";
|
||||
import type { WebSocketManager } from "../sync-client/src/services/websocket-manager";
|
||||
import type { EventBatch } from "../sync-client/src/services/types/EventBatch";
|
||||
import type { EventRecord } from "../sync-client/src/services/types/EventRecord";
|
||||
import type { PutFileContent } from "../sync-client/src/services/types/PutFileContent";
|
||||
import { MemoryDisk, MemoryPersistence } from "./storage";
|
||||
|
||||
const documentId = "00000000-0000-4000-8000-000000000001";
|
||||
const interval = 1_000;
|
||||
const retryInterval = 75;
|
||||
const bytes = (text: string) => new TextEncoder().encode(text);
|
||||
|
||||
/** Real sync loop over in-memory disk/persistence and an observable version
|
||||
* store. The transport never wakes the engine itself. */
|
||||
async function fixture(syncIntervalMs: number | undefined) {
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite("note.bin", bytes("base"));
|
||||
let headEventId = 2; // Initial content (1), then manifest (2).
|
||||
const versions = new Map([[1, bytes("base")]]);
|
||||
const head = (version: number) => ({
|
||||
documentId,
|
||||
vaultUpdateId: version,
|
||||
contentSize: versions.get(version)!.length,
|
||||
updatedDate: "2026-09-20T00:00:00Z",
|
||||
userId: "user",
|
||||
deviceId: "device"
|
||||
});
|
||||
let latest = head(1);
|
||||
const events: EventRecord[] = [];
|
||||
const writes: PutFileContent[] = [];
|
||||
let eventReads = 0;
|
||||
let failNextRead = false;
|
||||
const publishContent = (content: Uint8Array, requestId: string) => {
|
||||
const eventId = ++headEventId;
|
||||
versions.set(eventId, new Uint8Array(content));
|
||||
latest = head(eventId);
|
||||
events.push({ eventId, requestId, type: "content", document: latest });
|
||||
return latest;
|
||||
};
|
||||
const batch = (after = 2): EventBatch =>
|
||||
structuredClone({
|
||||
headEventId,
|
||||
events: events.filter((event) => event.eventId > after)
|
||||
});
|
||||
const service: Partial<SyncService> = {
|
||||
events: async (after) => {
|
||||
eventReads++;
|
||||
if (failNextRead) {
|
||||
failNextRead = false;
|
||||
throw new Error("offline");
|
||||
}
|
||||
return batch(after);
|
||||
},
|
||||
getDocumentVersionContent: async ({ vaultUpdateId }) => {
|
||||
const content = versions.get(vaultUpdateId);
|
||||
assert(content, `Missing immutable version ${vaultUpdateId}`);
|
||||
return content.slice();
|
||||
},
|
||||
putFileContent: async (id, request) => {
|
||||
assert.equal(id, documentId);
|
||||
assert.equal(request.parentVersionId, latest.vaultUpdateId);
|
||||
assert(request.content.type === "Snapshot");
|
||||
writes.push(structuredClone(request));
|
||||
return {
|
||||
type: "Accepted",
|
||||
...publishContent(
|
||||
Buffer.from(request.content.value, "base64"),
|
||||
request.requestId
|
||||
)
|
||||
};
|
||||
}
|
||||
};
|
||||
const initial = emptyState("polling-test");
|
||||
initial.initialized = true;
|
||||
initial.lastSeenUpdateId = 2;
|
||||
initial.local = { [documentId]: "note.bin" };
|
||||
initial.fileManifest = { fileManifestId: 2, entries: { ...initial.local } };
|
||||
initial.remoteHeads = { [documentId]: latest };
|
||||
const hash = (await toStoredSnapshot({ content: bytes("base") })).hash;
|
||||
initial.documents[documentId] = {
|
||||
materialized: true,
|
||||
observedHash: hash,
|
||||
base: { ...latest, hash }
|
||||
};
|
||||
const persistence = new MemoryPersistence({ database: initial });
|
||||
const logger = new Logger();
|
||||
const database = new Database(
|
||||
logger,
|
||||
initial,
|
||||
async (next) => persistence.save({ database: next }),
|
||||
initial.vaultKey,
|
||||
async () => (await persistence.load()).database as StoredDatabase
|
||||
);
|
||||
const settings = new Settings(
|
||||
logger,
|
||||
{ syncIntervalMs, networkRetryIntervalMs: retryInterval },
|
||||
async () => {}
|
||||
);
|
||||
const config = {
|
||||
initialize: async () => {},
|
||||
getConfig: async () => ({ mergeableFileExtensions: [] }),
|
||||
reset: () => {}
|
||||
} as unknown as ServerConfig;
|
||||
const websocket = {
|
||||
onWebSocketStatusChanged: new EventListeners<
|
||||
(connected: boolean) => unknown
|
||||
>(),
|
||||
onRemoteVaultUpdateReceived: new EventListeners<
|
||||
(batch: EventBatch) => Promise<void>
|
||||
>()
|
||||
};
|
||||
const syncer = new Syncer(
|
||||
"device",
|
||||
logger,
|
||||
database,
|
||||
settings,
|
||||
service as SyncService,
|
||||
websocket as unknown as WebSocketManager,
|
||||
new FileOperations(disk.session(), database, config),
|
||||
config,
|
||||
new SyncHistory(logger),
|
||||
new FixedSizeDocumentCache(0)
|
||||
);
|
||||
return {
|
||||
disk,
|
||||
database,
|
||||
syncer,
|
||||
writes,
|
||||
batch,
|
||||
websocket,
|
||||
reads: () => eventReads,
|
||||
serverContent: () => versions.get(latest.vaultUpdateId),
|
||||
failNextRead: () => {
|
||||
failNextRead = true;
|
||||
},
|
||||
changeRemote: () => {
|
||||
publishContent(bytes("remote edit"), "remote-content");
|
||||
const eventId = ++headEventId;
|
||||
events.push({
|
||||
eventId,
|
||||
requestId: "remote-rename",
|
||||
type: "fileManifest",
|
||||
fileManifest: {
|
||||
fileManifestId: eventId,
|
||||
entries: { [documentId]: "renamed.bin" }
|
||||
}
|
||||
});
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
for (const syncIntervalMs of [undefined, 0, interval]) {
|
||||
for (const side of ["local", "remote"] as const) {
|
||||
test(`${side} changes without notifications ${syncIntervalMs ? "are recovered by explicit polling" : `remain unobserved when polling is ${syncIntervalMs === 0 ? "zero" : "unset"}`}`, async (context) => {
|
||||
context.mock.timers.enable({ apis: ["setTimeout"] });
|
||||
const f = await fixture(syncIntervalMs);
|
||||
try {
|
||||
f.syncer.start();
|
||||
await f.syncer.waitUntilFinished();
|
||||
const reads = f.reads();
|
||||
if (side === "local")
|
||||
await f.disk.userWrite("note.bin", bytes("local edit"));
|
||||
else f.changeRemote();
|
||||
const assertUnobserved = () => {
|
||||
assert.equal(
|
||||
f.reads(),
|
||||
reads,
|
||||
"A hidden wakeup fetched events"
|
||||
);
|
||||
assert.equal(f.database.state.lastSeenUpdateId, 2);
|
||||
assert.equal(f.writes.length, 0);
|
||||
assert.deepEqual(
|
||||
f.serverContent(),
|
||||
bytes(side === "local" ? "base" : "remote edit")
|
||||
);
|
||||
assert.deepEqual(
|
||||
f.disk.userFiles(),
|
||||
new Map([
|
||||
[
|
||||
"note.bin",
|
||||
bytes(side === "local" ? "local edit" : "base")
|
||||
]
|
||||
])
|
||||
);
|
||||
};
|
||||
context.mock.timers.tick(
|
||||
syncIntervalMs ? interval - 1 : 60_000
|
||||
);
|
||||
await f.syncer.waitUntilFinished();
|
||||
assertUnobserved();
|
||||
if (syncIntervalMs) context.mock.timers.tick(1);
|
||||
else if (side === "local")
|
||||
await f.syncer.syncLocallyUpdatedFile({
|
||||
relativePath: "note.bin"
|
||||
});
|
||||
else
|
||||
await f.websocket.onRemoteVaultUpdateReceived.triggerAsync(
|
||||
f.batch()
|
||||
);
|
||||
await f.syncer.waitUntilFinished();
|
||||
const content = bytes(
|
||||
side === "local" ? "local edit" : "remote edit"
|
||||
);
|
||||
const path = side === "local" ? "note.bin" : "renamed.bin";
|
||||
assert.deepEqual(f.serverContent(), content);
|
||||
assert.deepEqual(
|
||||
f.disk.userFiles(),
|
||||
new Map([[path, content]])
|
||||
);
|
||||
assert.deepEqual(f.database.state.local, {
|
||||
[documentId]: path
|
||||
});
|
||||
assert.equal(
|
||||
f.database.state.lastSeenUpdateId,
|
||||
side === "local" ? 3 : 4
|
||||
);
|
||||
assert.equal(f.syncer.isBusy, false);
|
||||
assert.equal(f.writes.length, side === "local" ? 1 : 0);
|
||||
if (syncIntervalMs) {
|
||||
const beforeNextPoll = f.reads();
|
||||
context.mock.timers.tick(interval);
|
||||
await f.syncer.waitUntilFinished();
|
||||
assert.equal(
|
||||
f.reads(),
|
||||
beforeNextPoll + 1,
|
||||
"Polling stopped after its first successful run"
|
||||
);
|
||||
assert.equal(
|
||||
f.writes.length,
|
||||
side === "local" ? 1 : 0,
|
||||
"An idle poll resubmitted unchanged content"
|
||||
);
|
||||
}
|
||||
await f.syncer.stop();
|
||||
const stoppedReads = f.reads();
|
||||
context.mock.timers.tick(60_000);
|
||||
await f.syncer.waitUntilFinished();
|
||||
assert.equal(
|
||||
f.reads(),
|
||||
stoppedReads,
|
||||
"Polling continued after stop"
|
||||
);
|
||||
} finally {
|
||||
await f.syncer.stop();
|
||||
context.mock.timers.reset();
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
test("transient failures retry with polling disabled, then leave the client idle", async (context) => {
|
||||
context.mock.timers.enable({ apis: ["setTimeout"] });
|
||||
const f = await fixture(0);
|
||||
try {
|
||||
f.changeRemote();
|
||||
f.failNextRead();
|
||||
f.syncer.start();
|
||||
await assert.rejects(f.syncer.waitUntilFinished(), /offline/);
|
||||
const reads = f.reads();
|
||||
context.mock.timers.tick(retryInterval - 1);
|
||||
await assert.rejects(f.syncer.waitUntilFinished(), /offline/);
|
||||
assert.equal(f.reads(), reads);
|
||||
context.mock.timers.tick(1);
|
||||
await f.syncer.waitUntilFinished();
|
||||
assert.equal(f.reads(), reads + 1);
|
||||
assert.deepEqual(
|
||||
f.disk.userFiles(),
|
||||
new Map([["renamed.bin", bytes("remote edit")]])
|
||||
);
|
||||
assert.equal(f.database.state.lastSeenUpdateId, 4);
|
||||
assert.equal(f.syncer.isBusy, false);
|
||||
context.mock.timers.tick(60_000);
|
||||
await f.syncer.waitUntilFinished();
|
||||
assert.equal(
|
||||
f.reads(),
|
||||
reads + 1,
|
||||
"Network retries became clean-state polling"
|
||||
);
|
||||
} finally {
|
||||
await f.syncer.stop();
|
||||
context.mock.timers.reset();
|
||||
}
|
||||
});
|
||||
|
||||
test("a failed WebSocket connection catches up over HTTP even when polling is disabled", async () => {
|
||||
const f = await fixture(0);
|
||||
try {
|
||||
f.syncer.start();
|
||||
await f.syncer.waitUntilFinished();
|
||||
f.changeRemote();
|
||||
f.websocket.onWebSocketStatusChanged.trigger(false);
|
||||
await f.syncer.waitUntilFinished();
|
||||
assert.equal(f.database.state.lastSeenUpdateId, 4);
|
||||
assert.deepEqual(
|
||||
f.disk.userFiles(),
|
||||
new Map([["renamed.bin", bytes("remote edit")]])
|
||||
);
|
||||
} finally {
|
||||
await f.syncer.stop();
|
||||
}
|
||||
});
|
||||
392
frontend/test-support/protocol-audit.test.ts
Normal file
392
frontend/test-support/protocol-audit.test.ts
Normal file
|
|
@ -0,0 +1,392 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { ServerControl } from "../deterministic-tests/src/server-control";
|
||||
import { Logger } from "sync-client";
|
||||
import { SyncClient } from "../sync-client/src/sync-client";
|
||||
import { MemoryDisk, MemoryPersistence } from "./storage";
|
||||
import { resolve, join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
const root = resolve(__dirname, "../..");
|
||||
const token = "test-token-change-me";
|
||||
const bytes = (s: string) => new TextEncoder().encode(s);
|
||||
const headers = {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Device-Id": "audit",
|
||||
"Content-Type": "application/json"
|
||||
};
|
||||
async function put(url: string, body: unknown) {
|
||||
const r = await fetch(url, {
|
||||
method: "PUT",
|
||||
headers,
|
||||
body: JSON.stringify(body)
|
||||
});
|
||||
assert(r.ok, await r.clone().text());
|
||||
return r.json();
|
||||
}
|
||||
const content = (s: string) => ({
|
||||
requestId: randomUUID(),
|
||||
parentVersionId: null,
|
||||
content: { type: "Snapshot", value: Buffer.from(s).toString("base64") }
|
||||
});
|
||||
|
||||
test(
|
||||
"connection admission rejects excess clients and releases disconnected slots",
|
||||
{ timeout: 15000 },
|
||||
async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "vault-link-admission-"));
|
||||
const config = join(dir, "config.yml");
|
||||
await writeFile(
|
||||
config,
|
||||
(
|
||||
await readFile(root + "/sync-server/config-e2e.yml", "utf8")
|
||||
).replace("max_clients_per_vault: 256", "max_clients_per_vault: 1")
|
||||
);
|
||||
const server = new ServerControl(
|
||||
root + "/sync-server/target/release/sync_server",
|
||||
config,
|
||||
new Logger()
|
||||
);
|
||||
const sockets: WebSocket[] = [];
|
||||
try {
|
||||
await server.start();
|
||||
const url = server.remoteUri + "/vaults/" + randomUUID();
|
||||
await put(url + "/documents/" + randomUUID(), content("event"));
|
||||
const connect = async () => {
|
||||
const ws = new WebSocket(url.replace("http:", "ws:") + "/ws");
|
||||
sockets.push(ws);
|
||||
const accepted = await new Promise<boolean>((resolve) => {
|
||||
ws.onopen = () =>
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
type: "handshake",
|
||||
token,
|
||||
deviceId: randomUUID(),
|
||||
lastSeenVaultUpdateId: 0
|
||||
})
|
||||
);
|
||||
ws.onmessage = () => resolve(true);
|
||||
ws.onerror = () => resolve(false);
|
||||
ws.onclose = () => resolve(false);
|
||||
});
|
||||
return { ws, accepted };
|
||||
};
|
||||
const first = await connect();
|
||||
assert(first.accepted);
|
||||
assert.equal((await connect()).accepted, false);
|
||||
await new Promise<void>((resolve) => {
|
||||
first.ws.onclose = () => resolve();
|
||||
first.ws.close();
|
||||
});
|
||||
// A completed close releases the server's owned connection permit.
|
||||
assert.equal((await connect()).accepted, true);
|
||||
} finally {
|
||||
for (const ws of sockets) ws.close();
|
||||
await server.stop();
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
test(
|
||||
"an unauthenticated upgraded socket expires without a handshake",
|
||||
{ timeout: 10000 },
|
||||
async () => {
|
||||
const server = new ServerControl(
|
||||
root + "/sync-server/target/release/sync_server",
|
||||
root + "/sync-server/config-e2e.yml",
|
||||
new Logger()
|
||||
);
|
||||
let ws: WebSocket | undefined;
|
||||
try {
|
||||
await server.start();
|
||||
ws = new WebSocket(
|
||||
server.remoteUri.replace("http:", "ws:") +
|
||||
"/vaults/" +
|
||||
randomUUID() +
|
||||
"/ws"
|
||||
);
|
||||
const socket = ws;
|
||||
const closed = await new Promise<boolean>((resolve) => {
|
||||
const timer = setTimeout(() => resolve(false), 6500);
|
||||
socket.onclose = () => {
|
||||
clearTimeout(timer);
|
||||
resolve(true);
|
||||
};
|
||||
socket.onerror = () => {};
|
||||
});
|
||||
assert(closed, "handshake deadline must release idle sockets");
|
||||
} finally {
|
||||
ws?.close();
|
||||
await server.stop();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
test(
|
||||
"bootstrap notifications adopt the existing remote document identity",
|
||||
{ timeout: 15000 },
|
||||
async () => {
|
||||
const server = new ServerControl(
|
||||
root + "/sync-server/target/release/sync_server",
|
||||
root + "/sync-server/config-e2e.yml",
|
||||
new Logger()
|
||||
);
|
||||
let client: SyncClient | undefined;
|
||||
try {
|
||||
await server.start();
|
||||
const vault = randomUUID(),
|
||||
id = randomUUID();
|
||||
const url = server.remoteUri + "/vaults/" + vault;
|
||||
await put(url + "/documents/" + id, content("REMOTE"));
|
||||
await put(url + "/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
const disk = new MemoryDisk();
|
||||
const store = new MemoryPersistence({
|
||||
settings: {
|
||||
remoteUri: server.remoteUri,
|
||||
vaultName: vault,
|
||||
token,
|
||||
isSyncEnabled: true,
|
||||
syncIntervalMs: 0
|
||||
}
|
||||
});
|
||||
client = await SyncClient.create({ fs: disk, persistence: store });
|
||||
await disk.userWrite("a.md", bytes("LOCAL"));
|
||||
await client.syncLocallyCreatedFile("a.md");
|
||||
await disk.userWrite("b.md", bytes("B"));
|
||||
await client.syncLocallyCreatedFile("b.md");
|
||||
await client.start();
|
||||
const files = [...disk.userFiles().keys()];
|
||||
assert.equal(files.length, 2);
|
||||
} finally {
|
||||
await client?.destroy();
|
||||
await server.stop();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
test(
|
||||
"local undo survives a lost accepted reply followed by a rejected retry",
|
||||
{ timeout: 15000 },
|
||||
async () => {
|
||||
const server = new ServerControl(
|
||||
root + "/sync-server/target/release/sync_server",
|
||||
root + "/sync-server/config-e2e.yml",
|
||||
new Logger()
|
||||
);
|
||||
let client: SyncClient | undefined;
|
||||
class SilentWebSocket {
|
||||
static OPEN = 1;
|
||||
readyState = 1;
|
||||
onopen: ((event: object) => void) | null = null;
|
||||
onclose:
|
||||
| ((event: { code: number; reason: string }) => void)
|
||||
| null = null;
|
||||
onmessage: ((event: { data: string }) => void) | null = null;
|
||||
onerror: ((event: object) => void) | null = null;
|
||||
constructor(_url: string | URL) {
|
||||
queueMicrotask(() => this.onopen?.({}));
|
||||
}
|
||||
send(_data: string) {}
|
||||
close() {
|
||||
this.readyState = 3;
|
||||
this.onclose?.({ code: 1000, reason: "" });
|
||||
}
|
||||
}
|
||||
try {
|
||||
await server.start();
|
||||
const vault = randomUUID(),
|
||||
id = randomUUID();
|
||||
const url = server.remoteUri + "/vaults/" + vault;
|
||||
await put(url + "/documents/" + id, content("base"));
|
||||
await put(url + "/file-manifest", {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: { [id]: "a.md" }
|
||||
});
|
||||
const disk = new MemoryDisk();
|
||||
const store = new MemoryPersistence({
|
||||
settings: {
|
||||
remoteUri: server.remoteUri,
|
||||
vaultName: vault,
|
||||
token,
|
||||
isSyncEnabled: true,
|
||||
syncIntervalMs: 0,
|
||||
networkRetryIntervalMs: 0
|
||||
}
|
||||
});
|
||||
let mode: "normal" | "lose-reply" | "reject-retry" = "normal";
|
||||
let lostId: string | undefined;
|
||||
client = await SyncClient.create({
|
||||
fs: disk,
|
||||
persistence: store,
|
||||
webSocket: SilentWebSocket as unknown as typeof WebSocket,
|
||||
fetch: async (input, init) => {
|
||||
if (init?.method === "PUT" && mode === "reject-retry") {
|
||||
assert.equal(
|
||||
JSON.parse(String(init.body)).requestId,
|
||||
lostId
|
||||
);
|
||||
mode = "normal";
|
||||
return new Response("proxy body limit changed", {
|
||||
status: 413
|
||||
});
|
||||
}
|
||||
const response = await fetch(input, init);
|
||||
if (init?.method === "PUT" && mode === "lose-reply") {
|
||||
assert.equal(
|
||||
(await response.clone().json()).type,
|
||||
"Accepted"
|
||||
);
|
||||
lostId = JSON.parse(String(init.body)).requestId;
|
||||
mode = "normal";
|
||||
throw new Error("injected lost accepted response");
|
||||
}
|
||||
return response;
|
||||
}
|
||||
});
|
||||
await client.start();
|
||||
mode = "lose-reply";
|
||||
await disk.userWrite("a.md", bytes("base A"));
|
||||
await client.syncLocallyUpdatedFile({ relativePath: "a.md" });
|
||||
await assert.rejects(client.waitUntilFinished(), /lost accepted/);
|
||||
const original = await (
|
||||
await fetch(url + "/documents/" + id, { headers })
|
||||
).json();
|
||||
await put(url + "/documents/" + id, {
|
||||
...content("base A REMOTE"),
|
||||
parentVersionId: original.vaultUpdateId
|
||||
});
|
||||
mode = "reject-retry";
|
||||
await disk.userWrite("a.md", bytes("base"));
|
||||
await client.syncLocallyUpdatedFile({ relativePath: "a.md" });
|
||||
await client.waitUntilFinished();
|
||||
const result = Buffer.from(
|
||||
(await disk.readSnapshot("a.md"))!.content
|
||||
).toString();
|
||||
assert.equal(result, "base REMOTE");
|
||||
} finally {
|
||||
await client?.destroy();
|
||||
await server.stop();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
test(
|
||||
"closing an old connection cannot erase its replacement's cursors",
|
||||
{ timeout: 15000 },
|
||||
async () => {
|
||||
const server = new ServerControl(
|
||||
root + "/sync-server/target/release/sync_server",
|
||||
root + "/sync-server/config-e2e.yml",
|
||||
new Logger()
|
||||
);
|
||||
const sockets: WebSocket[] = [];
|
||||
try {
|
||||
await server.start();
|
||||
const url = server.remoteUri + "/vaults/" + randomUUID();
|
||||
const id = randomUUID();
|
||||
await put(url + "/documents/" + id, content("example"));
|
||||
const connect = async (deviceId: string) => {
|
||||
const ws = new WebSocket(url.replace("http:", "ws:") + "/ws");
|
||||
sockets.push(ws);
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
ws.onopen = () =>
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
type: "handshake",
|
||||
token,
|
||||
deviceId,
|
||||
lastSeenVaultUpdateId: 0
|
||||
})
|
||||
);
|
||||
ws.onmessage = () => resolve();
|
||||
ws.onerror = () => reject(new Error("connection failed"));
|
||||
});
|
||||
return ws;
|
||||
};
|
||||
const observer = await connect("observer");
|
||||
const first = await connect("same-device");
|
||||
const sendCursor = (ws: WebSocket, start: number) =>
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
type: "cursorPositions",
|
||||
documentsWithCursors: [
|
||||
{
|
||||
document_id: id,
|
||||
relative_path: "note.md",
|
||||
vault_update_id: 1,
|
||||
cursors: [{ start, end: start }]
|
||||
}
|
||||
]
|
||||
})
|
||||
);
|
||||
const seen = (start: number) =>
|
||||
new Promise<void>((resolve) => {
|
||||
observer.onmessage = (event) => {
|
||||
const message = JSON.parse(String(event.data));
|
||||
if (
|
||||
message.type === "cursorPositions" &&
|
||||
message.clients.some(
|
||||
(client: {
|
||||
deviceId: string;
|
||||
documentsWithCursors: {
|
||||
cursors: { start: number }[];
|
||||
}[];
|
||||
}) =>
|
||||
client.deviceId === "same-device" &&
|
||||
client.documentsWithCursors[0]?.cursors[0]
|
||||
?.start === start
|
||||
)
|
||||
)
|
||||
resolve();
|
||||
};
|
||||
});
|
||||
let observed = seen(1);
|
||||
sendCursor(first, 1);
|
||||
await observed;
|
||||
const replacement = await connect("same-device");
|
||||
observed = seen(2);
|
||||
sendCursor(replacement, 2);
|
||||
await observed;
|
||||
await new Promise<void>((resolve) => {
|
||||
first.onclose = () => resolve();
|
||||
first.close();
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
const current = new Promise<
|
||||
{
|
||||
deviceId: string;
|
||||
documentsWithCursors: { cursors: { start: number }[] }[];
|
||||
}[]
|
||||
>((resolve) => {
|
||||
observer.onmessage = (event) => {
|
||||
const message = JSON.parse(String(event.data));
|
||||
if (message.type === "cursorPositions")
|
||||
resolve(message.clients);
|
||||
};
|
||||
});
|
||||
observer.send(
|
||||
JSON.stringify({
|
||||
type: "cursorPositions",
|
||||
documentsWithCursors: []
|
||||
})
|
||||
);
|
||||
const clients = await current;
|
||||
assert.equal(
|
||||
clients.find((client) => client.deviceId === "same-device")
|
||||
?.documentsWithCursors[0]?.cursors[0]?.start,
|
||||
2
|
||||
);
|
||||
} finally {
|
||||
for (const ws of sockets) ws.close();
|
||||
await server.stop();
|
||||
}
|
||||
}
|
||||
);
|
||||
420
frontend/test-support/protocol.test.ts
Normal file
420
frontend/test-support/protocol.test.ts
Normal file
|
|
@ -0,0 +1,420 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { Worker } from "node:worker_threads";
|
||||
import * as path from "node:path";
|
||||
import { Logger } from "sync-client";
|
||||
import { ServerControl } from "../deterministic-tests/src/server-control";
|
||||
import { MemoryDisk, type DiskImage, type StoredClient } from "./storage";
|
||||
import { getJson, type CanonicalSnapshot } from "./canonical";
|
||||
import type { RequestRecord } from "./network";
|
||||
import type { CrashWorkerInput } from "../deterministic-tests/src/crash-worker";
|
||||
|
||||
const root = path.resolve(__dirname, "../..");
|
||||
const token = "test-token-change-me";
|
||||
const headers = {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Device-Id": "protocol-test",
|
||||
"Content-Type": "application/json"
|
||||
};
|
||||
const makeServer = () =>
|
||||
new ServerControl(
|
||||
path.join(root, "sync-server/target/release/sync_server"),
|
||||
path.join(root, "sync-server/config-e2e.yml"),
|
||||
new Logger()
|
||||
);
|
||||
async function put(url: string, body: unknown) {
|
||||
const response = await fetch(url, {
|
||||
method: "PUT",
|
||||
headers,
|
||||
body: JSON.stringify(body),
|
||||
signal: AbortSignal.timeout(5_000)
|
||||
});
|
||||
assert(
|
||||
response.ok,
|
||||
`HTTP ${response.status}: ${await response.clone().text()}`
|
||||
);
|
||||
return (await response.json()) as {
|
||||
type: string;
|
||||
vaultUpdateId: number;
|
||||
fileManifestId: number;
|
||||
};
|
||||
}
|
||||
const content = (text: string, parentVersionId: number | null = null) => ({
|
||||
requestId: randomUUID(),
|
||||
parentVersionId,
|
||||
content: { type: "Snapshot", value: Buffer.from(text).toString("base64") }
|
||||
});
|
||||
|
||||
interface WorkerSnapshot {
|
||||
type: string;
|
||||
disk: DiskImage;
|
||||
stored: StoredClient;
|
||||
requests: RequestRecord[];
|
||||
editedBase?: { documentId: string; vaultUpdateId: number };
|
||||
}
|
||||
async function runWorker(input: CrashWorkerInput): Promise<WorkerSnapshot> {
|
||||
const worker = new Worker(
|
||||
path.join(root, "frontend/deterministic-tests/dist/crash-worker.js"),
|
||||
{ workerData: input }
|
||||
);
|
||||
let timeout: ReturnType<typeof setTimeout> | undefined;
|
||||
try {
|
||||
return await new Promise<WorkerSnapshot>((resolve, reject) => {
|
||||
timeout = setTimeout(
|
||||
() =>
|
||||
reject(new Error("Worker checkpoint/completion timed out")),
|
||||
15_000
|
||||
);
|
||||
worker.once("error", reject);
|
||||
worker.once("exit", (code) =>
|
||||
reject(
|
||||
new Error(
|
||||
`Worker exited (${code}) before expected checkpoint`
|
||||
)
|
||||
)
|
||||
);
|
||||
worker.on(
|
||||
"message",
|
||||
(message: WorkerSnapshot & { error?: string }) => {
|
||||
if (message.type === "error")
|
||||
reject(new Error(message.error));
|
||||
else {
|
||||
assert.equal(
|
||||
message.type,
|
||||
input.stopAt ? "checkpoint" : "finished"
|
||||
);
|
||||
resolve(message);
|
||||
}
|
||||
}
|
||||
);
|
||||
});
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
// This is the fault injection, not graceful shutdown masquerading as it.
|
||||
await worker.terminate();
|
||||
}
|
||||
}
|
||||
|
||||
for (const [kind, format, originalPath] of [
|
||||
["create", "Snapshot", "original.md"],
|
||||
["content", "Snapshot", "original.bin"],
|
||||
["content", "Diff", "original.md"],
|
||||
["manifest", undefined, "original.md"]
|
||||
] as const) {
|
||||
test(
|
||||
`abrupt client death after accepted ${kind} ${format ?? ""} reply, newer head, server SIGKILL, durable retry`,
|
||||
{ timeout: 30_000 },
|
||||
async () => {
|
||||
const server = makeServer();
|
||||
try {
|
||||
await server.start();
|
||||
const vault = `crash-${randomUUID()}`;
|
||||
const url = `${server.remoteUri}/vaults/${vault}`;
|
||||
const disk = new MemoryDisk();
|
||||
await disk.userWrite(
|
||||
originalPath,
|
||||
new TextEncoder().encode("preserve-original\n")
|
||||
);
|
||||
let input: CrashWorkerInput = {
|
||||
disk: disk.image(),
|
||||
stored: {
|
||||
settings: {
|
||||
isSyncEnabled: true,
|
||||
enableTelemetry: false,
|
||||
remoteUri: server.remoteUri,
|
||||
vaultName: vault,
|
||||
token,
|
||||
requestTimeoutMs: 5_000,
|
||||
syncIntervalMs: 0
|
||||
}
|
||||
} as StoredClient
|
||||
};
|
||||
let originalId: string | undefined;
|
||||
let originalParent: number | undefined;
|
||||
let uploaded = "preserve-original\n";
|
||||
if (kind === "content" && format === "Diff") {
|
||||
// Keep the established base in the worker's real content
|
||||
// cache; a cold restart intentionally falls back to Snapshot.
|
||||
uploaded += "local-update\n";
|
||||
input.editAfterSync = {
|
||||
path: originalPath,
|
||||
content: uploaded
|
||||
};
|
||||
} else if (kind === "content") {
|
||||
const initialized = await runWorker(input);
|
||||
originalId = Object.keys(
|
||||
initialized.stored.database!.local!
|
||||
)[0];
|
||||
originalParent =
|
||||
initialized.stored.database!.documents![originalId]
|
||||
.base!.vaultUpdateId;
|
||||
disk.restore(initialized.disk);
|
||||
uploaded += "local-update\n";
|
||||
await disk.userWrite(
|
||||
originalPath,
|
||||
new TextEncoder().encode(uploaded)
|
||||
);
|
||||
input = { disk: disk.image(), stored: initialized.stored };
|
||||
}
|
||||
const checkpoint = await runWorker({ ...input, stopAt: kind });
|
||||
if (format === "Diff") {
|
||||
assert(
|
||||
checkpoint.editedBase,
|
||||
"Warm update baseline was not recorded"
|
||||
);
|
||||
originalId = checkpoint.editedBase.documentId;
|
||||
originalParent = checkpoint.editedBase.vaultUpdateId;
|
||||
}
|
||||
const pending = checkpoint.stored.database!.pending!;
|
||||
assert(
|
||||
pending,
|
||||
"The uncertain request was not persisted before sending"
|
||||
);
|
||||
const interrupted = checkpoint.requests.at(-1)!;
|
||||
assert.equal(
|
||||
interrupted.kind,
|
||||
kind,
|
||||
"Wrong CAS was interrupted"
|
||||
);
|
||||
if (kind !== "manifest") {
|
||||
assert.equal(pending.type, "content");
|
||||
assert(pending.type === "content");
|
||||
assert.equal(pending.request.content.type, format);
|
||||
assert.equal(
|
||||
pending.request.parentVersionId,
|
||||
kind === "create" ? null : originalParent
|
||||
);
|
||||
if (originalId)
|
||||
assert.equal(
|
||||
pending.documentId,
|
||||
originalId,
|
||||
"Update created a replacement UUID"
|
||||
);
|
||||
}
|
||||
const accepted = await getJson<CanonicalSnapshot>(
|
||||
`${url}/vault-snapshot`,
|
||||
token
|
||||
);
|
||||
// Unpublished content has no manifest membership yet, so it is
|
||||
// deliberately absent from /vault-snapshot's document list.
|
||||
const doc =
|
||||
pending.type === "content"
|
||||
? await getJson<{
|
||||
documentId: string;
|
||||
vaultUpdateId: number;
|
||||
}>(`${url}/documents/${pending.documentId}`, token)
|
||||
: accepted.documents[0];
|
||||
assert(doc);
|
||||
const remote = await put(
|
||||
`${url}/documents/${doc.documentId}`,
|
||||
content(`${uploaded}remote-update\n`, doc.vaultUpdateId)
|
||||
);
|
||||
assert.equal(remote.type, "Accepted");
|
||||
const remotePath = originalPath.replace("original", "remote");
|
||||
const renamed = await put(`${url}/file-manifest`, {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: accepted.fileManifest.fileManifestId,
|
||||
entries: { [doc.documentId]: remotePath }
|
||||
});
|
||||
assert.equal(renamed.type, "Accepted");
|
||||
const before = await getJson<CanonicalSnapshot>(
|
||||
`${url}/vault-snapshot`,
|
||||
token
|
||||
);
|
||||
await server.crash();
|
||||
await server.restart();
|
||||
assert.deepEqual(
|
||||
await getJson(`${url}/vault-snapshot`, token),
|
||||
before,
|
||||
"Server restart lost acknowledged state"
|
||||
);
|
||||
const restarted = await runWorker({
|
||||
disk: checkpoint.disk,
|
||||
stored: checkpoint.stored
|
||||
});
|
||||
const replay = restarted.requests.find(
|
||||
(r) => r.body.requestId === pending.request.requestId
|
||||
);
|
||||
assert(replay, "Restart never retried the uncertain request");
|
||||
assert.equal(
|
||||
replay.url,
|
||||
interrupted.url,
|
||||
"Restart changed the request target"
|
||||
);
|
||||
assert.deepEqual(
|
||||
replay.body,
|
||||
pending.request,
|
||||
"Restart changed the persisted request"
|
||||
);
|
||||
assert.equal(restarted.stored.database!.pending, undefined);
|
||||
assert.equal(restarted.stored.database!.application, undefined);
|
||||
const recovered = new MemoryDisk();
|
||||
recovered.restore(restarted.disk);
|
||||
assert.deepEqual(
|
||||
recovered.userFiles(),
|
||||
new Map([
|
||||
[
|
||||
remotePath,
|
||||
new TextEncoder().encode(
|
||||
`${uploaded}remote-update\n`
|
||||
)
|
||||
]
|
||||
])
|
||||
);
|
||||
assert.deepEqual(restarted.stored.database!.local, {
|
||||
[doc.documentId]: remotePath
|
||||
});
|
||||
const events = await getJson<{
|
||||
events: { requestId: string }[];
|
||||
}>(`${url}/events-since?after=0`, token);
|
||||
assert.equal(
|
||||
events.events.filter(
|
||||
(e) => e.requestId === pending.request.requestId
|
||||
).length,
|
||||
1,
|
||||
"Retry emitted a duplicate event"
|
||||
);
|
||||
} finally {
|
||||
await server.stop();
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
test(
|
||||
"concurrent CAS commits have identical HTTP/event-log/WebSocket total order and reconnect catchup",
|
||||
{ timeout: 20_000 },
|
||||
async () => {
|
||||
const server = makeServer();
|
||||
let ws: WebSocket | undefined;
|
||||
try {
|
||||
await server.start();
|
||||
const url = `${server.remoteUri}/vaults/order-${randomUUID()}`;
|
||||
const received: number[] = [];
|
||||
const connect = async (after: number) => {
|
||||
ws = new WebSocket(`${url.replace(/^http/, "ws")}/ws`);
|
||||
ws.addEventListener("message", (message) => {
|
||||
const batch = JSON.parse(String(message.data)) as {
|
||||
type: string;
|
||||
events: { eventId: number }[];
|
||||
};
|
||||
if (batch.type === "vaultEvents")
|
||||
received.push(
|
||||
...batch.events.map((event) => event.eventId)
|
||||
);
|
||||
});
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
ws!.onopen = () => resolve();
|
||||
ws!.onerror = () => reject(new Error("WebSocket failed"));
|
||||
});
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
type: "handshake",
|
||||
token,
|
||||
deviceId: "order-observer",
|
||||
lastSeenVaultUpdateId: after
|
||||
})
|
||||
);
|
||||
};
|
||||
await connect(0);
|
||||
const ids = Array.from({ length: 12 }, () => randomUUID());
|
||||
const requests = ids.map((id, i) => ({
|
||||
id,
|
||||
body: content(`bytes-${i}`)
|
||||
}));
|
||||
const replies = await Promise.all(
|
||||
requests.map(({ id, body }) =>
|
||||
put(`${url}/documents/${id}`, body)
|
||||
)
|
||||
);
|
||||
assert.deepEqual(
|
||||
replies.map((r) => r.vaultUpdateId).sort((a, b) => a - b),
|
||||
ids.map((_, i) => i + 1)
|
||||
);
|
||||
const emptyManifest = await getJson<{
|
||||
entries: Record<string, string>;
|
||||
}>(`${url}/file-manifest`, token);
|
||||
assert.deepEqual(
|
||||
emptyManifest.entries,
|
||||
{},
|
||||
"Content-only CAS changed membership"
|
||||
);
|
||||
const entries = Object.fromEntries(
|
||||
ids.map((id, i) => [id, `${i}.md`])
|
||||
);
|
||||
await put(`${url}/file-manifest`, {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries
|
||||
});
|
||||
// All retries must return original receipts without emitting events.
|
||||
assert.deepEqual(
|
||||
await Promise.all(
|
||||
requests.map(({ id, body }) =>
|
||||
put(`${url}/documents/${id}`, body)
|
||||
)
|
||||
),
|
||||
replies
|
||||
);
|
||||
const waitForEvents = async (head: number) => {
|
||||
const deadline = Date.now() + 5_000;
|
||||
while (received.length < head && Date.now() < deadline)
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
assert.deepEqual(
|
||||
received,
|
||||
Array.from({ length: head }, (_, i) => i + 1),
|
||||
"WebSocket reordered, lost or duplicated an event"
|
||||
);
|
||||
};
|
||||
await waitForEvents(13);
|
||||
await new Promise<void>((resolve) => {
|
||||
ws!.onclose = () => resolve();
|
||||
ws!.close();
|
||||
});
|
||||
await put(
|
||||
`${url}/documents/${ids[0]}`,
|
||||
content("next", replies[0].vaultUpdateId)
|
||||
);
|
||||
await connect(13);
|
||||
await waitForEvents(14);
|
||||
const log = await getJson<{ events: { eventId: number }[] }>(
|
||||
`${url}/events-since?after=0`,
|
||||
token
|
||||
);
|
||||
assert.deepEqual(
|
||||
received,
|
||||
log.events.map((event) => event.eventId)
|
||||
);
|
||||
const stale = await put(`${url}/file-manifest`, {
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 0,
|
||||
entries: {}
|
||||
});
|
||||
assert.equal(stale.type, "StaleBase");
|
||||
const invalid = await fetch(`${url}/file-manifest`, {
|
||||
method: "PUT",
|
||||
headers,
|
||||
body: JSON.stringify({
|
||||
requestId: randomUUID(),
|
||||
parentFileManifestId: 13,
|
||||
entries: { [ids[0]]: "same", [ids[1]]: "same" }
|
||||
})
|
||||
});
|
||||
assert.equal(invalid.status, 422);
|
||||
assert.equal(
|
||||
(
|
||||
await getJson<CanonicalSnapshot>(
|
||||
`${url}/vault-snapshot`,
|
||||
token
|
||||
)
|
||||
).headEventId,
|
||||
14,
|
||||
"Rejected CAS emitted an event"
|
||||
);
|
||||
} finally {
|
||||
ws?.close();
|
||||
await server.stop();
|
||||
}
|
||||
}
|
||||
);
|
||||
25
frontend/test-support/random.ts
Normal file
25
frontend/test-support/random.ts
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
/** A seed controls harness choices; production UUIDs and OS scheduling remain
|
||||
* nondeterministic. The concrete action trace is the stronger replay artifact. */
|
||||
export class Random {
|
||||
private state: number;
|
||||
public constructor(seed: number) {
|
||||
if (!Number.isInteger(seed) || seed < 0 || seed > 0xffffffff)
|
||||
throw new Error("Seed must be a uint32");
|
||||
this.state = seed;
|
||||
}
|
||||
public next(): number {
|
||||
this.state = (this.state + 0x6d2b79f5) >>> 0;
|
||||
let n = this.state;
|
||||
n = Math.imul(n ^ (n >>> 15), n | 1);
|
||||
n ^= n + Math.imul(n ^ (n >>> 7), n | 61);
|
||||
return ((n ^ (n >>> 14)) >>> 0) / 4294967296;
|
||||
}
|
||||
public int(max: number): number {
|
||||
if (!Number.isSafeInteger(max) || max < 1)
|
||||
throw new Error("Empty random choice");
|
||||
return Math.floor(this.next() * max);
|
||||
}
|
||||
public pick<T>(values: readonly T[]): T {
|
||||
return values[this.int(values.length)];
|
||||
}
|
||||
}
|
||||
137
frontend/test-support/real-disk.test.ts
Normal file
137
frontend/test-support/real-disk.test.ts
Normal file
|
|
@ -0,0 +1,137 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import * as fs from "node:fs/promises";
|
||||
import * as path from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { RealDisk } from "./real-disk";
|
||||
import { Database, emptyState } from "../sync-client/src/persistence/database";
|
||||
import { FileOperations } from "../sync-client/src/file-operations/file-operations";
|
||||
import { Logger } from "../sync-client/src/tracing/logger";
|
||||
import type { ServerConfig } from "../sync-client/src/services/server-config";
|
||||
import { toStoredSnapshot } from "../sync-client/src/sync-operations/content";
|
||||
|
||||
test(
|
||||
"real filesystem: exclusive operations, unsafe entries, and journal swaps",
|
||||
{ timeout: 20_000 },
|
||||
async () => {
|
||||
const directory = await fs.mkdtemp(
|
||||
path.join(tmpdir(), "vault-link-real-fs-")
|
||||
);
|
||||
try {
|
||||
const root = path.join(directory, "vault");
|
||||
await fs.mkdir(root);
|
||||
const helper = path.join(directory, "native-fs");
|
||||
await promisify(execFile)("cc", [
|
||||
"-Wall",
|
||||
"-Wextra",
|
||||
"-O2",
|
||||
path.join(__dirname, "native-fs.c"),
|
||||
"-o",
|
||||
helper
|
||||
]);
|
||||
const disk = new RealDisk(root, helper);
|
||||
const snapshot = (text: string) => ({
|
||||
content: new TextEncoder().encode(text)
|
||||
});
|
||||
const writes = await Promise.allSettled([
|
||||
disk.write("raced.bin", snapshot("first")),
|
||||
disk.write("raced.bin", snapshot("second"))
|
||||
]);
|
||||
assert.equal(
|
||||
writes.filter((result) => result.status === "fulfilled").length,
|
||||
1,
|
||||
"Exclusive create overwrote a racing writer"
|
||||
);
|
||||
assert(
|
||||
["first", "second"].includes(
|
||||
Buffer.from(
|
||||
(await disk.readSnapshot("raced.bin"))!.content
|
||||
).toString()
|
||||
)
|
||||
);
|
||||
await disk.write("a.md", snapshot("A"));
|
||||
await disk.write("b.md", snapshot("B"));
|
||||
// Recovery must flush the old source's ancestors even if another
|
||||
// writer has replaced one of those directories with a file.
|
||||
assert.equal(await disk.stat("a.md/absent.md"), undefined);
|
||||
assert.equal(await disk.readSnapshot("a.md/absent.md"), undefined);
|
||||
await disk.flushPaths(["a.md/absent.md"]);
|
||||
await assert.rejects(disk.rename("a.md", "b.md"));
|
||||
await assert.rejects(disk.delete("a.md"), /regular file/);
|
||||
await fs.symlink("a.md", path.join(root, "symlink"));
|
||||
await assert.rejects(disk.readSnapshot("symlink"), /Symlink/);
|
||||
await fs.unlink(path.join(root, "symlink"));
|
||||
await fs.link(path.join(root, "a.md"), path.join(root, "hardlink"));
|
||||
await assert.rejects(disk.readSnapshot("a.md"), /hardlink/);
|
||||
await fs.unlink(path.join(root, "hardlink"));
|
||||
// The host may be case/normalization insensitive. Whichever model
|
||||
// it exposes, an alias must never silently overwrite existing bytes.
|
||||
for (const [original, alias] of [
|
||||
["Case.bin", "case.bin"],
|
||||
["é.bin", "e\u0301.bin"]
|
||||
]) {
|
||||
await disk.write(original, snapshot("original bytes"));
|
||||
if (await disk.exists(alias))
|
||||
await assert.rejects(
|
||||
disk.write(alias, snapshot("alias bytes"))
|
||||
);
|
||||
else {
|
||||
await disk.write(alias, snapshot("alias bytes"));
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await disk.readSnapshot(alias))!.content
|
||||
).toString(),
|
||||
"alias bytes"
|
||||
);
|
||||
}
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await disk.readSnapshot(original))!.content
|
||||
).toString(),
|
||||
"original bytes"
|
||||
);
|
||||
}
|
||||
const initial = emptyState("real");
|
||||
initial.local = { a: "a.md", b: "b.md" };
|
||||
for (const [id, relative] of Object.entries(initial.local))
|
||||
initial.documents[id] = {
|
||||
materialized: true,
|
||||
observedHash: (
|
||||
await toStoredSnapshot(
|
||||
(await disk.readSnapshot(relative))!
|
||||
)
|
||||
).hash
|
||||
};
|
||||
const database = new Database(
|
||||
new Logger(),
|
||||
initial,
|
||||
async () => {},
|
||||
"real",
|
||||
async () => initial
|
||||
);
|
||||
const operations = new FileOperations(disk, database, {
|
||||
getConfig: async () => ({ mergeableFileExtensions: ["md"] })
|
||||
} as ServerConfig);
|
||||
const next = structuredClone(initial);
|
||||
next.local = { a: "b.md", b: "a.md" };
|
||||
await operations.apply(next);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await disk.readSnapshot("a.md"))!.content
|
||||
).toString(),
|
||||
"B"
|
||||
);
|
||||
assert.equal(
|
||||
Buffer.from(
|
||||
(await disk.readSnapshot("b.md"))!.content
|
||||
).toString(),
|
||||
"A"
|
||||
);
|
||||
assert.equal(database.state.application, undefined);
|
||||
} finally {
|
||||
await fs.rm(directory, { recursive: true });
|
||||
}
|
||||
}
|
||||
);
|
||||
215
frontend/test-support/real-disk.ts
Normal file
215
frontend/test-support/real-disk.ts
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
import assert from "node:assert/strict";
|
||||
import * as fs from "node:fs/promises";
|
||||
import * as path from "node:path";
|
||||
import { constants } from "node:fs";
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import type { FileSystemOperations, FileSnapshot } from "sync-client";
|
||||
|
||||
const exec = promisify(execFile);
|
||||
/** Small integration fixture, NOT a supported application adapter. Safety
|
||||
* preflights detect static symlinks/hardlinks; they are not an openat sandbox
|
||||
* against a hostile process replacing ancestors concurrently. */
|
||||
export class RealDisk implements FileSystemOperations {
|
||||
public constructor(
|
||||
private readonly root: string,
|
||||
private readonly helper: string
|
||||
) {}
|
||||
private async resolve(relative: string): Promise<string> {
|
||||
assert(
|
||||
relative === "" ||
|
||||
relative
|
||||
.split("/")
|
||||
.every(
|
||||
(p) => p && p !== "." && p !== ".." && !p.includes("\\")
|
||||
),
|
||||
"Unsafe path"
|
||||
);
|
||||
let current = this.root;
|
||||
for (const component of relative.split("/").filter(Boolean)) {
|
||||
current = path.join(current, component);
|
||||
const entry = await this.info(current);
|
||||
if (entry)
|
||||
assert(
|
||||
entry.isDirectory() ||
|
||||
(entry.isFile() && entry.nlink === 1),
|
||||
"Symlink, hardlink or special file"
|
||||
);
|
||||
}
|
||||
return current;
|
||||
}
|
||||
private async info(absolute: string) {
|
||||
try {
|
||||
return await fs.lstat(absolute);
|
||||
} catch (error) {
|
||||
if (
|
||||
["ENOENT", "ENOTDIR"].includes(
|
||||
(error as NodeJS.ErrnoException).code ?? ""
|
||||
)
|
||||
)
|
||||
return undefined;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
private async flush(absolute: string): Promise<void> {
|
||||
const handle = await fs.open(
|
||||
absolute,
|
||||
constants.O_RDONLY | constants.O_NOFOLLOW
|
||||
);
|
||||
try {
|
||||
await handle.sync();
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
}
|
||||
public async stat(relative: string) {
|
||||
const info = await this.info(await this.resolve(relative));
|
||||
return info
|
||||
? {
|
||||
kind: info.isFile()
|
||||
? ("file" as const)
|
||||
: ("directory" as const),
|
||||
size: info.size
|
||||
}
|
||||
: undefined;
|
||||
}
|
||||
public async exists(relative: string) {
|
||||
return (await this.stat(relative)) !== undefined;
|
||||
}
|
||||
public async readSnapshot(
|
||||
relative: string
|
||||
): Promise<FileSnapshot | undefined> {
|
||||
const absolute = await this.resolve(relative);
|
||||
let handle;
|
||||
try {
|
||||
handle = await fs.open(
|
||||
absolute,
|
||||
constants.O_RDONLY | constants.O_NOFOLLOW
|
||||
);
|
||||
} catch (error) {
|
||||
if (
|
||||
["ENOENT", "ENOTDIR"].includes(
|
||||
(error as NodeJS.ErrnoException).code ?? ""
|
||||
)
|
||||
)
|
||||
return undefined;
|
||||
throw error;
|
||||
}
|
||||
try {
|
||||
const before = await handle.stat({ bigint: true });
|
||||
assert(
|
||||
before.isFile() && before.nlink === 1n,
|
||||
"Not an independent regular file"
|
||||
);
|
||||
const content = new Uint8Array(await handle.readFile());
|
||||
const after = await handle.stat({ bigint: true });
|
||||
const atPath = await fs.lstat(absolute, { bigint: true });
|
||||
assert(
|
||||
before.ino === after.ino &&
|
||||
after.ino === atPath.ino &&
|
||||
before.size === after.size &&
|
||||
before.mtimeNs === after.mtimeNs &&
|
||||
before.ctimeNs === after.ctimeNs,
|
||||
"File changed during read"
|
||||
);
|
||||
return { content };
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
}
|
||||
public async listFilesRecursively(root = ""): Promise<string[]> {
|
||||
const result: string[] = [];
|
||||
for (const name of await fs.readdir(await this.resolve(root))) {
|
||||
const relative = root ? `${root}/${name}` : name;
|
||||
if ((await this.stat(relative))!.kind === "directory")
|
||||
result.push(...(await this.listFilesRecursively(relative)));
|
||||
else result.push(relative);
|
||||
}
|
||||
return result.sort();
|
||||
}
|
||||
public async createDirectory(relative: string): Promise<void> {
|
||||
await this.resolve(relative);
|
||||
let current = this.root;
|
||||
for (const component of relative.split("/").filter(Boolean)) {
|
||||
const next = path.join(current, component);
|
||||
try {
|
||||
await fs.mkdir(next);
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "EEXIST")
|
||||
throw error;
|
||||
}
|
||||
assert((await fs.lstat(next)).isDirectory());
|
||||
await this.flush(next);
|
||||
await this.flush(current);
|
||||
current = next;
|
||||
}
|
||||
}
|
||||
public async write(
|
||||
relative: string,
|
||||
snapshot: FileSnapshot
|
||||
): Promise<void> {
|
||||
assert(
|
||||
!snapshot.cursors?.length,
|
||||
"This disk-only fixture has no editor selections"
|
||||
);
|
||||
const destination = await this.resolve(relative);
|
||||
const staging = ".vault-link-sync/adapter-staging";
|
||||
await this.createDirectory(staging);
|
||||
const source = path.join(this.root, staging, randomUUID());
|
||||
const handle = await fs.open(source, "wx", 0o600);
|
||||
try {
|
||||
await handle.writeFile(snapshot.content);
|
||||
await handle.sync();
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
try {
|
||||
await exec(this.helper, ["rename", source, destination]);
|
||||
await this.flush(path.dirname(source));
|
||||
await this.flush(path.dirname(destination));
|
||||
} finally {
|
||||
await fs.unlink(source).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code !== "ENOENT") throw error;
|
||||
});
|
||||
}
|
||||
}
|
||||
public async rename(from: string, to: string): Promise<void> {
|
||||
const source = await this.resolve(from),
|
||||
destination = await this.resolve(to);
|
||||
assert((await this.info(source))?.isFile());
|
||||
await this.flush(source);
|
||||
await exec(this.helper, ["rename", source, destination]);
|
||||
await this.flush(path.dirname(source));
|
||||
await this.flush(path.dirname(destination));
|
||||
}
|
||||
public async flushPaths(paths: readonly string[]): Promise<void> {
|
||||
for (const relative of paths) {
|
||||
let absolute = await this.resolve(relative);
|
||||
for (;;) {
|
||||
if (await this.info(absolute)) await this.flush(absolute);
|
||||
if (absolute === this.root) break;
|
||||
absolute = path.dirname(absolute);
|
||||
}
|
||||
}
|
||||
}
|
||||
public async delete(relative: string): Promise<void> {
|
||||
assert(relative, "Cannot prune root");
|
||||
const absolute = await this.resolve(relative);
|
||||
const entry = await this.info(absolute);
|
||||
if (!entry) return;
|
||||
assert(entry.isDirectory(), "Cannot delete regular file");
|
||||
for (const child of await fs.readdir(absolute))
|
||||
await this.delete(`${relative}/${child}`);
|
||||
await fs.rmdir(absolute);
|
||||
await this.flush(path.dirname(absolute));
|
||||
}
|
||||
public async deleteFile(relative: string): Promise<void> {
|
||||
const absolute = await this.resolve(relative);
|
||||
const entry = await this.info(absolute);
|
||||
if (!entry) return;
|
||||
assert(entry.isFile() && entry.nlink === 1, "Cannot unlink non-file");
|
||||
await fs.unlink(absolute);
|
||||
await this.flush(path.dirname(absolute));
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue