mor

frontend/test-client/README.md

@@ -1,36 +1,71 @@
 # Seeded sync workloads
 
-The old random mock filesystem/agent have been replaced by the same v4 adapter,
-client wrapper, and canonical-state assertions as the scripted suite.
-Periodic polling is explicitly disabled so it cannot repair missed notifications
-and hide failures in these workloads. Polling has separate fake-clock tests.
+The fuzzer runs three real clients against an isolated real server. It shares the
+scripted suite's filesystem adapter and canonical-state checks. Polling is disabled
+so it cannot repair missing notifications; separate fake-clock tests cover polling.
 
-From `frontend`:
+From `frontend`, after building the server:
 
 ```sh
-npm run build --workspace sync-client --workspace test-client
+npm run build --workspace sync-client --workspace deterministic-tests --workspace test-client
 node test-client/dist/cli.js --seed 42 --iterations 25
 node test-client/dist/cli.js --replay /path/to/trace-42.json
 ```
 
-The server binary must already be built. Each invocation starts and cleans up its
-own server/database, and prints the replay command before executing any actions.
-`--artifacts` chooses the trace/result directory. Failures include database,
-visible/durable disk images, and CAS request records. The default 25 iterations
-are scenarios selected in shuffled cycles of nine operations, using two or three
-clients. Existing documents are repeatedly edited, moved, swapped, deleted and
-recreated. Concurrent edits share an established base; delayed editor events stay
-queued through reconnection and a subsequent remote edit. Create, content-update
-and manifest requests receive before/after network faults.
+Each invocation owns its server and database. `--artifacts` selects the output
+directory. The trace is written before execution; results retain exercised request
+and crash checkpoints, response types and coverage counts. Failures also retain
+metadata, visible/durable disk images and CAS requests before cleanup.
 
-Each checkpoint serializes a complete expected file set with exact bytes, paths
-and persistent identity keys. Independent creates must remain distinct, and a
-recreated file must obtain a new UUID. Later checkpoints reassert earlier edits
-and moves, rather than retaining only their markers. Old traces without a client
-count still replay with two clients.
+## Workload and oracle
 
-All preservation assertions run even with delayed editor events. Explicit deletes
-and overwrites exempt only their specific markers; unrelated content still must
-appear exactly once. UUID continuity and canonical server state are checked too.
-Seeds reproduce choices, not OS scheduling. See the
+Version 2 traces contain an editor-intent program and its compiled steps.
+`model.ts` independently interprets that intent into complete expected file sets:
+exact bytes, paths and persistent identity keys. It imports no production merge
+implementation and never adopts observed output as the expected state. Deleted
+identity keys remain reserved. Concurrent text cases specify exact allowed results;
+binary conflicts specify the documented first-committed winner.
+
+Each epoch composes arbitrary creates, replacements, range deletions, renames,
+swaps and immediate delete/recreate operations before synchronization. Conflict
+copies stay in the document pool and are subsequently edited, renamed and deleted.
+Sixteen fault/concurrency families run in seeded shuffled cycles, including:
+
+- Three writers editing separated fields and two writers making overlapping edits.
+- Forced stale-base races: hold a content CAS while another writer commits, rename
+  the local file, then require a new `StaleBase` response and the exact merged bytes.
+- Online primitive histories and offline batches, delayed editor notifications,
+  and failures before or after durable metadata saves.
+- Requests held before send or after receipt while further writes/renames occur;
+  dropped create, content and manifest requests with exact retry checks.
+- Abrupt worker termination after accepted CAS responses, at metadata-save
+  boundaries, and during remote file installation, followed by user edits and
+  recovery. Some histories also kill and restart the server.
+
+Inputs include empty text, invalid UTF-8 bytes, Unicode, combining characters,
+emoji, BOMs, CRLFs, size/cache boundaries, long filenames, case/normalization
+aliases and file/directory conflicts. Initial inventories include 70 and 130
+documents to exercise paginated catchup; caches range from zero to 64 KiB.
+Every checkpoint checks the full expected state on all replicas and the server.
+Full-state checks also repeat after delayed events and crash recovery.
+
+## Campaigns, replay and reduction
+
+`scripts/e2e.sh` replays every checked-in `corpus/*.json` trace before fresh random
+seeds. `E2E_SEED` pins a campaign; without it the runner prints and saves a fresh
+uint32 seed range. Scheduled CI runs use 100 rounds per worker. The aggregate
+`coverage.json` reports actions, CAS responses/payloads and exercised fault
+boundaries. These are behavioral counters, not source branch-coverage percentages.
+
+On failure, the CLI automatically tries to remove epochs and individual actions,
+recompiling every expected state from the remaining intent. Dependency-invalid
+histories are skipped. A reduction must reproduce the same failure twice before
+acceptance. `--shrink-attempts` controls the budget (default 32); `--no-shrink`
+retains just the original reproducer. Replay `minimized-<seed>.json`, fix the bug,
+and commit that concrete program to `corpus/`. Version 1 action traces still replay.
+
+Request gates and crash targets are replayed and must actually fire. UUID-derived
+paths use symbolic identity keys. Seeds do not control OS or server scheduling,
+and replay does not reuse production UUIDs. This is broader bug discovery, not a
+proof of all possible interleavings or correctness. See the
 [full harness documentation](../deterministic-tests/README.md).