.
This commit is contained in:
parent
7198639db4
commit
b5f448706e
1 changed files with 314 additions and 147 deletions
125
CLAUDE.md
125
CLAUDE.md
|
|
@ -105,6 +105,131 @@ The map is keyed by `record.path`; the invariant `documents.get(record.path) ===
|
|||
|
||||
**Server catch-up.** The server's WS handshake replays events newer than the client's `last_seen_vault_update_id` from the `latest_document_versions` view (one row per doc, the latest). On those replayed rows `is_new_file` means *new to this client* (`creation_vault_update_id > last_seen_vault_update_id`), not "this row is the doc's first version" — necessary because the catch-up only carries the latest version; if a doc was created and updated past the watermark, the client never sees its create otherwise.
|
||||
|
||||
## Edge-case patterns the sync engine has to survive
|
||||
|
||||
These are non-obvious from reading any single file; they fall out of the
|
||||
interaction between the queue, the watcher, the WebSocket, and the
|
||||
server's commit ordering. Treat the engine as a black box and what
|
||||
follows is the kinds of bugs you should expect to see:
|
||||
|
||||
**FIFO drain order ≠ user's perceived order.** The queue is single-consumer
|
||||
and FIFO at processing time, but the producers are concurrent and async
|
||||
indirected: user FS actions go through watcher → microtask → enqueue
|
||||
(several microtasks deep), while WS messages go through the onmessage
|
||||
handler. A WS-driven event can land in the queue *between* two user
|
||||
actions even when the user "did them in order". When you read a log,
|
||||
"Decided to ..." timestamps mark the user's intent; they do **not** map
|
||||
to the order of `events.push`.
|
||||
|
||||
**`event.path` is a side channel through disk.** Drain serialises which
|
||||
event runs, but it can't lock disk between events. Between an event's
|
||||
enqueue and its drain, another in-band event can have rewritten the
|
||||
file at that path (a remote-create that landed on the slot, a delete +
|
||||
re-create cycle by the user). Reading at drain time gets *current* disk
|
||||
content — which may be a different doc's bytes — and uploading them as
|
||||
the queued event's content is a duplicate-create / wrong-content bug.
|
||||
|
||||
**Pending-create docId is a `Promise`, not a string, until the create
|
||||
acks.** Any event queued behind a still-in-flight LocalCreate that
|
||||
references the same doc carries the create's `resolvers.promise` as its
|
||||
`documentId`. Two consequences: (a) `===` comparisons against the
|
||||
resolved string in any rewrite loop silently fail; (b) the order of
|
||||
"swap Promise→docId" vs "rewrite paths in events" matters — swap first
|
||||
or the rewrite walks past the events you wanted to retarget. This is
|
||||
load-bearing in any code that touches the queue right after a create
|
||||
resolves.
|
||||
|
||||
**`record.path` is mutated in place across awaits.** When a user rename
|
||||
runs while a drain handler is awaiting an HTTP roundtrip, the queue
|
||||
mutates the in-flight event's record so subsequent reads see the new
|
||||
path. Snapshotting `record.path` into a local at function entry and
|
||||
using it after an `await` writes/reads from a now-vacated slot.
|
||||
Snapshot only for the *deliberate* "did the path change while I was
|
||||
awaiting" comparison; everywhere else, read `record.path` live.
|
||||
|
||||
**Conflict-uuid stashes are local-only divergence.** Whenever a slot
|
||||
collision deflects a doc to `conflict-<uuid>-…`, only the agent that
|
||||
deflected has that file. The cross-agent fuzz assertion ("every path
|
||||
matches across clients") will fire on it. By design these are awaiting
|
||||
manual user resolution — but if your fix silently creates one in a
|
||||
race that *would* converge given more time, the e2e fuzz will show it.
|
||||
|
||||
**`MoveOnConflict.NEW` vs `EXISTING` is a policy choice, not a default.**
|
||||
NEW preserves the occupant and stashes us at conflict-uuid; EXISTING
|
||||
evicts the occupant and stashes *them*. Picking wrong creates either an
|
||||
orphaned stash on us or an orphaned tracking entry on the occupant.
|
||||
The right choice depends on whether the occupant is tracked, whether
|
||||
they have a pending RemoteChange that will move them, and which side
|
||||
the server has already committed to.
|
||||
|
||||
**Pause / disable-sync mid-flight is a destabiliser.** A request whose
|
||||
HTTP committed server-side but whose response was discarded by an abort
|
||||
leaves the server holding a doc the client has no record of. The next
|
||||
re-enable's offline scan re-derives state from disk vs. the (now
|
||||
incomplete) `documents` map and emits a fresh LocalCreate — a duplicate
|
||||
of a doc already on the server, with a new docId. The catch-up then
|
||||
delivers the orphan as a "new" doc and writes it to disk. Final state:
|
||||
two files, two docIds, same content. Anything that aborts in-flight
|
||||
HTTPs (start-reset, vault change, destroy) needs the queue's documents
|
||||
map to be wiped or rebuilt from the server, not just the events array.
|
||||
|
||||
**`scheduleSyncForOfflineChanges` clears `events[]` but not `documents`.**
|
||||
Every enable-sync wipes pending local events. The offline scan
|
||||
re-derives them by comparing disk to the documents map (matching by
|
||||
content hash to recognise renames). This is correct *if* the documents
|
||||
map reflects the last server state we committed to. If it lags (an
|
||||
in-flight create whose response we lost; a remote update we haven't
|
||||
applied yet), the scan misclassifies — a real rename becomes a delete
|
||||
+ create with a new docId; a still-tracked doc whose file we deleted
|
||||
becomes a delete the server hasn't seen.
|
||||
|
||||
**Watermark advancement is load-bearing both ways.** Branches that *skip*
|
||||
a remote event without advancing `lastSeenUpdateId` create permanent
|
||||
gaps that re-deliver forever. Branches that *advance* the watermark
|
||||
without applying the content lose data — the server has no further
|
||||
event to re-deliver, the catch-up only carries the latest version, and
|
||||
any state in between is gone. When in doubt: don't advance unless the
|
||||
event was actually applied (or deliberately discarded after weighing
|
||||
both halves).
|
||||
|
||||
**`isNewFile` semantics differ between catch-up and real-time.** On WS
|
||||
handshake replay it means *new to this client* (`creation_vault_update_id
|
||||
> last_seen_vault_update_id`); on real-time broadcasts it means *this
|
||||
version is the create* (`creation_vault_update_id == vault_update_id`).
|
||||
A handler that receives "untracked doc + isNewFile=false" and decides
|
||||
based on one of the two interpretations will be wrong on the other
|
||||
channel. Reasoning about whether to fetch-and-treat-as-new vs. ignore
|
||||
needs to know which channel delivered the event.
|
||||
|
||||
**Race-shape catalogue.** Bugs in this codebase tend to fall into a
|
||||
small set of shapes; recognising the shape from the log gets you most
|
||||
of the way to the cause:
|
||||
|
||||
- *Same-path dedup race*: two clients create at the same path. Server
|
||||
deconflicts the second to `path (1)`. The losing client must
|
||||
relocate locally; mishandling routes the local file to a stash.
|
||||
- *Concurrent rename of same doc*: both clients rename. Server
|
||||
applies in commit order; the loser's local-rename HTTP must rebase
|
||||
against the server's new path or be dropped.
|
||||
- *Local rename + remote rename of same doc*: the local rename's HTTP
|
||||
needs to find the doc at the (now-different) server path; the
|
||||
matching disk file needs to follow without stranding.
|
||||
- *Pending create + remote create at same path*: the agent's pending
|
||||
file is already at the slot the remote wants; the remote's pending
|
||||
bytes will reach the slot the agent is trying to upload from.
|
||||
- *Create + delete + remote create at same path*: the user's local
|
||||
cycle queues two events; a remote create lands in between. The
|
||||
queued LocalCreate (or a re-emitted offline-scan one) reads disk
|
||||
content placed by the remote and uploads it as a third doc.
|
||||
- *Pause-mid-flight*: in-flight HTTP committed server-side, response
|
||||
abandoned client-side. After re-enable the offline scan can't tell
|
||||
the doc was already created and creates a duplicate.
|
||||
|
||||
When triaging a fuzz failure, find the divergent file in `e2e-run.log`'s
|
||||
final dump (it shows each agent's tracked docs), grep the `log_<i>.log`
|
||||
for that path/docId, and match the lifecycle against this catalogue
|
||||
before going deeper.
|
||||
|
||||
## Two complementary E2E harnesses
|
||||
|
||||
- **`test-client` (fuzz):** random ops across N parallel processes for many minutes. Used by `scripts/e2e.sh`. Catches bugs nobody thought to write a test for, but reproductions are noisy.
|
||||
|
|
|
|||
Loading…
Reference in a new issue