Fix the recurring session-stall pair: MainActor lock-reconcile hang + vminitd relay spin
Host (dominant): AppStore.reconcileLocks polls every 3s on the MainActor while any lock is held — effectively forever, since interrupted/errored sessions deliberately retain locks. Each pass ran heldPathDisposition's diverges() as three held×unmerged scans with two split-allocations per pathsOverlap call, pinning the main thread for tens of seconds per pass on a diverged trunk (hang-reports 2026-07-28: 100% of samples in reconcileLocks→heldPathDisposition→pathsOverlap). That froze running sessions' transcripts and starved the spawn path into the 60s "produced no output" watchdog. pathsOverlap is now allocation-free bytewise comparison with identical semantics, and divergentHeldPaths answers all three questions from one O((held+unmerged)·depth) set. Guest (persistence): VsockProxy threaded ONE offset pair through BOTH relay directions; once the EAGAIN-return backpressure patch let pending bytes persist, traffic in the other direction skewed the shared counters, made the write leg unreachable, and spun the single ProcessSupervisor poller thread forever — container-wide dead control plane until VM recreation, triggered by exactly the backpressure the host hang created. Each direction now owns its own pipe and counters (OSFile.RelayDirection), and source EOF is only surfaced after the pipe drains so SHUT_WR can't truncate a parked tail. Vendored patch docs updated (#15); inert until the initfs image is rebuilt+repointed. Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
+28
-3
@@ -233,7 +233,30 @@ rebuild whenever a guest patch changes. Built locally, not in CI: the host frame
|
||||
non-blocking before either is registered.
|
||||
All three are guest-side and INERT until the initfs image is rebuilt (`make vminit-image`, tag
|
||||
`0.34.0-nucleic4`) and `ContainerEngine.vminitReference` is bumped after runtime validation.
|
||||
Marked `[Nucleic vendored patch]`.
|
||||
Marked `[Nucleic vendored patch]`. **NOTE:** the splice EAGAIN-return introduced a latent
|
||||
cross-direction accounting hazard fixed by patch #15.
|
||||
|
||||
15. **Per-direction relay state in `OSFile+Splice.swift` + `VsockProxy.swift` — fixes the
|
||||
poller-thread spin patch #14 made reachable.** The old `SpliceFile` design threaded ONE offset
|
||||
pair through BOTH directions of a proxied connection: each fd's struct served as the
|
||||
read-counter for one direction and the write-counter for the other, so the loop guards compared
|
||||
*differences of two directions' counters*. That was survivable only while every splice call
|
||||
fully drained its transfer pipe before returning. Once patch #14's EAGAIN branch let pending
|
||||
bytes persist across calls, one parked direction skewed the shared counters for the other: its
|
||||
write leg's `to.offset < from.offset` guard went false with data still in the pipe, and the
|
||||
outer `while true` then alternated read-EAGAIN/skip-write forever — a hard 100% spin on the
|
||||
single `ProcessSupervisor` poller thread (epoll never runs again, so the event that would
|
||||
un-skew the counters can never be processed). Container-wide dead control plane + frozen stdio
|
||||
until the VM is recreated; triggered in practice by host-side backpressure (any slow host
|
||||
reader), and bidirectional traffic on one connection (e.g. the control plane's SSE stream +
|
||||
requests). Replaced `SpliceFile`/`OSFile.splice` with `OSFile.RelayDirection` (each direction
|
||||
owns its OWN pipe, `bytesIn`/`bytesOut`, and `sawSourceEOF`) and `OSFile.relay`, which also
|
||||
fixes a second latent defect: source EOF is now reported only after the pipe fully drains, so
|
||||
the caller's SHUT_WR can never truncate a parked tail (the old read leg returned `.eof`
|
||||
immediately, dropping pending bytes). `VsockProxy.handleConn` now tracks two directions with
|
||||
independent done flags; a broken destination ends both. Guest-side and INERT until the initfs
|
||||
image is rebuilt and repointed (mind the `vminit.ext4.reference` cache sidecar). Marked
|
||||
`[Nucleic vendored patch]`.
|
||||
|
||||
## Re-vendoring a newer upstream commit
|
||||
|
||||
@@ -256,8 +279,10 @@ rebuild whenever a guest patch changes. Built locally, not in CI: the host frame
|
||||
`KeychainQuery` reads: `withoutInteractiveUI` + the `errSecInteractionNotAllowed` handling +
|
||||
the `save` duplicate retry), and patch #14 (the non-blocking/non-spinning guest I/O plane:
|
||||
`IOPair` backpressure, the `OSFile.splice` EAGAIN return, and the `VsockProxy` pre-registration
|
||||
non-blocking fds — all in `vminitd/`). After re-applying any
|
||||
`vminitd/` patch, rebuild + publish the custom init image
|
||||
non-blocking fds — all in `vminitd/`), and patch #15 (the per-direction
|
||||
`OSFile.RelayDirection`/`OSFile.relay` rewrite + the two-direction `VsockProxy.handleConn`,
|
||||
which supersede the upstream `SpliceFile`/`splice` shapes entirely — in `vminitd/`). After
|
||||
re-applying any `vminitd/` patch, rebuild + publish the custom init image
|
||||
with `make vminit-image` + `make vminit-image-push`, and bump `ContainerEngine.vminitReference`.
|
||||
5. Update the commit hash above and in the root `Package.swift` comment.
|
||||
6. `swift build` and run the balloon tests.
|
||||
|
||||
Reference in New Issue
Block a user