Repository navigation
Add an independent .NET adapter - #27
Merged
Merged
Conversation
A .NET 8 library in dotnet/src/JacquardHost (package JacquardLang.Host, no runtime dependencies beyond the base class library) implementing jacquard-host-v0: a strict hand-written JSON codec that parses integers exactly from their text as a long, the v0 value and type codecs, limits, the session state machine, a bounded process carrier for `jacquard host worker`, a Host with a closed operation registry, and replay of the pinned conformance kit, carried as embedded resources. .NET cannot start the worker in its own process group, and its pipe streams have no read or write a deadline or cancellation can interrupt. So the carrier never calls one that can block: the caller's thread waits on the C library's poll(2), then reads only bytes already in the pipe or writes at most PIPE_BUF bytes. Every wait rechecks one lifetime deadline, the CancellationToken, and disposal. Expiry kills the worker's process tree with Process.Kill(true), and a descendant that keeps stdout or stderr open after the worker exits is reported as a carrier loss. Frame encoding counts UTF-8 bytes as it writes and stops at max_frame_bytes; text validation refuses a text longer than max_text_bytes before scanning it and charges every text against the frame; a caller's raw map is bounded before it is copied or sorted. No handler starts after a stop: the caller checks after decoding a request, and the handler thread checks again before calling it. Replay keeps at most one frame past each recording's. core_pin.py generates Compat.cs and keeps the .NET kit copy in sync, with its tests extended. NOTICE names the new copy, and every adapter's NOTICE copy stays identical to it.
NUnit tests on .NET 8 and 10 cover the strict codecs, exact integers, limits, Core-order descriptor checks, the session state machine, and bounded encoding and validation. A stand-in worker program replays all 21 recordings and simulates faulty and hostile workers: malformed, extra, and oversized frames; silent, stalled, truncated, and nonzero workers; a worker that never reads its input; descendants that hold the pipes, daemonize, or flood stdout or stderr. Test seams (internal, null outside tests) stop a call right after a request is decoded and right before the handler thread's check. Linux tests count descriptors and handler threads across successes and failures. Live tests replay the kit against Core and run the README example, failure pairs, and handler deadlines; with JACQUARD_REQUIRE_LIVE set they fail rather than skip. The README documents the API, exact integers and money as integer minor units, cancellation, and the carrier; a test keeps its example identical to the compiled one.
A dotnet job runs the offline tests on .NET 8 and 10, and the shared pinned-Core job runs the live tests on both with JACQUARD_REQUIRE_LIVE set, so a misconfigured job fails rather than skips.
The handlers' token is cancelled by a timer at the deadline, and the timer and the worker's clock differ: a live run saw a cooperative handler return on its token a moment before the worker reached its deadline, and its answer was sent. After a handler returns, a cancelled token now means the call is stopping, and the host waits for the worker to report the stop instead of answering. A test seam (null outside tests) makes the timer fire 300 ms early; without the guard the test sees the answer sent.
LIMITS.md records the process-tree kill, the poll-based pipe I/O and its Linux and macOS scope, cooperative handler cancellation, inherited descriptors, and the protocol-level scope. EVIDENCE.md records the offline and live results on .NET 8 and 10 against jacquard-core-0.3.0, and which test fails when each guard is removed. The README documents bounded work on the caller's thread. CONTRIBUTING, COMPATIBILITY, the root README, and the changelog name the new adapter, and the source archive carries it.
Cancelling the handlers' token ran every callback a handler had registered synchronously: on the invoking thread when the call ended, on the thread that cancelled the caller's token, and on the CancelAfter timer. A blocking callback held Invoke past its deadline, a throwing one escaped Invoke as AggregateException or crashed the process from the timer, and disposing the caller-token registration waited for a running callback. The token is now cancelled on a thread-pool work item that discards callback exceptions. The deadline uses a Timer that queues that work item (re-armed in waits of at most a day, so no millisecond conversion overflows), the caller-token link only queues it, and End unregisters without waiting.
The poll timeout multiplied the ticks left by 1000 before applying the 5 ms cap. A deadline beyond about 107 days overflowed to a negative timeout, and poll with a negative timeout waits forever, so cancellation and disposal no longer ended reads, writes or exit waits. The +1 also let a poll end up to a millisecond after the deadline. The timeout is now capped in ticks before any multiplication and rounded down. The lifetime clamp is named (Worker.MaxDeadline, about 73 years, for any positive TimeSpan up to MaxValue) and guarded against overflow when added to the current timestamp.
HandlerException accepted any cast FailureCategory value, and dispatch trusted it, so throw new HandlerException((FailureCategory)999, ...) reached the wire-name mapping and threw ArgumentOutOfRangeException out of Invoke. The constructor now refuses an undefined category with ArgumentOutOfRangeException, and dispatch treats any undefined category that still arrives as an unexpected exception: completion unknown.
Capabilities were copied from the registry when the invoke frame was built, but effect requests were looked up in the live registry, so a concurrent Bind could make the host run an operation it never advertised. Invoke now takes one immutable snapshot (bounded by max_operations before it is copied) and uses it for both.
They are internal, null outside tests, and global to the process, which is why the tests run serially.
Without a global.json, dotnet uses the newest SDK on the runner, so the 8.0 job could compile with SDK 10. The job now writes a global.json pinning 8.0.x (rollForward latestFeature) and checks the selected SDK before testing. The 10.0 job and local builds are unchanged.
The README and LIMITS now say that handler token callbacks run on a pool thread the host never waits for, that the token is cancelled shortly after the deadline while IsCancelled is exact, which deadlines are accepted, that dispatch uses the advertised registry snapshot, and that Process.Kill(true) signals descendants by pid, so a reused pid can be signalled. EVIDENCE records the new tests and which fail without their fixes.
…tion ends Remove the handler token's timer and its registration on the caller's token. The caller's thread already sees the deadline and cancellation in its 5 ms waits and ends the invocation; End now starts one short-lived background thread that cancels the token, before the worker is disposed. A lifetime over a day no longer cancels the token a day early, and a saturated thread pool no longer delays the cancellation. Add a lifetime clock seam to Worker so a test can move a 36- or 48-hour lifetime to its deadline, and tests that starve the pool and check the token is cancelled at the deadline, at the caller's cancellation, and at the end of a successful invocation before any later pool work item runs.
Killing the worker's process tree scans the process table and took about 240 ms here, and the token was cancelled only after it. The handler wait now ends as soon as the call is stopping, the caller's thread starts the token's cancellation (at most once per invocation), and only then does the fault check kill the worker. The deadline-to-cancellation time measured in the clock-seam test fell from about 244 ms to 2-4 ms. A test seam runs just before the kill and waits for the token's callback; with the old order it times out. Another test pins where callbacks run: on the host's cancelling thread, or, when registered after the token is cancelled, synchronously on the handler's thread.
… run The token is cancelled when the call stops or ends and never earlier; while a handler runs, the caller's thread checks the deadline and the caller's token at least every 5 ms and starts the cancellation before it kills the worker. Callbacks run on the host's own cancelling thread, not a pool thread, and one registered after the cancellation runs at once on the handler's thread.
The wait for a handler read the stop condition in its predicate and again in its fault check, so a deadline, caller cancellation, or disposal landing between the two reads (or between the post-handler stopping and fault checks) killed the worker before the token's cancellation started. The worker now gives a stop notice before every kill, from whichever thread kills it, and the host starts the cancellation from that notice.
…disposal Starting the cancellation of the handlers' token now catches any failure to create or start its thread, falls back to a thread-pool work item, and if that fails too records InvocationResult.HandlerTokenNotCancelled and gives up; it never runs callbacks on the calling thread. The worker is disposed in a nested finally, so it is released even if ending the call fails.
…ancellation fallbacks The interval between the caller's checks can exceed 5 ms (stderr drains and scheduling), so the docs now call 5 ms the requested poll timeout rather than a wall-clock guarantee. They also state that every kill comes after the token's cancellation starts, and what happens when no thread, and then no pool work item, can be started.
…line and 95 live tests
When Process.Start succeeded but taking the worker's pipes failed, Start killed the process directly and disposed it at once: no stop notice, no kill bookkeeping, no bounded exit wait, and a failed kill was swallowed while the last handle was dropped. The Worker now owns the process from the spawn. Attach takes the pipes all or none; on failure FailAfterSpawn kills through KillTree (after the stop notice), waits up to one second for the exit, records it, and reports a spawn failure that says so if the worker had not exited. Disposal releases the handle. The host passes its stop notice to Start, so it is in place before any kill. A test seam (SpawnedHook, null outside tests) fails the setup after the spawn.
StopNoticeTests now fails notice, kill, kill: each kill must consume a notice given on its own thread before it. It covers disposal killing on the disposer's own thread and on a waiting operation's thread, and the kill when setting up the spawned worker's pipes fails.
The DeadlineAfterTheHandler case slept 100 ms and assumed the handler's answer was published. A seam (HandlerAnsweredHook, null outside tests) now runs after the answer is published, and the test waits for it before it arms the clock. Each case asserts from the killing thread's stack which check killed the worker: the handler wait's, or the one after the handler returned.
The flag is a volatile field, since a thread disposing the worker may set it. It is shown by ToString when true and stays out of HostEvidence, which keeps the shared evidence schema and stays empty on success.
On .NET 10 the full suite saw System.Security.Claims.dll opened during the resource test's loop, after the added spawn-failure test ran first: the runtime loads an assembly when it first needs it and keeps the file open. The counts now skip descriptors on .dll files, which the adapter never opens.
…and 99 live tests The README and EVIDENCE now say that a failure to set up the spawned worker's pipes kills it through the common kill path, after the stop notice, with a bounded exit wait; that every kill has a notice of its own; and that HandlerTokenNotCancelled is shown by ToString and kept out of HostEvidence. EVIDENCE records which new tests fail against 4e32b9f with only the seams added, the two kill-order mutations, and 50 clean runs of the kill-order and stop-notice tests on each runtime.
… descriptor counts
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an independent .NET implementation of Core's pinned
jacquard-host-v0protocol indotnet/. The package isJacquardLang.Host, it targets .NET 8 and is tested on .NET 8 and 10. It includes:jacquard host worker;Hostwith a closed operation registry;It has no runtime dependencies beyond the .NET base class library and one P/Invoke to libc
poll(2). NUnit is used at test scope only.long, never throughdoubleordecimal. Money crosses as integer minor units, asdotnet/README.mdsays.scripts/core_pin.pytracks it and generatesCompat.cs. Replay passes all 21 recordings, both through the stand-in worker and against Core 0.3.0.dotnetjob runs the offline suite on .NET 8, pinned to the .NET 8 SDK, and on .NET 10.live-examplesruns the live suite on both runtimes against the pinned Core with live tests required. Locally that is 101 tests on each runtime, none skipped..NET differs from the other adapters in ways that
docs/LIMITS.mdanddotnet/README.mdrecord:poll(2): .NET's pipe reads and writes cannot be interrupted. So the carrier never makes a call that can block: the caller's thread polls, then reads only bytes already in the pipe, or writes at mostPIPE_BUF. There is no reader or stdin thread. Workers run on Linux and macOS only, and macOS has no CI.Process.Kill(true). That call finds and signals descendants by pid, so it misses a descendant that has already been reparented. If a descendant still holds stdout or stderr after the worker exits, the host reports a carrier loss, as Go and Rust do.CancellationToken. When the call stops, the caller's thread starts the token's cancellation before any kill. It runs on a dedicated thread, so a callback that blocks or throws cannot hold or crashInvoke, and the cancellation does not depend on the thread pool. If no thread can be started, it falls back to a pool work item. If that also fails, it gives up and setsInvocationResult.HandlerTokenNotCancelled.Every wait observes the one lifetime deadline, the caller's
CancellationTokenand disposal. Deadlines up toTimeSpan.MaxValueare accepted. Frame encoding stops atmax_frame_bytes, and text validation is charged against the frame. No handler starts after a stop, and no late response is sent. Kit replay keeps at most one frame beyond the recording. Dispatch uses the same registry snapshot the invocation advertised.Each of these has a regression test, and each test is shown failing on the code before its fix, by an old-code run or by mutation.
docs/EVIDENCE.mdlists them. The descriptor and thread tests check, on every path, that nothing leaks afterInvokereturns. The paths are success, deadline, cancellation, truncated frames, spawn failures, escaped descendants and output floods.The SDK layer follows in a separate PR.