From cac6a2ad5c9291bb8a8e745c00f8ff03c5325e25 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 01:22:23 -0400 Subject: [PATCH 01/33] Add an independent .NET adapter A .NET 8 library in dotnet/src/JacquardHost (package JacquardLang.Host, no runtime dependencies beyond the base class library) implementing jacquard-host-v0: a strict hand-written JSON codec that parses integers exactly from their text as a long, the v0 value and type codecs, limits, the session state machine, a bounded process carrier for `jacquard host worker`, a Host with a closed operation registry, and replay of the pinned conformance kit, carried as embedded resources. .NET cannot start the worker in its own process group, and its pipe streams have no read or write a deadline or cancellation can interrupt. So the carrier never calls one that can block: the caller's thread waits on the C library's poll(2), then reads only bytes already in the pipe or writes at most PIPE_BUF bytes. Every wait rechecks one lifetime deadline, the CancellationToken, and disposal. Expiry kills the worker's process tree with Process.Kill(true), and a descendant that keeps stdout or stderr open after the worker exits is reported as a carrier loss. Frame encoding counts UTF-8 bytes as it writes and stops at max_frame_bytes; text validation refuses a text longer than max_text_bytes before scanning it and charges every text against the frame; a caller's raw map is bounded before it is copied or sorted. No handler starts after a stop: the caller checks after decoding a request, and the handler thread checks again before calling it. Replay keeps at most one frame past each recording's. core_pin.py generates Compat.cs and keeps the .NET kit copy in sync, with its tests extended. NOTICE names the new copy, and every adapter's NOTICE copy stays identical to it. --- NOTICE | 7 +- dotnet/.editorconfig | 8 + dotnet/.gitignore | 3 + dotnet/Directory.Build.props | 17 + dotnet/LICENSE | 202 ++ dotnet/NOTICE | 15 + .../JacquardHost/Carrier/CarrierException.cs | 45 + dotnet/src/JacquardHost/Carrier/Posix.cs | 77 + dotnet/src/JacquardHost/Carrier/Worker.cs | 809 +++++ .../src/JacquardHost/Carrier/WorkerOptions.cs | 64 + dotnet/src/JacquardHost/Compat.cs | 36 + dotnet/src/JacquardHost/Conformance/Kit.cs | 221 ++ .../src/JacquardHost/Conformance/Replayer.cs | 386 +++ dotnet/src/JacquardHost/Handlers.cs | 126 + dotnet/src/JacquardHost/Host.cs | 518 +++ dotnet/src/JacquardHost/InvocationResult.cs | 190 + dotnet/src/JacquardHost/JacquardHost.csproj | 33 + .../src/JacquardHost/Protocol/BoundaryType.cs | 159 + dotnet/src/JacquardHost/Protocol/Budget.cs | 77 + dotnet/src/JacquardHost/Protocol/Codec.cs | 705 ++++ dotnet/src/JacquardHost/Protocol/Frames.cs | 218 ++ dotnet/src/JacquardHost/Protocol/Hash.cs | 68 + dotnet/src/JacquardHost/Protocol/Json.cs | 395 +++ dotnet/src/JacquardHost/Protocol/JsonCodec.cs | 695 ++++ dotnet/src/JacquardHost/Protocol/Limits.cs | 118 + dotnet/src/JacquardHost/Protocol/Session.cs | 271 ++ dotnet/src/JacquardHost/Protocol/Value.cs | 169 + dotnet/src/JacquardHost/ProtocolException.cs | 33 + .../kit/host-protocol-v0/kit/README.md | 96 + .../kit/host-protocol-v0/kit/fixtures.jac | 21 + .../kit/host-protocol-v0/kit/kit.json | 106 + .../kit/host-protocol-v0/kit/transcripts.json | 3066 +++++++++++++++++ .../kit/host-protocol-v0/vectors.json | 775 +++++ go/NOTICE | 7 +- java/NOTICE | 7 +- rust/NOTICE | 7 +- scripts/core_pin.py | 50 +- tests/test_core_pin.py | 16 +- 38 files changed, 9799 insertions(+), 17 deletions(-) create mode 100644 dotnet/.editorconfig create mode 100644 dotnet/.gitignore create mode 100644 dotnet/Directory.Build.props create mode 100644 dotnet/LICENSE create mode 100644 dotnet/NOTICE create mode 100644 dotnet/src/JacquardHost/Carrier/CarrierException.cs create mode 100644 dotnet/src/JacquardHost/Carrier/Posix.cs create mode 100644 dotnet/src/JacquardHost/Carrier/Worker.cs create mode 100644 dotnet/src/JacquardHost/Carrier/WorkerOptions.cs create mode 100644 dotnet/src/JacquardHost/Compat.cs create mode 100644 dotnet/src/JacquardHost/Conformance/Kit.cs create mode 100644 dotnet/src/JacquardHost/Conformance/Replayer.cs create mode 100644 dotnet/src/JacquardHost/Handlers.cs create mode 100644 dotnet/src/JacquardHost/Host.cs create mode 100644 dotnet/src/JacquardHost/InvocationResult.cs create mode 100644 dotnet/src/JacquardHost/JacquardHost.csproj create mode 100644 dotnet/src/JacquardHost/Protocol/BoundaryType.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Budget.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Codec.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Frames.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Hash.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Json.cs create mode 100644 dotnet/src/JacquardHost/Protocol/JsonCodec.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Limits.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Session.cs create mode 100644 dotnet/src/JacquardHost/Protocol/Value.cs create mode 100644 dotnet/src/JacquardHost/ProtocolException.cs create mode 100644 dotnet/src/JacquardHost/kit/host-protocol-v0/kit/README.md create mode 100644 dotnet/src/JacquardHost/kit/host-protocol-v0/kit/fixtures.jac create mode 100644 dotnet/src/JacquardHost/kit/host-protocol-v0/kit/kit.json create mode 100644 dotnet/src/JacquardHost/kit/host-protocol-v0/kit/transcripts.json create mode 100644 dotnet/src/JacquardHost/kit/host-protocol-v0/vectors.json diff --git a/NOTICE b/NOTICE index 09bc7d2..8de2b4f 100644 --- a/NOTICE +++ b/NOTICE @@ -5,9 +5,10 @@ This product includes conformance artifacts from Jacquard Core, https://github.com/jbwinters/jacquard-lang, licensed under Apache License 2.0. The artifacts under src/jacquard_host/data/conformance/host-protocol-v0/, and their byte-identical copies under go/kit/data/host-protocol-v0/, -rust/kit/host-protocol-v0/, and -java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, are -copied verbatim from the commits recorded in docs/COMPATIBILITY.md. +rust/kit/host-protocol-v0/, +java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, and +dotnet/src/JacquardHost/kit/host-protocol-v0/, are copied verbatim from the +commits recorded in docs/COMPATIBILITY.md. Upstream notice: Jacquard diff --git a/dotnet/.editorconfig b/dotnet/.editorconfig new file mode 100644 index 0000000..f9fd46f --- /dev/null +++ b/dotnet/.editorconfig @@ -0,0 +1,8 @@ +# Analyzer choices for the .NET adapter. Everything else follows +# AnalysisLevel 8.0-recommended with warnings as errors. +[*.cs] +# Hash and OperationEntry implement IComparable only so they sort in protocol +# order; comparison operators would add surface nothing uses. +dotnet_diagnostic.CA1036.severity = none +# JsonObject is named for JSON's object, not as a collection. +dotnet_diagnostic.CA1710.severity = none diff --git a/dotnet/.gitignore b/dotnet/.gitignore new file mode 100644 index 0000000..d78675a --- /dev/null +++ b/dotnet/.gitignore @@ -0,0 +1,3 @@ +bin/ +obj/ +*.binlog diff --git a/dotnet/Directory.Build.props b/dotnet/Directory.Build.props new file mode 100644 index 0000000..17cb152 --- /dev/null +++ b/dotnet/Directory.Build.props @@ -0,0 +1,17 @@ + + + 12.0 + enable + disable + true + 8 + 8.0-recommended + false + true + true + false + 0.2.0 + Jacquard Host contributors + Copyright 2026 Jacquard Host contributors + + diff --git a/dotnet/LICENSE b/dotnet/LICENSE new file mode 100644 index 0000000..d645695 --- /dev/null +++ b/dotnet/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/dotnet/NOTICE b/dotnet/NOTICE new file mode 100644 index 0000000..8de2b4f --- /dev/null +++ b/dotnet/NOTICE @@ -0,0 +1,15 @@ +Jacquard Host +Copyright 2026 Jacquard Host contributors + +This product includes conformance artifacts from Jacquard Core, +https://github.com/jbwinters/jacquard-lang, licensed under Apache License 2.0. +The artifacts under src/jacquard_host/data/conformance/host-protocol-v0/, +and their byte-identical copies under go/kit/data/host-protocol-v0/, +rust/kit/host-protocol-v0/, +java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, and +dotnet/src/JacquardHost/kit/host-protocol-v0/, are copied verbatim from the +commits recorded in docs/COMPATIBILITY.md. + +Upstream notice: +Jacquard +Copyright 2026 Josh Winters diff --git a/dotnet/src/JacquardHost/Carrier/CarrierException.cs b/dotnet/src/JacquardHost/Carrier/CarrierException.cs new file mode 100644 index 0000000..a79214b --- /dev/null +++ b/dotnet/src/JacquardHost/Carrier/CarrierException.cs @@ -0,0 +1,45 @@ +using System; + +namespace JacquardLang.Host.Carrier; + +/// +/// A carrier failure: either a loss (the worker could not start, closed a +/// pipe, exited unsuccessfully, was aborted, or outlived its deadline) or a +/// frame that violated the protocol. +/// +public sealed class CarrierException : Exception +{ + private CarrierException(string message, bool timeout, bool aborted, ProtocolException? protocolError) + : base(message, protocolError) + { + IsTimeout = timeout; + IsAborted = aborted; + ProtocolError = protocolError; + } + + /// Whether this is a loss rather than a protocol violation. + public bool IsLoss => ProtocolError is null; + + /// Whether the worker outlived its lifetime deadline. + public bool IsTimeout { get; } + + /// Whether the invocation was cancelled or the worker disposed. + public bool IsAborted { get; } + + /// The protocol violation, or null for a loss. + public ProtocolException? ProtocolError { get; } + + /// A carrier loss. + public static CarrierException Loss(string message) => new(message, false, false, null); + + /// A frame that violated the protocol. + public static CarrierException FromProtocol(ProtocolException error) + { + ArgumentNullException.ThrowIfNull(error); + return new CarrierException(error.Message, false, false, error); + } + + internal static CarrierException Timeout(string message) => new(message, true, false, null); + + internal static CarrierException Aborted(string message) => new(message, false, true, null); +} diff --git a/dotnet/src/JacquardHost/Carrier/Posix.cs b/dotnet/src/JacquardHost/Carrier/Posix.cs new file mode 100644 index 0000000..3e52049 --- /dev/null +++ b/dotnet/src/JacquardHost/Carrier/Posix.cs @@ -0,0 +1,77 @@ +using System; +using System.Runtime.InteropServices; + +namespace JacquardLang.Host.Carrier; + +/// +/// The C library's poll(2), the one native call the carrier makes. It +/// lets the caller's thread wait for a pipe to become readable or writable +/// with a timeout, so no read or write on a worker pipe ever blocks. The +/// pollfd layout and the event bits used here are the same on Linux +/// and macOS. +/// +internal static partial class Posix +{ + internal const short PollIn = 0x0001; + internal const short PollOut = 0x0004; + internal const short PollErr = 0x0008; + internal const short PollHup = 0x0010; + internal const short PollNval = 0x0020; + + private const int Eintr = 4; + + /// Whether the carrier can run here: Linux or macOS. + internal static bool Supported => OperatingSystem.IsLinux() || OperatingSystem.IsMacOS(); + + /// + /// The most bytes one write may carry so that it cannot block once poll + /// reports the pipe writable: PIPE_BUF, the space POLLOUT + /// guarantees (one page on Linux, 512 bytes on macOS). + /// + internal static int WriteChunk => OperatingSystem.IsMacOS() ? 512 : 4096; + + [StructLayout(LayoutKind.Sequential)] + internal struct PollFd + { + public int Fd; + public short Events; + public short Revents; + } + + [LibraryImport("libc", EntryPoint = "poll", SetLastError = true)] + private static unsafe partial int NativePoll(PollFd* fds, nuint count, int timeout); + + /// + /// Polls for up to + /// milliseconds (0 returns at once). A signal interrupting the wait counts + /// as a timeout; the caller rechecks its state and polls again. + /// + /// The number of descriptors with events. + /// poll failed for another reason; the worker is lost. + internal static unsafe int Poll(Span fds, int timeoutMs) + { + for (int i = 0; i < fds.Length; i++) + { + fds[i].Revents = 0; + } + + int result; + fixed (PollFd* pointer = fds) + { + result = NativePoll(pointer, (nuint)fds.Length, timeoutMs); + } + + if (result >= 0) + { + return result; + } + + int error = Marshal.GetLastPInvokeError(); + if (error == Eintr) + { + return 0; + } + + throw CarrierException.Loss("could not wait on the worker's pipes (poll failed with errno " + error.ToString(System.Globalization.CultureInfo.InvariantCulture) + ")"); + } +} diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs new file mode 100644 index 0000000..15b8544 --- /dev/null +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -0,0 +1,809 @@ +using System; +using System.Buffers.Binary; +using System.Collections.Generic; +using System.ComponentModel; +using System.Diagnostics; +using System.IO; +using System.IO.Pipes; +using System.Threading; +using JacquardLang.Host.Protocol; + +namespace JacquardLang.Host.Carrier; + +/// +/// One bounded stdio-u32-json-v0 worker lifetime. +/// +/// +/// +/// .NET cannot start a process in its own process group, and its pipe +/// streams have no read or write that a timeout or a cancellation can +/// interrupt (on Unix an anonymous pipe's asynchronous read is a blocking +/// read on a pool thread). So the carrier never calls a read or write that +/// can block. It asks the C library's poll(2) whether a pipe is ready, +/// on the caller's thread, in waits of at most 5 ms, and only then reads the +/// bytes already in the pipe or writes at most PIPE_BUF bytes, which +/// a writable pipe always accepts at once. Every wait, whether for a frame, a +/// write, a handler, or the exit, rechecks the single lifetime deadline, the +/// , and disposal. No thread is ever left +/// blocked on a worker pipe. +/// +/// +/// .NET's does not read redirected output on its own +/// unless asked to (BeginOutputReadLine, which this carrier never +/// calls), so nothing else holds or drains the pipes. +/// +/// +/// At expiry, cancellation, or disposal while the worker runs, the worker's +/// process tree is killed with : .NET stops +/// the worker, lists its children from the process table, kills it, and +/// does the same for each child. A descendant that has already been +/// reparented (because its parent exited or it daemonized) is not found and +/// keeps running. +/// +/// +/// The runtime reaps the worker itself when it exits. After the exit, a +/// descendant that still holds stdout or stderr is detected (the pipe has a +/// writer but the worker has gone): the carrier waits up to one second, then +/// reports a loss. It cannot stop that descendant. +/// +/// A worker is used from one thread at a time; may come from any thread. +/// +public sealed class Worker : IDisposable +{ + /// How long disposal waits for a killed worker to exit, and how long a pipe may stay held after the exit. + private const int ExitGraceMs = 1000; + + /// The longest single wait between checks. + private const int PollSliceMs = 5; + + /// The most stderr one drain reads, so a fast writer cannot hold the caller. + private const int StderrSlice = 1 << 20; + + /// The exit status .NET reports for a process ended by SIGKILL. + private const int SigkillExit = 128 + 9; + + private const short Ready = Posix.PollIn | Posix.PollHup | Posix.PollErr | Posix.PollNval; + + private readonly object gate = new(); + private readonly Process? process; + private readonly StreamWriter? stdinWriter; + private readonly Stream? stdin; + private readonly Stream? stdout; + private readonly Stream? stderr; + private readonly int stdinFd = -1; + private readonly int stdoutFd = -1; + private readonly int stderrFd = -1; + private readonly long expires; + private readonly CancellationToken cancel; + private readonly byte[] chunk = new byte[8192]; + private volatile bool expired; + private volatile bool disposeRequested; + private byte[] stderrBytes = Array.Empty(); + private int stderrLength; + private int stderrLimit = Limits.Hard.MaxStderrBytes; + private bool stderrEnded; + private CarrierException? fault; + private bool inputClosed; + private bool killed; + private bool exitRecorded; + private bool disposed; + + private Worker(long expires, string? spawnError, CancellationToken cancel) + { + this.expires = expires; + this.cancel = cancel; + SpawnError = spawnError; + } + + private Worker(Process process, long expires, CancellationToken cancel) + : this(expires, null, cancel) + { + this.process = process; + stdinWriter = process.StandardInput; + stdin = stdinWriter.BaseStream; + stdout = process.StandardOutput.BaseStream; + stderr = process.StandardError.BaseStream; + stdinFd = Descriptor(stdin); + stdoutFd = Descriptor(stdout); + stderrFd = Descriptor(stderr); + } + + /// The worker lifetime when is null. + public static TimeSpan DefaultDeadline { get; } = TimeSpan.FromSeconds(30); + + /// The spawn error, when the worker never started. + public string? SpawnError { get; } + + /// The exit status once the worker has exited, unless the host's SIGKILL ended it. + public int? ExitStatus { get; private set; } + + /// + /// "SIGKILL" when the host killed the worker and .NET reported exit + /// status 137 (128 + 9); otherwise null. .NET reports any other signal as + /// exit status 128 + its number. + /// + public string? Signal { get; private set; } + + /// Whether the worker's lifetime ended by expiry, cancellation, or disposal. + public bool IsExpired => expired; + + /// + /// Whether the call is cancelled, the worker expired or is being disposed, + /// or its lifetime passed. Safe from any thread, and exact: it does not + /// wait for the caller's next poll to notice the deadline. + /// + public bool Stopping => expired || disposeRequested || cancel.IsCancellationRequested || Stopwatch.GetTimestamp() >= expires; + + /// The retained stderr prefix. + public byte[] Stderr + { + get + { + lock (gate) + { + return stderrBytes.AsSpan(0, stderrLength).ToArray(); + } + } + } + + /// + /// Spawns COMMAND host worker --store STORE. A spawn failure is + /// reported by the first operation. + /// + /// The options are invalid. + public static Worker Start(string store, WorkerOptions options, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(store); + ArgumentNullException.ThrowIfNull(options); + options.Validate(); + TimeSpan deadline = options.Deadline ?? DefaultDeadline; + + // Clamp to about 73 years so the deadline in timestamp ticks cannot overflow. + double ticks = Math.Min(deadline.TotalSeconds, 73 * 365.25 * 86400) * Stopwatch.Frequency; + long expires = Stopwatch.GetTimestamp() + (long)ticks; + if (!Posix.Supported) + { + return Failed(expires, "unsupported platform", "the .NET host runs workers on Linux and macOS only", cancellationToken); + } + + if (Stopped(expires, cancellationToken)) + { + return StoppedBeforeSpawn(expires, cancellationToken); + } + + IReadOnlyList command = options.ResolvedCommand(); + var info = new ProcessStartInfo(command[0]) + { + UseShellExecute = false, + RedirectStandardInput = true, + RedirectStandardOutput = true, + RedirectStandardError = true, + CreateNoWindow = true, + }; + for (int i = 1; i < command.Count; i++) + { + info.ArgumentList.Add(command[i]); + } + + foreach (string argument in new[] { "host", "worker", "--store", store }) + { + info.ArgumentList.Add(argument); + } + + if (options.Environment is { } environment) + { + info.Environment.Clear(); + foreach (KeyValuePair variable in environment) + { + info.Environment[variable.Key] = variable.Value; + } + } + + Process process; + try + { + process = Process.Start(info) ?? throw new InvalidOperationException("no process was started"); + } + catch (Exception error) when (error is Win32Exception or InvalidOperationException or IOException or PlatformNotSupportedException) + { + return Failed( + expires, + error.Message, + "could not start " + command[0] + " (" + error.Message + "); check the configured Core installation", + cancellationToken); + } + + try + { + return new Worker(process, expires, cancellationToken); + } + catch (Exception error) when (error is InvalidOperationException or NotSupportedException) + { + // The streams are not the pipes this carrier polls. + KillQuietly(process); + process.Dispose(); + return Failed(expires, error.Message, "could not poll the worker's pipes (" + error.Message + ")", cancellationToken); + } + } + + private static int Descriptor(Stream stream) => + stream is PipeStream pipe + ? (int)pipe.SafePipeHandle.DangerousGetHandle() + : throw new NotSupportedException("worker pipes are " + stream.GetType().Name + ", not pipe streams"); + + private static bool Stopped(long expires, CancellationToken cancel) => cancel.IsCancellationRequested || Stopwatch.GetTimestamp() >= expires; + + private static Worker Failed(long expires, string spawnError, string message, CancellationToken cancel) => + new(expires, spawnError, cancel) { fault = CarrierException.Loss(message), exitRecorded = true }; + + /// + /// A worker that was never spawned because the call was cancelled, or its + /// lifetime ended, first. Its first operation reports the abort or the + /// deadline, as for a running worker. + /// + private static Worker StoppedBeforeSpawn(long expires, CancellationToken cancel) + { + var stopped = new Worker(expires, null, cancel) { exitRecorded = true }; + stopped.Fault(); + return stopped; + } + + /// The carrier failure that ended the worker, if any; null otherwise. + public CarrierException? Fault() + { + lock (gate) + { + return FaultCore(); + } + } + + private CarrierException? FaultCore() + { + if (fault is null) + { + if (cancel.IsCancellationRequested || disposeRequested) + { + Expire(); + fault = CarrierException.Aborted("invocation aborted; outside completion is unknown"); + } + else if (Stopwatch.GetTimestamp() >= expires) + { + Expire(); + fault = CarrierException.Timeout("worker lifetime deadline exceeded"); + } + } + + return fault; + } + + private void ThrowIfFault() + { + CarrierException? current = FaultCore(); + if (current is not null) + { + throw current; + } + } + + private CarrierException FaultOr(CarrierException fallback) => FaultCore() ?? fallback; + + /// Kills the worker's process tree and closes its input. + private void Expire() + { + expired = true; + KillTree(); + CloseInputCore(); + } + + private bool HasExited() + { + try + { + return process is null || process.HasExited; + } + catch (InvalidOperationException) + { + return true; + } + } + + /// Kills the worker and the descendants .NET finds, while the worker is alive. + private void KillTree() + { + if (process is null || HasExited()) + { + return; + } + + killed = true; + KillQuietly(process); + } + + private static void KillQuietly(Process process) + { + try + { + process.Kill(entireProcessTree: true); + } + catch (Exception error) when (error is InvalidOperationException or Win32Exception or AggregateException or NotSupportedException) + { + // Already gone, or a descendant could not be signalled. + } + } + + private static int RemainingMs(long expires) + { + long left = expires - Stopwatch.GetTimestamp(); + if (left <= 0) + { + return 0; + } + + long ms = (left * 1000 / Stopwatch.Frequency) + 1; + return (int)Math.Min(ms, PollSliceMs); + } + + /// + /// Waits up to one slice, never past the deadline, for + /// to report (or for nothing when it is -1), + /// reading whatever stderr arrives meanwhile. + /// + private void Pause(int fd, short events) + { + Span fds = stackalloc Posix.PollFd[2]; + int count = 0; + if (fd >= 0) + { + fds[count++] = new Posix.PollFd { Fd = fd, Events = events }; + } + + bool watchStderr = stderr is not null && !stderrEnded; + if (watchStderr) + { + fds[count++] = new Posix.PollFd { Fd = stderrFd, Events = Posix.PollIn }; + } + + int timeout = RemainingMs(expires); + if (count == 0) + { + Thread.Sleep(timeout); + return; + } + + Posix.Poll(fds[..count], timeout); + if (watchStderr && (fds[count - 1].Revents & Ready) != 0) + { + DrainStderr(); + } + } + + private static short Events(int fd, short events) + { + Span fds = stackalloc Posix.PollFd[1]; + fds[0] = new Posix.PollFd { Fd = fd, Events = events }; + Posix.Poll(fds, 0); + return fds[0].Revents; + } + + /// Selects how much stderr is retained. + public void SetStderrLimit(int limit) + { + lock (gate) + { + stderrLimit = Math.Max(0, limit); + stderrLength = Math.Min(stderrLength, stderrLimit); + } + } + + /// Reads whatever stderr is in the pipe, keeping a bounded prefix. Never blocks. + private void DrainStderr() + { + if (stderr is null || stderrEnded) + { + return; + } + + try + { + for (int budget = StderrSlice; budget > 0;) + { + if ((Events(stderrFd, Posix.PollIn) & Ready) == 0) + { + return; + } + + int n = stderr.Read(chunk, 0, chunk.Length); + if (n <= 0) + { + stderrEnded = true; + return; + } + + int keep = Math.Min(n, stderrLimit - stderrLength); + if (keep > 0) + { + if (stderrLength + keep > stderrBytes.Length) + { + Array.Resize(ref stderrBytes, Math.Min(stderrLimit, Math.Max(stderrLength + keep, stderrBytes.Length * 2))); + } + + Array.Copy(chunk, 0, stderrBytes, stderrLength, keep); + stderrLength += keep; + } + + budget -= n; + } + } + catch (Exception error) when (error is IOException or ObjectDisposedException or CarrierException) + { + stderrEnded = true; + } + } + + /// + /// One non-blocking step on stdout: bytes read, 0 when none are ready yet, + /// -1 at end of stream (every writer closed it), or -2 when the worker has + /// exited but another process still holds stdout. The exit is observed + /// before the pipe is checked, so all of the worker's own output has been + /// read when -2 is returned. + /// + private int ReadStep(byte[] buffer, int offset, int length) + { + bool exited = HasExited(); + try + { + if ((Events(stdoutFd, Posix.PollIn) & Ready) != 0) + { + int n = stdout!.Read(buffer, offset, length); + return n <= 0 ? -1 : n; + } + } + catch (Exception error) when (error is IOException or ObjectDisposedException) + { + return -1; + } + + return exited ? -2 : 0; + } + + /// Fills ; returns its length, or ~filled at end of stream. + private int Fill(byte[] buffer) + { + int filled = 0; + long heldSince = 0; + while (filled < buffer.Length) + { + ThrowIfFault(); + int n = ReadStep(buffer, filled, buffer.Length - filled); + if (n > 0) + { + filled += n; + continue; + } + + if (n == -1) + { + return ~filled; + } + + if (n == -2) + { + long now = Stopwatch.GetTimestamp(); + if (heldSince == 0) + { + heldSince = now; + } + else if (now - heldSince >= Stopwatch.Frequency * ExitGraceMs / 1000) + { + fault ??= CarrierException.Loss("worker descendants kept its pipes open after exit"); + CloseInputCore(); + throw fault; + } + } + + Pause(stdoutFd, Posix.PollIn); + } + + return filled; + } + + /// Reads one frame under . Returns null at a clean end of stream. + /// A loss, or a frame that violates the protocol. + public JsonObject? ReadFrame(Limits limits) + { + ArgumentNullException.ThrowIfNull(limits); + lock (gate) + { + ThrowIfFault(); + if (stdout is null) + { + return null; + } + + byte[] prefix = new byte[4]; + int got = Fill(prefix); + if (got < 0) + { + if (~got == 0) + { + return null; + } + + throw CarrierException.Loss("worker closed stdout during a frame"); + } + + uint length = BinaryPrimitives.ReadUInt32BigEndian(prefix); + if (length == 0 || length > limits.MaxFrameBytes) + { + throw CarrierException.FromProtocol(ProtocolException.Exceeded("frame length exceeds max_frame_bytes")); + } + + byte[] payload = new byte[length]; + if (Fill(payload) < 0) + { + throw CarrierException.Loss("worker closed stdout during a frame"); + } + + // A frame that arrives in the final poll slice is not acted on. + ThrowIfFault(); + try + { + return JsonCodec.DecodeObject(payload, limits.MaxJsonDepth); + } + catch (ProtocolException error) + { + throw CarrierException.FromProtocol(error); + } + } + } + + /// + /// Writes raw carrier bytes. Each write carries at most PIPE_BUF + /// bytes and starts only once poll reports the pipe writable, so it never + /// blocks; between writes the caller waits under the lifetime. + /// + /// A loss: the input is closed, the write failed, or the lifetime ended. + public void WriteBytes(ReadOnlySpan bytes) + { + lock (gate) + { + ThrowIfFault(); + if (stdin is null || inputClosed) + { + throw CarrierException.Loss("worker write failed (worker input is closed)"); + } + + int chunkSize = Posix.WriteChunk; + int offset = 0; + while (offset < bytes.Length) + { + ThrowIfFault(); + if ((Events(stdinFd, Posix.PollOut) & (Posix.PollOut | Posix.PollErr | Posix.PollHup | Posix.PollNval)) == 0) + { + Pause(stdinFd, Posix.PollOut); + continue; + } + + int n = Math.Min(chunkSize, bytes.Length - offset); + try + { + stdin.Write(bytes.Slice(offset, n)); + } + catch (Exception error) when (error is IOException or ObjectDisposedException) + { + throw FaultOr(CarrierException.Loss("worker write failed (" + error.Message + ")")); + } + + offset += n; + } + + ThrowIfFault(); + } + } + + /// Encodes and writes one frame under . + /// A frame that cannot be encoded within the limits, or a write failure. + public void WriteFrame(IReadOnlyDictionary frame, Limits limits) + { + ArgumentNullException.ThrowIfNull(frame); + ArgumentNullException.ThrowIfNull(limits); + lock (gate) + { + ThrowIfFault(); + byte[] bytes; + try + { + bytes = JsonCodec.FrameBytes(frame, limits.MaxFrameBytes, limits.MaxJsonDepth); + } + catch (ProtocolException error) + { + throw CarrierException.FromProtocol(error); + } + + WriteBytes(bytes); + } + } + + /// + /// Waits until returns true, the deadline + /// passes, the call is cancelled, or the worker is disposed, reading + /// stderr meanwhile. + /// + /// The lifetime ended first. + public void Await(Func isDone) + { + ArgumentNullException.ThrowIfNull(isDone); + lock (gate) + { + while (true) + { + ThrowIfFault(); + if (isDone()) + { + return; + } + + Pause(-1, 0); + } + } + } + + /// The time left in the lifetime, or zero. + internal TimeSpan Remaining => TimeSpan.FromSeconds(Math.Max(0, expires - Stopwatch.GetTimestamp()) / (double)Stopwatch.Frequency); + + /// Closes the worker's stdin. Nothing is writing, so this never blocks. + public void CloseInput() + { + lock (gate) + { + CloseInputCore(); + } + } + + private void CloseInputCore() + { + if (inputClosed) + { + return; + } + + inputClosed = true; + try + { + stdinWriter?.Dispose(); + } + catch (Exception error) when (error is IOException or ObjectDisposedException) + { + // The worker may already have closed its end. + } + } + + /// + /// Closes stdin and waits for the worker to exit, bounded by the deadline. + /// Call only after stdout reached end of stream. After the exit, stderr + /// must close within one second: a descendant that keeps holding it is + /// reported as a loss. + /// + /// The lifetime ended first, or a descendant kept stderr open. + public void AwaitExit() + { + lock (gate) + { + CloseInputCore(); + if (process is not null) + { + while (!HasExited() && FaultCore() is null) + { + Pause(-1, 0); + } + + if (fault is not null) + { + // FaultCore killed the worker; wait briefly so the exit is recorded. + WaitForExit(ExitGraceMs); + } + + RecordExit(); + DrainStderr(); + AwaitStderrClose(); + } + + ThrowIfFault(); + } + } + + /// Waits up to one second, within the lifetime, for every writer to close stderr. + private void AwaitStderrClose() + { + long until = Stopwatch.GetTimestamp() + (Stopwatch.Frequency * ExitGraceMs / 1000); + while (fault is null && stderr is not null && !stderrEnded) + { + if (FaultCore() is not null) + { + return; + } + + if (Stopwatch.GetTimestamp() >= until) + { + fault = CarrierException.Loss("worker descendants kept its pipes open after exit"); + return; + } + + Pause(-1, 0); + DrainStderr(); + } + } + + /// Waits up to for the worker to exit, regardless of the deadline. + private void WaitForExit(int milliseconds) + { + try + { + process?.WaitForExit(milliseconds); + } + catch (Exception error) when (error is InvalidOperationException or Win32Exception) + { + // No process to wait for. + } + } + + private void RecordExit() + { + if (exitRecorded || process is null || !HasExited()) + { + return; + } + + exitRecorded = true; + int value = process.ExitCode; + if (killed && value == SigkillExit) + { + Signal = "SIGKILL"; + } + else + { + ExitStatus = value; + } + } + + /// + /// Kills a live worker's process tree, waits up to one second for its + /// exit, reads the stderr left in the pipe, and closes the pipes. Safe + /// from any thread: an operation in progress on another thread sees the + /// disposal within one poll slice and ends first. + /// + public void Dispose() + { + disposeRequested = true; + lock (gate) + { + if (disposed) + { + return; + } + + disposed = true; + CloseInputCore(); + if (process is null) + { + return; + } + + if (!HasExited()) + { + KillTree(); + WaitForExit(ExitGraceMs); + } + + RecordExit(); + DrainStderr(); + + // Closing the host's ends leaves a descendant still writing with + // no reader, so its next write fails. + stdout?.Dispose(); + stderr?.Dispose(); + stderrEnded = true; + process.Dispose(); + } + } +} diff --git a/dotnet/src/JacquardHost/Carrier/WorkerOptions.cs b/dotnet/src/JacquardHost/Carrier/WorkerOptions.cs new file mode 100644 index 0000000..aee13f5 --- /dev/null +++ b/dotnet/src/JacquardHost/Carrier/WorkerOptions.cs @@ -0,0 +1,64 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace JacquardLang.Host.Carrier; + +/// Options for one worker lifetime. +public sealed record WorkerOptions +{ + private readonly string[] command = Array.Empty(); + private readonly Dictionary? environment; + + /// The default command, inherited environment, and default deadline. + public static WorkerOptions Defaults { get; } = new(); + + /// + /// The Core executable and leading arguments; host worker --store DIR + /// is appended. Empty means $JACQUARD_BIN, then jacquard. + /// + public IReadOnlyList Command + { + get => command; + init => command = (value ?? throw new ArgumentNullException(nameof(value))).ToArray(); + } + + /// Replaces the inherited environment when not null. + public IReadOnlyDictionary? Environment + { + get => environment; + init => environment = value is null ? null : new Dictionary(value, StringComparer.Ordinal); + } + + /// + /// Bounds the whole lifetime: spawn, pipes, handlers, and exit; null + /// means . + /// + public TimeSpan? Deadline { get; init; } + + /// Refuses an empty or null command element or a deadline that is not positive. + /// The options are invalid. + public void Validate() + { + if (command.Any(string.IsNullOrEmpty)) + { + throw new ArgumentException("command must be a nonempty executable argument list", nameof(Command)); + } + + if (Deadline is { } deadline && deadline <= TimeSpan.Zero) + { + throw new ArgumentException("deadline must be positive", nameof(Deadline)); + } + } + + internal IReadOnlyList ResolvedCommand() + { + if (command.Length > 0) + { + return command; + } + + string? bin = System.Environment.GetEnvironmentVariable("JACQUARD_BIN"); + return new[] { string.IsNullOrEmpty(bin) ? "jacquard" : bin }; + } +} diff --git a/dotnet/src/JacquardHost/Compat.cs b/dotnet/src/JacquardHost/Compat.cs new file mode 100644 index 0000000..40a4e54 --- /dev/null +++ b/dotnet/src/JacquardHost/Compat.cs @@ -0,0 +1,36 @@ +// Generated from core-pin.json by scripts/core_pin.py; do not edit by hand. + +using System.Collections.Generic; + +namespace JacquardLang.Host; + +/// The Jacquard Core release and host protocol this adapter is pinned to. +public static class Compat +{ + /// The host protocol. + public const string Protocol = "jacquard-host-v0"; + + /// The carrier. + public const string Carrier = "stdio-u32-json-v0"; + + /// The pinned Core version. + public const string CoreVersion = "0.3.0"; + + /// The pinned Core release tag. + public const string CoreTag = "jacquard-core-0.3.0"; + + /// The pinned Core commit. + public const string CoreCommit = "daf87fb4837cc69339116746818f64821dc4cb8e"; + + /// SHA-256 digests of the pinned conformance kit files. + public static IReadOnlyList> KitDigests { get; } = new KeyValuePair[] + { + new("kit/kit.json", "a01b9401d7c2df4e17771b9c0d22f9ea5ce1a923dfb7d3396c27821c0e058a61"), + new("kit/transcripts.json", "ebaf590e3d1b7e8399f56de5ad85043ac3658416dd7f0af7b55b9f9f3ce93240"), + new("kit/fixtures.jac", "037bb8209ca0344ba0a2cd0fcc16e69f13c8f9b91e3404dc3f3161e351831d80"), + new("vectors.json", "9ba1bff8eee0b6dbea6475dc0c7e9cc6313d847ad80bd8d832457115eb1c738a"), + }; + + /// Decisions Core recorded as pending; replay reproduces them. + public static IReadOnlyList PendingDecisions { get; } = new[] { "noncanonical-target-hash" }; +} diff --git a/dotnet/src/JacquardHost/Conformance/Kit.cs b/dotnet/src/JacquardHost/Conformance/Kit.cs new file mode 100644 index 0000000..ad5ae53 --- /dev/null +++ b/dotnet/src/JacquardHost/Conformance/Kit.cs @@ -0,0 +1,221 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography; +using JacquardLang.Host.Protocol; + +namespace JacquardLang.Host.Conformance; + +/// The pinned kit could not be loaded or verified. +public sealed class KitException : Exception +{ + /// A failure with its reason. + public KitException(string message) + : base(message) + { + } +} + +/// One host-side action in a recording. +/// "core_to_host" for a read, else a host write. +/// Raw carrier bytes to write, or null. +/// The frame to write; empty for reads and raw writes. +/// Whether the recording sends this after Core's terminal. +public sealed record TranscriptStep(string Direction, string? RawHex, JsonObject Frame, bool AfterTerminal); + +/// One positive or hostile recording. +/// The recording's name. +/// "positive" or a hostile kind. +/// The phase a hostile input targets. +/// The recorded expectations. +/// The recorded summary. +/// "conforms" or "diverges". +/// The worker's recorded exit status. +/// Core's recorded frames. +/// The host's recorded actions. +public sealed record Transcript( + string Name, + string Kind, + string Phase, + JsonObject Expect, + JsonObject Observed, + string Status, + long ExitStatus, + IReadOnlyList CoreOutput, + IReadOnlyList HostInput); + +/// +/// The pinned host-protocol-v0 conformance kit, carried as embedded +/// resources and verified against the pinned digests when loaded. +/// +public sealed class Kit +{ + /// The number of recordings in the pinned kit. + public const int TranscriptCount = 21; + + private static readonly string[] Files = { "kit/kit.json", "kit/transcripts.json", "kit/fixtures.jac", "kit/README.md", "vectors.json" }; + + private readonly Dictionary identities; + + private Kit(JsonObject manifest, IReadOnlyList transcripts, Dictionary identities) + { + Manifest = manifest; + Transcripts = transcripts; + this.identities = identities; + } + + /// The decoded kit/kit.json. + public JsonObject Manifest { get; } + + /// The recordings, in kit order. + public IReadOnlyList Transcripts { get; } + + /// The kit's named identities, such as "kit-echo". + public IReadOnlyDictionary Identities => identities; + + /// One pinned kit file, such as "kit/fixtures.jac", or null. + public static byte[]? File(string name) + { + if (!Files.Contains(name, StringComparer.Ordinal)) + { + return null; + } + + using Stream? stream = typeof(Kit).Assembly.GetManifestResourceStream("host-protocol-v0/" + name); + if (stream is null) + { + return null; + } + + using var copy = new MemoryStream(); + stream.CopyTo(copy); + return copy.ToArray(); + } + + private static string Text(JsonObject obj, string key) => obj.Get(key)?.AsString() ?? string.Empty; + + private static Json DecodeFile(string name) + { + byte[] data = File(name) ?? throw new KitException("missing kit file " + name); + try + { + return JsonCodec.Decode(data, 256); + } + catch (ProtocolException error) + { + throw new KitException(name + ": " + error.Message); + } + } + + /// Verifies the embedded files against the pinned digests and decodes them. + /// A file is missing, differs from the pin, or has an unexpected shape. + public static Kit Load() + { + foreach (KeyValuePair entry in Compat.KitDigests) + { + byte[] data = File(entry.Key) ?? throw new KitException("missing kit file " + entry.Key); + if (!string.Equals(Convert.ToHexString(SHA256.HashData(data)), entry.Value, StringComparison.OrdinalIgnoreCase)) + { + throw new KitException("pinned conformance digest mismatch: " + entry.Key); + } + } + + if (DecodeFile("kit/kit.json") is not JsonObject manifest || DecodeFile("kit/transcripts.json") is not JsonArray entries) + { + throw new KitException("kit files have an unexpected shape"); + } + + var pending = new List(); + if (manifest.Get("pending_decisions")?.AsArray() is { } decisions) + { + foreach (Json decision in decisions) + { + pending.Add(decision is JsonObject obj ? Text(obj, "name") : string.Empty); + } + } + + if (!Json.Of(Compat.Protocol).Equals(manifest.Get("protocol")) + || !Json.Of(Compat.Carrier).Equals(manifest.Get("carrier")) + || !Json.Of(Compat.CoreVersion).Equals(manifest.Get("core_version")) + || entries.Items.Count != TranscriptCount + || !pending.SequenceEqual(Compat.PendingDecisions)) + { + throw new KitException("kit metadata does not match the pin"); + } + + var identities = new Dictionary(StringComparer.Ordinal); + if (manifest.Get("identities") is JsonObject named) + { + foreach (KeyValuePair entry in named) + { + try + { + identities[entry.Key] = Hash.Parse(entry.Value.AsString() ?? string.Empty); + } + catch (ProtocolException error) + { + throw new KitException(error.Message); + } + } + } + + return new Kit(manifest, entries.Items.Select(ParseTranscript).ToArray(), identities); + } + + private static List Objects(JsonObject entry, string key, string name) + { + var found = new List(); + if (entry.Get(key)?.AsArray() is not { } list) + { + return found; + } + + foreach (Json item in list) + { + found.Add(item as JsonObject ?? throw new KitException(name + ": " + key + " entry is not an object")); + } + + return found; + } + + private static Transcript ParseTranscript(Json value) + { + JsonObject entry = value as JsonObject ?? throw new KitException("transcript is not an object"); + string name = Text(entry, "name"); + List coreOutput = Objects(entry, "core_output", name); + var hostInput = new List(); + foreach (JsonObject step in Objects(entry, "host_input", name)) + { + hostInput.Add(new TranscriptStep( + Text(step, "direction"), + step.Get("raw_hex")?.AsString(), + step.Get("frame") as JsonObject ?? JsonObject.Empty, + Json.Of(true).Equals(step.Get("after_terminal")))); + } + + if (entry.Get("expect") is not JsonObject expect || entry.Get("observed") is not JsonObject observed || name.Length == 0) + { + throw new KitException("transcript is missing required fields"); + } + + return new Transcript( + name, + Text(entry, "kind"), + Text(entry, "phase"), + expect, + observed, + Text(entry, "status"), + entry.Get("exit_status")?.AsInt() ?? 0, + coreOutput, + hostInput); + } + + /// The transcript with this name, or null. + public Transcript? Find(string name) => Transcripts.FirstOrDefault(transcript => transcript.Name == name); + + /// The named identity. + /// The kit has no identity by that name. + public Hash Identity(string name) => + identities.TryGetValue(name, out Hash? hash) ? hash : throw new ArgumentException("the kit has no identity named " + name, nameof(name)); +} diff --git a/dotnet/src/JacquardHost/Conformance/Replayer.cs b/dotnet/src/JacquardHost/Conformance/Replayer.cs new file mode 100644 index 0000000..ec18354 --- /dev/null +++ b/dotnet/src/JacquardHost/Conformance/Replayer.cs @@ -0,0 +1,386 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.Linq; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Protocol; + +namespace JacquardLang.Host.Conformance; + +/// The observed result of replaying one recording. +/// The recording's name. +/// Whether nothing differed from the recording. +/// "conforms" or "diverges". +/// Semantic differences from the recording. +/// Paths of non-normative diagnostic prose that changed. +/// The observed summary, comparable with the recording's. +/// The frames Core sent, at most one past the recording's count. +/// The worker's exit status, when known. +/// "SIGKILL" when the host killed the worker. +/// The retained stderr prefix. +public sealed record Replay( + string Name, + bool MatchesRecording, + string Status, + IReadOnlyList Differences, + IReadOnlyList ProseDrift, + JsonObject Summary, + IReadOnlyList CoreFrames, + int? ExitStatus, + string? Signal, + IReadOnlyList Stderr); + +/// +/// Drives the kit's recordings, including the intentional hostile host +/// inputs, against a worker through the .NET carrier, and compares what Core +/// sends with each recording, modulo non-normative diagnostic prose. +/// +public static class Replayer +{ + private const string DiagnosticSchema = "jacquard-diagnostic-v1"; + private static readonly string[] Prose = { "summary", "cause", "next_step", "contrast" }; + + private static string? KindOf(JsonObject frame) => frame.Get("kind")?.AsString(); + + private static bool IsTerminal(JsonObject frame) => KindOf(frame) is "outcome" or "fatal" or "shutdown_ack"; + + private static bool IsDiagnostic(JsonObject obj) => Json.Of(DiagnosticSchema).Equals(obj.Get("schema")); + + private static Json StripProse(Json value) => value switch + { + JsonArray array => Json.Of(array.Items.Select(StripProse)), + JsonObject obj => Json.Object(obj + .Where(field => !(IsDiagnostic(obj) && Prose.Contains(field.Key, StringComparer.Ordinal))) + .Select(field => new KeyValuePair(field.Key, StripProse(field.Value)))), + _ => value, + }; + + private static List ProseDriftOf(Json? expected, Json? actual, string path) + { + var drift = new List(); + if (expected is JsonArray items && actual is JsonArray others) + { + for (int i = 0; i < items.Items.Count; i++) + { + drift.AddRange(ProseDriftOf(items.Items[i], i < others.Items.Count ? others.Items[i] : null, path + "/" + i.ToString(CultureInfo.InvariantCulture))); + } + } + else if (expected is JsonObject fields && actual is JsonObject actualFields) + { + if (IsDiagnostic(fields)) + { + foreach (string key in Prose) + { + if (CanonicalOption(fields.Get(key)) != CanonicalOption(actualFields.Get(key))) + { + drift.Add(path + "/" + key); + } + } + } + else + { + foreach (KeyValuePair field in fields) + { + drift.AddRange(ProseDriftOf(field.Value, actualFields.Get(field.Key), path + "/" + field.Key)); + } + } + } + + return drift; + } + + /// Mirrors JavaScript's String() for the JSON values kit.ts coerces. + public static string JsString(Json value) => value switch + { + JsonString text => text.Value, + JsonNull => "null", + JsonBool flag => flag.Value ? "true" : "false", + JsonInt number => number.Value.ToString(CultureInfo.InvariantCulture), + JsonArray array => string.Join(",", array.Items.Select(item => item is JsonNull ? string.Empty : JsString(item))), + JsonObject => "[object Object]", + _ => string.Empty, + }; + + /// + /// Compares decoded frames. Frames are at most max_json_depth deep, + /// so the 256 bound never refuses one; keys are sorted by code point, + /// matching the kit's canonical JSON. + /// + private static string Canonical(Json value) + { + try + { + return JsonCodec.Stringify(value, 256); + } + catch (ProtocolException error) + { + return "unencodable: " + error.Message; + } + } + + /// An absent prose field compares equal to null, as in kit.ts. + private static string CanonicalOption(Json? value) => Canonical(value ?? Json.Null); + + private static JsonArray Frames(IEnumerable list) => Json.Of(list); + + private static byte[] HexBytes(string text) + { + if (text.Length % 2 != 0 || !text.All(Uri.IsHexDigit)) + { + throw new ReplayFailure("raw_hex is not hexadecimal"); + } + + return Convert.FromHexString(text); + } + + private sealed class ReplayFailure : Exception + { + internal ReplayFailure(string message) + : base(message) + { + } + } + + private sealed class Drive + { + /// The most frames kept: one past the recording's, enough to show a divergence. + private readonly int limit; + + internal Drive(int limit) + { + this.limit = limit; + } + + internal List Observed { get; } = new(); + + internal bool Ended { get; private set; } + + internal long Skipped { get; private set; } + + internal Json Refusal { get; private set; } = Json.Null; + + /// + /// Reads one frame. A worker that keeps sending past the recording is + /// not read further, so it cannot fill memory before the deadline; + /// disposal then kills it. + /// + private bool Read(Worker worker) + { + if (Observed.Count >= limit) + { + throw new ReplayFailure("Core sent more frames than the recording"); + } + + JsonObject? frame = worker.ReadFrame(Limits.Hard); + if (frame is null) + { + return false; + } + + Ended |= IsTerminal(frame); + Observed.Add(frame); + return true; + } + + internal void Run(Worker worker, Transcript transcript) + { + try + { + foreach (TranscriptStep step in transcript.HostInput) + { + if (step.Direction == "core_to_host") + { + Read(worker); + } + else if (step.RawHex is not null) + { + worker.WriteBytes(HexBytes(step.RawHex)); + worker.CloseInput(); + } + else if (step.AfterTerminal && Ended) + { + Refusal = Json.Of("E1608"); + } + else if (Ended) + { + Skipped++; + } + else + { + Write(worker, step.Frame); + } + } + + worker.CloseInput(); + while (Read(worker)) + { + // Read to end of stream. + } + + worker.AwaitExit(); + } + catch (CarrierException error) + { + throw new ReplayFailure(error.Message); + } + } + + private static void Write(Worker worker, JsonObject frame) + { + byte[] data; + try + { + data = JsonCodec.FrameBytes(frame, Limits.Hard.MaxFrameBytes, Limits.Hard.MaxJsonDepth); + } + catch (ProtocolException error) + { + throw new ReplayFailure(error.Message); + } + + try + { + worker.WriteBytes(data); + } + catch (CarrierException error) when (error.IsLoss && error.Message.StartsWith("worker write failed", StringComparison.Ordinal)) + { + // Core may close its input before a hostile recording ends. + } + } + } + + /// + /// Drives one recording against the worker and compares what Core sends + /// with the recording, modulo non-normative diagnostic prose. + /// + public static Replay ReplayOne(Transcript transcript, string store, WorkerOptions options, CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(transcript); + ArgumentNullException.ThrowIfNull(store); + ArgumentNullException.ThrowIfNull(options); + var differences = new List(); + Worker worker; + try + { + worker = Worker.Start(store, options, cancellationToken); + } + catch (ArgumentException error) + { + differences.Add(error.Message); + return new Replay(transcript.Name, false, "diverges", differences, Array.Empty(), JsonObject.Empty, Array.Empty(), null, null, Array.Empty()); + } + + var drive = new Drive(transcript.CoreOutput.Count + 1); + try + { + drive.Run(worker, transcript); + } + catch (ReplayFailure error) + { + differences.Add(error.Message); + } + finally + { + worker.Dispose(); + } + + return Compare(transcript, drive, worker, differences); + } + + private static Replay Compare(Transcript transcript, Drive drive, Worker worker, List differences) + { + List observed = drive.Observed; + int? exitStatus = worker.ExitStatus; + JsonObject end = observed.FirstOrDefault(IsTerminal) ?? JsonObject.Empty; + string? endKind = KindOf(end); + JsonObject? result = endKind == "outcome" ? end.Get("result") as JsonObject : null; + Json? diagnostics = endKind == "fatal" ? end.Get("diagnostics") + : result is not null && Json.Of("error").Equals(result.Get("kind")) ? result.Get("diagnostics") + : null; + JsonObject diagnostic = diagnostics?.AsArray() is { Count: > 0 } entries && entries[0] is JsonObject first ? first : JsonObject.Empty; + Json primary = diagnostic.Get("code") ?? Json.Null; + Json terminalName = endKind switch + { + "fatal" => Json.Of("fatal"), + "shutdown_ack" => Json.Of("shutdown"), + _ => (end.Get("evidence") as JsonObject)?.Get("core") is JsonObject core ? core.Get("terminal") ?? Json.Null : Json.Null, + }; + long requests = observed.Count(frame => KindOf(frame) == "effect_request"); + long terminals = observed.Count(IsTerminal); + JsonObject summary = Json.Object( + ("effect_requests", Json.Of(requests)), + ("host_inputs_skipped_after_terminal", Json.Of(drive.Skipped)), + ("host_local_refusal", drive.Refusal), + ("primary_code", primary), + ("terminal", terminalName), + ("terminal_frames", Json.Of(terminals)), + ("terminal_kind", end.Get("kind") ?? Json.Null)); + JsonObject expect = transcript.Expect; + bool sameFrames = Canonical(StripProse(Frames(observed))) == Canonical(StripProse(Frames(transcript.CoreOutput))); + bool conforms; + if (transcript.Kind == "positive") + { + // As in kit.ts, an absent expectation never equals an observed value or null. + conforms = terminalName.Equals(expect.Get("terminal")) && ExitIs(exitStatus, 0) && sameFrames; + TranscriptStep? failure = transcript.HostInput.FirstOrDefault(step => Json.Of("effect_failure").Equals(step.Frame.Get("kind"))); + if (failure?.Frame.Get("message") is { } message) + { + string expected = JsString(message); + string? cause = diagnostic.Get("cause")?.AsString(); + conforms &= cause is not null && cause.EndsWith(expected, StringComparison.Ordinal); + } + } + else + { + long? maximum = expect.Get("terminal_frames_max")?.AsInt(); + bool forbid = Json.Of(true).Equals(expect.Get("forbid_outside_action")); + int exit = Json.Of("E1611").Equals(expect.Get("code")) ? 74 : 0; + conforms = Json.Of(requests).Equals(expect.Get("effect_requests_before_failure")) + && maximum is not null && terminals <= maximum + && (!forbid || requests == 0) + && ExitIs(exitStatus, exit); + if (transcript.Phase == "terminal") + { + conforms &= drive.Refusal.Equals(expect.Get("code")) && terminalName.Equals(Json.Of("ok")); + } + else + { + conforms &= primary.Equals(expect.Get("code")); + } + } + + string status = conforms ? "conforms" : "diverges"; + if (status != transcript.Status) + { + differences.Add("conformance status differs from the recording"); + } + + if (!ExitIs(exitStatus, transcript.ExitStatus)) + { + differences.Add("exit status differs from the recording"); + } + + if (Canonical(summary) != Canonical(transcript.Observed)) + { + differences.Add("observed summary differs from the recording"); + } + + if (!sameFrames) + { + differences.Add("Core frames differ semantically from the recording"); + } + + List drift = ProseDriftOf(Frames(transcript.CoreOutput), Frames(observed), string.Empty); + return new Replay(transcript.Name, differences.Count == 0, status, differences, drift, summary, observed, exitStatus, worker.Signal, worker.Stderr); + } + + private static bool ExitIs(int? exitStatus, long code) => exitStatus is not null && exitStatus.Value == code; + + /// Replays every recording in order, loading the pinned kit when is null. + /// The pinned kit cannot be loaded. + public static IReadOnlyList ReplayKit(string store, WorkerOptions options, Kit? kit = null, CancellationToken cancellationToken = default) + { + Kit loaded = kit ?? Kit.Load(); + return loaded.Transcripts.Select(transcript => ReplayOne(transcript, store, options, cancellationToken)).ToList(); + } +} diff --git a/dotnet/src/JacquardHost/Handlers.cs b/dotnet/src/JacquardHost/Handlers.cs new file mode 100644 index 0000000..ecf85fe --- /dev/null +++ b/dotnet/src/JacquardHost/Handlers.cs @@ -0,0 +1,126 @@ +using System; +using System.Collections.Generic; +using System.Threading; +using JacquardLang.Host.Protocol; + +namespace JacquardLang.Host; + +/// +/// One effect operation's handler. Return a value, or throw a +/// for a structured failure; any other +/// exception is reported as completion unknown, without its message. +/// +/// Whether the invocation is still live. +/// The decoded request arguments. +/// The operation's result. +public delegate Value Handler(HandlerContext context, IReadOnlyList arguments); + +/// What a handler sees: whether the invocation is still live. +public sealed class HandlerContext +{ + private readonly Func stopping; + private readonly Func ended; + + internal HandlerContext(Func stopping, Func ended, CancellationToken cancellationToken) + { + this.stopping = stopping; + this.ended = ended; + CancellationToken = cancellationToken; + } + + /// + /// Whether the worker expired, the caller cancelled, the lifetime passed, + /// or the invocation ended. Handlers should stop; the host no longer + /// waits for them. + /// + public bool IsCancelled => stopping() || ended() || CancellationToken.IsCancellationRequested; + + /// + /// A token cancelled when the caller cancels, the lifetime deadline + /// passes, the host stops the worker, or the invocation ends. + /// + public CancellationToken CancellationToken { get; } +} + +/// +/// A handler's structured refusal or failure, reported to Core with its +/// category and message. Any other exception a handler throws is reported as +/// completion unknown, without its message. +/// +public sealed class HandlerException : Exception +{ + /// A failure in one of the four permitted categories, with a message for Core. + public HandlerException(FailureCategory category, string message) + : base(message ?? throw new ArgumentNullException(nameof(message))) + { + Category = category; + } + + /// The failure category. + public FailureCategory Category { get; } + + /// The host does not provide the operation (not started). + public static HandlerException Unsupported(string message) => new(FailureCategory.UnsupportedOperation, message); + + /// The host refused to authorize the operation (not started). + public static HandlerException Refused(string message) => new(FailureCategory.RefusedAuthority, message); + + /// An outside action failed without effect. + public static HandlerException OutsideFailed(string message) => new(FailureCategory.OutsideFailure, message); + + /// An outside action's completion is unknown. + public static HandlerException CompletionUnknown(string message) => new(FailureCategory.CompletionUnknown, message); +} + +/// A closed map from exact operation identities to handlers. +public sealed class OperationRegistry +{ + private readonly SortedDictionary handlers = new(); + private readonly object sync = new(); + + /// Binds a handler to one effect operation identity. + /// The identity is already bound. + public OperationRegistry Bind(Hash effect, Hash operation, Handler handler) + { + var key = new OperationEntry(effect, operation); + ArgumentNullException.ThrowIfNull(handler); + lock (sync) + { + if (!handlers.TryAdd(key, handler)) + { + throw new ArgumentException("operation identity is already bound", nameof(operation)); + } + } + + return this; + } + + /// The number of bound operations. + public int Count + { + get + { + lock (sync) + { + return handlers.Count; + } + } + } + + /// The bound operations in protocol order. + public IReadOnlyList Entries() + { + lock (sync) + { + return new List(handlers.Keys); + } + } + + internal Handler? Find(EffectRequest request) + { + lock (sync) + { + return handlers.TryGetValue(new OperationEntry(request.Effect, request.Operation), out Handler? handler) ? handler : null; + } + } +} diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs new file mode 100644 index 0000000..417ed37 --- /dev/null +++ b/dotnet/src/JacquardHost/Host.cs @@ -0,0 +1,518 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using System.Linq; +using System.Text; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Protocol; + +namespace JacquardLang.Host; + +/// Options for a . +public sealed record HostOptions +{ + /// Default carrier options and the hard limits. + public static HostOptions Defaults { get; } = new(); + + /// The worker command, environment, and lifetime deadline. + public WorkerOptions Carrier { get; init; } = WorkerOptions.Defaults; + + /// The session limits; null selects the hard limits. + public Limits? Limits { get; init; } +} + +/// +/// Invokes checked Jacquard terms in one prepared store through +/// jacquard-host-v0 and serves their effect requests from a closed +/// registry of .NET handlers. Each invocation runs one fresh, serial worker. +/// +public sealed class Host +{ + /// Host evidence schema recorded beside Core's frames. + public const string EvidenceSchema = "jacquard-host-dotnet-evidence-v0"; + + /// The call stopped before the handler began; the fault check after the wait reports why. + private static readonly Answer NotStarted = new(null, FailureCategory.OutsideFailure, "handler was not started"); + + private readonly WorkerOptions carrier; + + /// Validates options before any worker starts. + /// The limits or carrier options are invalid. + public Host(string store, HostOptions? options = null) + { + ArgumentNullException.ThrowIfNull(store); + options ??= HostOptions.Defaults; + Limits selected = options.Limits ?? Limits.Hard; + try + { + selected.Validate(); + } + catch (ProtocolException error) + { + throw new ArgumentException(error.Message, nameof(options), error); + } + + ArgumentNullException.ThrowIfNull(options.Carrier, nameof(options)); + options.Carrier.Validate(); + Store = store; + Limits = selected; + carrier = options.Carrier; + } + + /// The prepared store directory. + public string Store { get; } + + /// The limits each session selects. + public Limits Limits { get; } + + /// + /// Test seam: runs on the caller's thread once an effect request is + /// decoded, just before the caller's check. Null outside tests. It is + /// global, so tests that set it must not run in parallel. + /// + internal static volatile Action? RequestDecodedHook; + + /// + /// Test seam: runs on the caller's thread just after a handler thread + /// starts. Null outside tests. + /// + internal static volatile Action? HandlerThreadStartedHook; + + /// + /// Test seam: runs on the handler thread just before its check. Null + /// outside tests. + /// + internal static volatile Action? HandlerCheckHook; + + /// + /// Runs with . The invoke + /// frame is validated before a worker starts. Cancelling + /// kills the worker; nothing is + /// retried. Every outcome, including a lost worker, is returned as an + /// . + /// + /// An argument is null; no worker has started. + public InvocationResult Invoke(Hash target, CallInterface spec, IReadOnlyList args, OperationRegistry registry, CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(target); + ArgumentNullException.ThrowIfNull(spec); + ArgumentNullException.ThrowIfNull(args); + ArgumentNullException.ThrowIfNull(registry); + IReadOnlyList entries; + Value[] arguments; + try + { + // The caller's registry and arguments are bounded before they are copied. + if (registry.Count > Limits.MaxOperations) + { + throw ProtocolException.Exceeded("registry exceeds max_operations"); + } + + entries = registry.Entries(); + JsonObject frame = Codec.InvokeFrame(target, spec, args, entries, Limits); + JsonCodec.FrameBytes(frame, Limits.MaxFrameBytes, Limits.MaxJsonDepth); + arguments = args.ToArray(); + } + catch (ProtocolException error) + { + return new InvocationResult(ResultKind.HostError) + { + HostEvidence = Json.Object( + ("code", Json.Of(error.Code)), + ("detail", Json.Of(error.Detail)), + ("reason", Json.Of("invoke_refused_locally")), + ("schema", Json.Of(EvidenceSchema))), + }; + } + + return Run(session => session.Invoke(target, spec, arguments, entries), registry, cancellationToken); + } + + /// Negotiates a session and shuts it down without invoking. + public InvocationResult ShutdownProbe(CancellationToken cancellationToken = default) => + Run(session => session.Shutdown(), new OperationRegistry(), cancellationToken); + + /// + /// At most UTF-8 bytes of , + /// cut at a code point boundary; a lone surrogate, which has no encoding, + /// becomes U+FFFD. The scan stops at the limit. + /// + internal static string Bounded(string text, int limit) + { + var output = new StringBuilder(); + int used = 0; + for (int i = 0; i < text.Length;) + { + int point; + if (char.IsHighSurrogate(text[i]) && i + 1 < text.Length && char.IsLowSurrogate(text[i + 1])) + { + point = char.ConvertToUtf32(text[i], text[i + 1]); + i += 2; + } + else + { + point = char.IsSurrogate(text[i]) ? 0xFFFD : text[i]; + i++; + } + + int size = point < 0x80 ? 1 : point < 0x800 ? 2 : point < 0x10000 ? 3 : 4; + if (used + size > limit) + { + break; + } + + output.Append(char.ConvertFromUtf32(point)); + used += size; + } + + return output.ToString(); + } + + private InvocationResult Run(Func opening, OperationRegistry registry, CancellationToken cancellationToken) + { + var session = new Session(); + var result = new InvocationResult(ResultKind.CarrierLoss); + Worker worker; + try + { + worker = Worker.Start(Store, carrier, cancellationToken); + } + catch (ArgumentException error) + { + result.HostEvidence = Json.Object(("detail", Json.Of(error.Message)), ("reason", Json.Of("invalid_options")), ("schema", Json.Of(EvidenceSchema))); + return result; + } + + var call = new Call(worker, session, result, registry, cancellationToken); + try + { + call.Drive(opening, Limits); + } + catch (CarrierException error) + { + Record(call, error); + } + finally + { + worker.Dispose(); + call.End(); + } + + result.Requests = session.Requests.ToArray(); + result.ExitStatus = worker.ExitStatus; + result.Signal = worker.Signal; + result.Stderr = worker.Stderr; + return result; + } + + private static void Record(Call call, CarrierException error) + { + ResultKind kind; + string code; + string reason; + if (!error.IsLoss) + { + code = error.ProtocolError!.Code; + kind = call.Sending ? ResultKind.HostError : ResultKind.ProtocolError; + reason = call.Sending ? "host_frame_refused" : "core_frame_refused"; + } + else + { + kind = ResultKind.CarrierLoss; + code = "E1611"; + reason = error.IsTimeout ? "deadline" + : call.Worker.SpawnError is not null ? "worker_spawn_failed" + : error.IsAborted ? "aborted" + : "carrier_loss"; + } + + call.Result.Kind = kind; + long? outstanding = call.Session.Pending; + call.Result.HostEvidence = Json.Object( + ("code", Json.Of(code)), + ("detail", Json.Of(error.Message)), + ("outstanding_request", outstanding is null ? Json.Null : Json.Of(outstanding.Value)), + ("reason", Json.Of(reason)), + ("schema", Json.Of(EvidenceSchema)), + ("state", Json.Of(call.Session.State.WireName()))); + } + + /// A handler's answer: a value, or a failure category and message. + private sealed record Answer(Value? Value, FailureCategory Category, string? Message); + + /// One invocation's worker, session, and handler state. + [SuppressMessage("Design", "CA1001", Justification = "The handlers' token source is cancelled, never disposed: an abandoned handler may still hold its token.")] + private sealed class Call + { + private readonly OperationRegistry registry; + private readonly CancellationToken cancellationToken; + private volatile bool ended; + private CancellationTokenSource? handlerStop; + private CancellationTokenRegistration callerRegistration; + + internal Call(Worker worker, Session session, InvocationResult result, OperationRegistry registry, CancellationToken cancellationToken) + { + Worker = worker; + Session = session; + Result = result; + this.registry = registry; + this.cancellationToken = cancellationToken; + } + + internal Worker Worker { get; } + + internal Session Session { get; } + + internal InvocationResult Result { get; } + + /// Whether a host frame was being encoded or sent when a failure occurred. + internal bool Sending { get; private set; } + + /// Marks the invocation ended and cancels the handlers' token. + internal void End() + { + ended = true; + callerRegistration.Dispose(); + handlerStop?.Cancel(); + } + + private void Send(JsonObject frame, Limits limits) + { + Sending = true; + Worker.WriteFrame(frame, limits); + Result.HostFrameList.Add(frame); + Sending = false; + } + + private static CarrierException Refused(ProtocolException error) => CarrierException.FromProtocol(error); + + internal void Drive(Func opening, Limits limits) + { + JsonObject hello = Worker.ReadFrame(Limits.Hard) ?? throw CarrierException.Loss("worker exited before core_hello"); + Result.CoreFrameList.Add(hello); + JsonObject selected; + try + { + Session.Receive(hello); + selected = Session.Select(limits); + } + catch (ProtocolException error) + { + throw Refused(error); + } + + Send(selected, Limits.Hard); + Worker.SetStderrLimit(limits.MaxStderrBytes); + JsonObject first; + try + { + first = opening(Session); + } + catch (ProtocolException error) + { + throw Refused(error); + } + + Send(first, Session.Limits); + while (Session.State != State.Closed) + { + JsonObject frame = Worker.ReadFrame(Session.Limits) ?? throw CarrierException.Loss("worker exited before a terminal frame"); + Result.CoreFrameList.Add(frame); + CoreFrame? parsed; + try + { + parsed = Session.Receive(frame); + } + catch (ProtocolException error) + { + throw Refused(error); + } + + switch (parsed) + { + case EffectRequest request: + Send(Respond(request), Session.Limits); + break; + case Outcome outcome: + Result.Kind = outcome.Value is not null ? ResultKind.Ok : ResultKind.Error; + Result.Value = outcome.Value; + Result.Diagnostics = outcome.Diagnostics; + Result.Terminal = outcome.TerminalName; + Result.Evidence = outcome.Evidence; + break; + case Fatal fatal: + Result.Kind = ResultKind.Fatal; + Result.Diagnostics = fatal.Diagnostics; + break; + case ShutdownAck: + Result.Kind = ResultKind.ShutdownAck; + break; + } + } + + Worker.CloseInput(); + JsonObject? extra = Worker.ReadFrame(Session.Limits); + if (extra is not null) + { + Result.CoreFrameList.Add(extra); + try + { + Session.Receive(extra); + } + catch (ProtocolException error) + { + throw Refused(error); + } + } + + Worker.AwaitExit(); + if (Worker.ExitStatus != 0) + { + throw CarrierException.Loss("worker exited unsuccessfully after its terminal"); + } + } + + /// + /// Runs the handler on its own thread, bounded by the worker lifetime, + /// and renders the response. A handler still running at the deadline is + /// abandoned: no late response is sent and nothing is retried. + /// + private JsonObject Respond(EffectRequest request) + { + Handler? handler = registry.Find(request); + Answer answer; + if (handler is null) + { + answer = new Answer(null, FailureCategory.UnsupportedOperation, "operation is not configured"); + } + else + { + // Decoding the request takes time; an outside action must not + // begin once the call is cancelled or the lifetime passed. The + // caller checks before creating the handler thread, and the + // thread checks again before calling the handler. + RequestDecodedHook?.Invoke(); + CarrierException? early = Worker.Fault(); + if (early is not null) + { + throw early; + } + + answer = RunHandler(handler, request.Arguments); + } + + CarrierException? fault = Worker.Fault(); + if (fault is not null) + { + throw fault; + } + + try + { + if (answer.Message is null) + { + try + { + Session.OkFrame(answer.Value); + return Session.RespondOk(answer.Value); + } + catch (ProtocolException) + { + answer = new Answer(null, FailureCategory.OutsideFailure, "handler result has no encoding within the selected limits"); + } + } + + return Session.RespondFailure(answer.Category, Bounded(answer.Message ?? string.Empty, Session.Limits.MaxHostMessageBytes)); + } + catch (ProtocolException error) + { + throw Refused(error); + } + } + + private Answer RunHandler(Handler handler, IReadOnlyList arguments) + { + HandlerContext context = new(() => Worker.Stopping, () => ended, HandlerToken()); + var completion = new Completion(); + var thread = new Thread(() => + { + HandlerCheckHook?.Invoke(); + completion.Answer = Worker.Stopping ? NotStarted : CallHandler(handler, context, arguments); + }) + { + IsBackground = true, + Name = "jacquard-handler", + }; + try + { + thread.Start(); + } + catch (OutOfMemoryException) + { + // The handler never ran. + return new Answer(null, FailureCategory.OutsideFailure, "handler thread could not start"); + } + + HandlerThreadStartedHook?.Invoke(); + Worker.Await(() => completion.Answer is not null); + return completion.Answer ?? new Answer(null, FailureCategory.CompletionUnknown, "handler threw an exception"); + } + + /// + /// The handlers' token: cancelled by the caller's token, at the + /// lifetime deadline, and when the invocation ends. It is never + /// disposed, because an abandoned handler may still hold it; once + /// cancelled it holds no timer. + /// + private CancellationToken HandlerToken() + { + if (handlerStop is null) + { + handlerStop = new CancellationTokenSource(); + // A timer cannot wait longer than about 24 days; IsCancelled + // sees a later deadline exactly in any case. + TimeSpan remaining = Worker.Remaining; + if (remaining <= TimeSpan.FromMilliseconds(int.MaxValue)) + { + handlerStop.CancelAfter(remaining); + } + callerRegistration = cancellationToken.UnsafeRegister(static state => ((CancellationTokenSource)state!).Cancel(), handlerStop); + } + + return handlerStop.Token; + } + + /// Calls a handler, turning every exception into a failure answer. + [SuppressMessage("Design", "CA1031", Justification = "Any exception a handler throws is reported to Core as completion unknown.")] + private static Answer CallHandler(Handler handler, HandlerContext context, IReadOnlyList arguments) + { + try + { + Value value = handler(context, arguments); + return new Answer(value, FailureCategory.OutsideFailure, null); + } + catch (HandlerException failure) + { + return new Answer(null, failure.Category, failure.Message); + } + catch (Exception) + { + // Its message may carry private detail and is never sent. + return new Answer(null, FailureCategory.CompletionUnknown, "handler threw an exception"); + } + } + } + + /// The handler thread's answer, published once. + private sealed class Completion + { + private volatile Answer? answer; + + internal Answer? Answer + { + get => answer; + set => answer = value; + } + } +} diff --git a/dotnet/src/JacquardHost/InvocationResult.cs b/dotnet/src/JacquardHost/InvocationResult.cs new file mode 100644 index 0000000..fea707f --- /dev/null +++ b/dotnet/src/JacquardHost/InvocationResult.cs @@ -0,0 +1,190 @@ +using System; +using System.Collections.Generic; +using JacquardLang.Host.Protocol; + +namespace JacquardLang.Host; + +/// The kind of an invocation's result. +public enum ResultKind +{ + /// Core returned a value. + Ok, + + /// Core returned an error result with diagnostics. + Error, + + /// Core ended the session with a fatal frame. + Fatal, + + /// Core acknowledged a shutdown. + ShutdownAck, + + /// The worker was lost: spawn failure, early exit, deadline, or abort. + CarrierLoss, + + /// Core sent a frame the protocol refuses. + ProtocolError, + + /// The host refused its own frame. + HostError, +} + +/// The classification of an unsuccessful invocation. +public enum Failure +{ + /// The worker's lifetime deadline passed. + DeadlineExceeded, + + /// The worker was lost or the call aborted. + WorkerLost, + + /// An operation failed or its completion is unknown. + OperationFailed, + + /// Any other failure. + Other, +} + +/// Core's frames and evidence beside the host's observations. +public sealed class InvocationResult +{ + internal InvocationResult(ResultKind kind) + { + Kind = kind; + } + + /// The result kind. + public ResultKind Kind { get; internal set; } + + /// The value of a successful outcome, else null. + public Value? Value { get; internal set; } + + /// Core's diagnostics, for an error outcome or a fatal frame. + public IReadOnlyList Diagnostics { get; internal set; } = Array.Empty(); + + /// Core's evidence terminal name for an outcome, else null. + public string? Terminal { get; internal set; } + + /// Core's evidence for an outcome, else null. + public JsonObject? Evidence { get; internal set; } + + /// Every frame Core sent, in order. + public IReadOnlyList CoreFrames => CoreFrameList; + + /// Every frame the host sent, in order. + public IReadOnlyList HostFrames => HostFrameList; + + /// The effect requests Core made. + public IReadOnlyList Requests { get; internal set; } = Array.Empty(); + + /// The worker's exit status, when known and not ended by the host's SIGKILL. + public int? ExitStatus { get; internal set; } + + /// "SIGKILL" when the host killed the worker; see . + public string? Signal { get; internal set; } + + /// The retained stderr prefix. + public IReadOnlyList Stderr { get; internal set; } = Array.Empty(); + + /// The host's evidence (jacquard-host-dotnet-evidence-v0); empty on success. + public JsonObject HostEvidence { get; internal set; } = JsonObject.Empty; + + /// Whether Core returned a successful outcome. + public bool Ok => Kind == ResultKind.Ok; + + internal List CoreFrameList { get; } = new(); + + internal List HostFrameList { get; } = new(); + + /// The first Core diagnostic code, else the host evidence code. + public string Code + { + get + { + if (Diagnostics.Count > 0 && Diagnostics[0].Code.Length > 0) + { + return Diagnostics[0].Code; + } + + return HostEvidence.Get("code")?.AsString() ?? string.Empty; + } + } + + /// A summary for people. + public string Message + { + get + { + if (Ok) + { + return "Jacquard invocation succeeded"; + } + + string detail = Diagnostics.Count > 0 + ? Diagnostics[0].Summary + " " + Diagnostics[0].Cause + : HostEvidence.Get("detail")?.AsString() ?? KindName(Kind); + string code = Code; + return (code.Length == 0 ? KindName(Kind) : code) + ": " + detail; + } + } + + /// The kind's name, as the other adapters spell it, such as "carrier_loss". + public static string KindName(ResultKind kind) => kind switch + { + ResultKind.Ok => "ok", + ResultKind.Error => "error", + ResultKind.Fatal => "fatal", + ResultKind.ShutdownAck => "shutdown_ack", + ResultKind.CarrierLoss => "carrier_loss", + ResultKind.ProtocolError => "protocol_error", + ResultKind.HostError => "host_error", + _ => throw new ArgumentOutOfRangeException(nameof(kind)), + }; + + /// The value, or an classifying the failure. + /// The invocation did not succeed. + public Value ValueOrThrow() + { + if (Ok && Value is not null) + { + return Value; + } + + throw new InvocationException(this); + } + + /// + public override string ToString() => "InvocationResult[" + KindName(Kind) + ", " + Message + "]"; +} + +/// An unsuccessful invocation. +public sealed class InvocationException : Exception +{ + internal InvocationException(InvocationResult result) + : base(result.Message) + { + Result = result; + } + + /// The result, with Core's frames and the host's evidence. + public InvocationResult Result { get; } + + /// The classification. + public Failure Failure + { + get + { + if (Json.Of("deadline").Equals(Result.HostEvidence.Get("reason"))) + { + return Failure.DeadlineExceeded; + } + + if (Result.Kind == ResultKind.CarrierLoss) + { + return Failure.WorkerLost; + } + + return Result.Terminal is "host_failure" or "completion_unknown" ? Failure.OperationFailed : Failure.Other; + } + } +} diff --git a/dotnet/src/JacquardHost/JacquardHost.csproj b/dotnet/src/JacquardHost/JacquardHost.csproj new file mode 100644 index 0000000..7fba143 --- /dev/null +++ b/dotnet/src/JacquardHost/JacquardHost.csproj @@ -0,0 +1,33 @@ + + + + net8.0 + JacquardLang.Host + JacquardLang.Host + true + true + true + JacquardLang.Host + An independent .NET adapter for Jacquard Core's experimental jacquard-host-v0 protocol. + Apache-2.0 + README.md + https://github.com/jbwinters/jacquard-host + + + + + + + + + + + + + + + + + + + diff --git a/dotnet/src/JacquardHost/Protocol/BoundaryType.cs b/dotnet/src/JacquardHost/Protocol/BoundaryType.cs new file mode 100644 index 0000000..7c6004b --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/BoundaryType.cs @@ -0,0 +1,159 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace JacquardLang.Host.Protocol; + +/// A closed v0 boundary type: or . +public abstract record BoundaryType +{ + private protected BoundaryType() + { + } + + internal static BoundaryType[] Copy(IEnumerable items, string name) + { + ArgumentNullException.ThrowIfNull(items, name); + BoundaryType[] copy = items.ToArray(); + foreach (BoundaryType item in copy) + { + ArgumentNullException.ThrowIfNull(item, name); + } + + return copy; + } + + internal static bool Same(BoundaryType[] a, BoundaryType[] b) => a.Length == b.Length && a.Zip(b).All(pair => pair.First.Equals(pair.Second)); + + internal static int HashItems(BoundaryType[] items) + { + HashCode hash = default; + hash.Add(items.Length); + foreach (BoundaryType item in items) + { + hash.Add(item); + } + + return hash.ToHashCode(); + } +} + +/// A type identity applied to arguments. +public sealed record NominalType : BoundaryType +{ + private readonly BoundaryType[] arguments; + + /// A nominal type holding a copy of . + public NominalType(Hash identity, IEnumerable arguments) + { + ArgumentNullException.ThrowIfNull(identity); + Identity = identity; + this.arguments = Copy(arguments, nameof(arguments)); + } + + /// A nominal type with no arguments. + public NominalType(Hash identity) + : this(identity, Array.Empty()) + { + } + + /// The type identity. + public Hash Identity { get; } + + /// The type arguments. + public IReadOnlyList Arguments => arguments; + + /// + public bool Equals(NominalType? other) => other is not null && Identity.Equals(other.Identity) && Same(arguments, other.arguments); + + /// + public override int GetHashCode() => HashCode.Combine(Identity, HashItems(arguments)); +} + +/// A positional product type. +public sealed record TupleType : BoundaryType +{ + private readonly BoundaryType[] items; + + /// A tuple type holding a copy of . + public TupleType(IEnumerable items) + { + this.items = Copy(items, nameof(items)); + } + + /// A tuple type of . + public TupleType(params BoundaryType[] items) + : this((IEnumerable)items) + { + } + + /// The item types. + public IReadOnlyList Items => items; + + /// + public bool Equals(TupleType? other) => other is not null && Same(items, other.items); + + /// + public override int GetHashCode() => HashItems(items); +} + +/// The checked callable signature sent with an invocation. +public sealed record CallInterface +{ + private readonly BoundaryType[] parameters; + private readonly Hash[] effects; + + /// A signature holding copies of and . + public CallInterface(IEnumerable parameters, BoundaryType result, IEnumerable effects) + { + ArgumentNullException.ThrowIfNull(result); + ArgumentNullException.ThrowIfNull(effects); + this.parameters = BoundaryType.Copy(parameters, nameof(parameters)); + Result = result; + this.effects = effects.ToArray(); + foreach (Hash effect in this.effects) + { + ArgumentNullException.ThrowIfNull(effect, nameof(effects)); + } + } + + /// The parameter types. + public IReadOnlyList Parameters => parameters; + + /// The result type. + public BoundaryType Result { get; } + + /// The effect row: a strictly sorted set of effect identities. + public IReadOnlyList Effects => effects; + + /// + public bool Equals(CallInterface? other) => + other is not null && BoundaryType.Same(parameters, other.parameters) && Result.Equals(other.Result) && effects.SequenceEqual(other.effects); + + /// + public override int GetHashCode() => HashCode.Combine(BoundaryType.HashItems(parameters), Result, effects.Length); +} + +/// An exact effect operation granted for the invocation. +/// The effect identity. +/// The operation identity. +public sealed record OperationEntry(Hash Effect, Hash Operation) : IComparable +{ + /// The effect identity. + public Hash Effect { get; } = Effect ?? throw new ArgumentNullException(nameof(Effect)); + + /// The operation identity. + public Hash Operation { get; } = Operation ?? throw new ArgumentNullException(nameof(Operation)); + + /// + public int CompareTo(OperationEntry? other) + { + if (other is null) + { + return 1; + } + + int order = Effect.CompareTo(other.Effect); + return order != 0 ? order : Operation.CompareTo(other.Operation); + } +} diff --git a/dotnet/src/JacquardHost/Protocol/Budget.cs b/dotnet/src/JacquardHost/Protocol/Budget.cs new file mode 100644 index 0000000..c1ef6a8 --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Budget.cs @@ -0,0 +1,77 @@ +using System; + +namespace JacquardLang.Host.Protocol; + +/// +/// Counts nodes and checks collection and text sizes for one frame. Text is +/// charged against the frame too, so validating a value scans at most about +/// one frame of text, however many times a long text is referenced. +/// +public sealed class Budget +{ + private int nodes; + + /// The UTF-8 bytes of every text checked so far. + private long textBytes; + + /// A fresh budget under . + public Budget(Limits limits) + { + ArgumentNullException.ThrowIfNull(limits); + Limits = limits; + } + + /// The limits this budget charges against. + public Limits Limits { get; } + + internal void Consume() + { + if (nodes >= Limits.MaxValueNodes) + { + throw ProtocolException.Exceeded("frame exceeds max_value_nodes"); + } + + nodes++; + } + + internal void Count(int n, bool arguments) + { + if (n > Limits.MaxCollectionItems || (arguments && n > Limits.MaxArguments)) + { + throw ProtocolException.Exceeded("collection exceeds selected limits"); + } + } + + internal void Text(string text) + { + // A string's UTF-8 length is at least its length in chars, so these + // refuse before scanning: the scan below covers at most one text and + // one frame. + if (text.Length > Limits.MaxTextBytes) + { + throw ProtocolException.Exceeded("Text exceeds max_text_bytes"); + } + + if (textBytes + text.Length > Limits.MaxFrameBytes) + { + throw ProtocolException.Exceeded("payload exceeds max_frame_bytes"); + } + + if (!JsonCodec.IsScalar(text)) + { + throw ProtocolException.Malformed("a JSON string contains a lone surrogate"); + } + + long length = JsonCodec.Utf8Length(text); + if (length > Limits.MaxTextBytes) + { + throw ProtocolException.Exceeded("Text exceeds max_text_bytes"); + } + + textBytes += length; + if (textBytes > Limits.MaxFrameBytes) + { + throw ProtocolException.Exceeded("payload exceeds max_frame_bytes"); + } + } +} diff --git a/dotnet/src/JacquardHost/Protocol/Codec.cs b/dotnet/src/JacquardHost/Protocol/Codec.cs new file mode 100644 index 0000000..a664d5b --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Codec.cs @@ -0,0 +1,705 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.Linq; + +namespace JacquardLang.Host.Protocol; + +/// +/// Frozen v0 value and type codecs, the invoke frame, and Core frame +/// validation. Descriptors are checked in Core's order: kind, exact fields, +/// one node, identity, then counts. +/// +public static class Codec +{ + /// The only invocation identity a v0 host session uses. + public const string InvocationId = "0000000000000000"; + + private static readonly HashSet Terminals = new(StringComparer.Ordinal) { "ok", "diagnostic", "cancelled", "host_failure", "completion_unknown" }; + + private static readonly HashSet Categories = new(StringComparer.Ordinal) + { + "ok", "unsupported_operation", "refused_authority", "outside_failure", "completion_unknown", "cancelled", "timeout", "host_shutdown", + }; + + private static readonly HashSet Completions = new(StringComparer.Ordinal) { "completed", "not_started", "failed", "unknown" }; + + private static readonly string[] DiagnosticKeys = { "cause", "code", "domain", "next_step", "schema", "severity", "span", "summary" }; + + private static readonly string[] DiagnosticKeysWithContrast = DiagnosticKeys.Append("contrast").ToArray(); + + internal static JsonObject AsObject(Json? value) => value as JsonObject ?? throw ProtocolException.Malformed("expected a JSON object"); + + /// The object's fields, refusing any key set other than exactly . + /// E1601 for a non-object or another key set. + public static JsonObject Exact(Json? value, params string[] keys) + { + ArgumentNullException.ThrowIfNull(keys); + JsonObject fields = AsObject(value); + bool ok = fields.Count == keys.Length; + foreach (string key in keys) + { + ok &= fields.ContainsKey(key); + } + + if (!ok) + { + throw ProtocolException.Malformed("expected exactly: " + string.Join(", ", keys)); + } + + return fields; + } + + private static string AsStr(Json? value) => value?.AsString() ?? throw ProtocolException.Malformed("expected a string"); + + private static IReadOnlyList AsArray(Json? value) => value?.AsArray() ?? throw ProtocolException.Malformed("expected an array"); + + private static string Kind(Json value) + { + JsonObject fields = AsObject(value); + return fields.Get("kind")?.AsString() ?? throw ProtocolException.Malformed("a boundary descriptor needs one kind string"); + } + + private static IReadOnlyList Counted(Json? value, Budget budget, bool arguments) + { + IReadOnlyList items = AsArray(value); + budget.Count(items.Count, arguments); + return items; + } + + /// Parses one wire boundary type, charging . + /// E1601, E1602, or E1604 as Core refuses the descriptor. + public static BoundaryType DecodeType(Json value, Budget budget) + { + ArgumentNullException.ThrowIfNull(value); + ArgumentNullException.ThrowIfNull(budget); + switch (Kind(value)) + { + case "nominal": + { + JsonObject fields = Exact(value, "arguments", "identity", "kind"); + budget.Consume(); + Hash identity = Hash.FromJson(fields["identity"]); + var arguments = new List(); + foreach (Json item in Counted(fields["arguments"], budget, false)) + { + arguments.Add(DecodeType(item, budget)); + } + + return new NominalType(identity, arguments); + } + + case "tuple": + { + JsonObject fields = Exact(value, "items", "kind"); + budget.Consume(); + var items = new List(); + foreach (Json item in Counted(fields["items"], budget, false)) + { + items.Add(DecodeType(item, budget)); + } + + return new TupleType(items); + } + + default: + throw ProtocolException.Unsupported("unsupported boundary type kind"); + } + } + + /// Renders a boundary type, charging . + /// E1602 when a limit is exceeded. + public static Json EncodeType(BoundaryType type, Budget budget) + { + ArgumentNullException.ThrowIfNull(budget); + return EncodeTypeAt(type, budget, 1); + } + + private static JsonObject EncodeTypeAt(BoundaryType type, Budget budget, int depth) + { + ArgumentNullException.ThrowIfNull(type); + if (depth > budget.Limits.MaxJsonDepth) + { + throw ProtocolException.Exceeded("type nesting exceeds max_json_depth"); + } + + budget.Consume(); + IReadOnlyList children = type switch + { + NominalType nominal => nominal.Arguments, + TupleType tuple => tuple.Items, + _ => throw ProtocolException.Unsupported("unsupported boundary type kind"), + }; + budget.Count(children.Count, false); + var items = new List(children.Count); + foreach (BoundaryType child in children) + { + items.Add(EncodeTypeAt(child, budget, depth + 2)); + } + + return type is NominalType named + ? Json.Object(("arguments", Json.Of(items)), ("identity", Json.Of(named.Identity.Hex)), ("kind", Json.Of("nominal"))) + : Json.Object(("items", Json.Of(items)), ("kind", Json.Of("tuple"))); + } + + private static long CanonicalInt(string text) + { + string digits = text.StartsWith('-') ? text[1..] : text; + bool canonical = digits.Length > 0 + && digits.All(c => c >= '0' && c <= '9') + && (digits == "0" || !digits.StartsWith('0')) + && text != "-0" + && text.Length <= 20; + if (!canonical) + { + throw ProtocolException.Malformed("Int must use canonical 63-bit decimal text"); + } + + if (!long.TryParse(text, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out long value) || value < Json.IntMin || value > Json.IntMax) + { + throw ProtocolException.Malformed("Int is outside the 63-bit range"); + } + + return value; + } + + /// Parses one wire boundary value, charging . + /// E1601, E1602, or E1604 as Core refuses the descriptor. + public static Value DecodeValue(Json value, Budget budget) + { + ArgumentNullException.ThrowIfNull(value); + ArgumentNullException.ThrowIfNull(budget); + string kind = Kind(value); + switch (kind) + { + case "int": + case "text": + case "hash": + { + JsonObject fields = Exact(value, "kind", "value"); + budget.Consume(); + switch (kind) + { + case "int": + return new IntValue(CanonicalInt(AsStr(fields["value"]))); + case "text": + string text = AsStr(fields["value"]); + budget.Text(text); + return new TextValue(text); + default: + return new HashValue(Hash.FromJson(fields["value"])); + } + } + + case "real": + { + JsonObject fields = Exact(value, "bits", "kind"); + budget.Consume(); + string bits = AsStr(fields["bits"]); + if (!Hash.LowerHex(bits, 16)) + { + throw ProtocolException.Malformed("Real must have exactly 16 lowercase hexadecimal digits"); + } + + return new RealValue(ulong.Parse(bits, NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture)); + } + + case "tuple": + { + JsonObject fields = Exact(value, "items", "kind"); + budget.Consume(); + var items = new List(); + foreach (Json item in Counted(fields["items"], budget, false)) + { + items.Add(DecodeValue(item, budget)); + } + + return new TupleValue(items); + } + + case "constructor": + { + JsonObject fields = Exact(value, "arguments", "identity", "kind"); + budget.Consume(); + Hash identity = Hash.FromJson(fields["identity"]); + var arguments = new List(); + foreach (Json item in Counted(fields["arguments"], budget, true)) + { + arguments.Add(DecodeValue(item, budget)); + } + + return new ConstructorValue(identity, arguments); + } + + default: + throw ProtocolException.Unsupported("unsupported boundary value kind"); + } + } + + /// Renders a boundary value, charging . + /// E1601 for a lone surrogate, E1602 when a limit is exceeded, E1604 for null or an Int outside the 63-bit range. + public static Json EncodeValue(Value? value, Budget budget) + { + ArgumentNullException.ThrowIfNull(budget); + return EncodeValueAt(value, budget, 1); + } + + private static JsonObject EncodeValueAt(Value? value, Budget budget, int depth) + { + if (value is null) + { + throw ProtocolException.Unsupported("null is not a boundary value"); + } + + if (depth > budget.Limits.MaxJsonDepth) + { + throw ProtocolException.Exceeded("value nesting exceeds max_json_depth"); + } + + budget.Consume(); + switch (value) + { + case IntValue number: + if (number.Value < Json.IntMin || number.Value > Json.IntMax) + { + throw ProtocolException.Unsupported("Int is outside the 63-bit range"); + } + + return Json.Object(("kind", Json.Of("int")), ("value", Json.Of(number.Value.ToString(CultureInfo.InvariantCulture)))); + case TextValue text: + budget.Text(text.Value); + return Json.Object(("kind", Json.Of("text")), ("value", Json.Of(text.Value))); + case RealValue real: + return Json.Object(("bits", Json.Of(real.Bits.ToString("x16", CultureInfo.InvariantCulture))), ("kind", Json.Of("real"))); + case HashValue hash: + return Json.Object(("kind", Json.Of("hash")), ("value", Json.Of(hash.Hash.Hex))); + case TupleValue tuple: + { + budget.Count(tuple.Items.Count, false); + var items = new List(tuple.Items.Count); + foreach (Value item in tuple.Items) + { + items.Add(EncodeValueAt(item, budget, depth + 2)); + } + + return Json.Object(("items", Json.Of(items)), ("kind", Json.Of("tuple"))); + } + + case ConstructorValue constructor: + { + budget.Count(constructor.Arguments.Count, true); + var items = new List(constructor.Arguments.Count); + foreach (Value item in constructor.Arguments) + { + items.Add(EncodeValueAt(item, budget, depth + 2)); + } + + return Json.Object(("arguments", Json.Of(items)), ("identity", Json.Of(constructor.Identity.Hex)), ("kind", Json.Of("constructor"))); + } + + default: + throw ProtocolException.Unsupported("unsupported boundary value kind"); + } + } + + private static long StringBytes(Json value) => value switch + { + JsonString text => JsonCodec.Utf8Length(text.Value), + JsonArray array => array.Items.Sum(StringBytes), + JsonObject obj => obj.Values.Sum(StringBytes), + _ => 0, + }; + + private static List Diagnostics(Json? value, Limits limits) + { + IReadOnlyList entries = AsArray(value); + if (entries.Count == 0) + { + throw ProtocolException.Malformed("diagnostics must be nonempty"); + } + + if (entries.Count > limits.MaxDiagnostics || StringBytes(value!) > limits.MaxDiagnosticBytes) + { + throw ProtocolException.Exceeded("diagnostics exceed selected limits"); + } + + var parsed = new List(entries.Count); + foreach (Json entry in entries) + { + JsonObject fields = AsObject(entry); + Exact(entry, fields.ContainsKey("contrast") ? DiagnosticKeysWithContrast : DiagnosticKeys); + if (!Json.Of("jacquard-diagnostic-v1").Equals(fields["schema"])) + { + throw ProtocolException.Malformed("unknown diagnostic schema"); + } + + Json codeField = fields["code"]; + string code = codeField is JsonNull ? string.Empty : AsStr(codeField); + if (fields["span"] is not JsonNull) + { + AsObject(fields["span"]); + } + + Json? contrast = fields.Get("contrast"); + if (contrast is not null && contrast is not JsonNull) + { + JsonObject pair = Exact(contrast, "intended", "mistaken"); + AsStr(pair["intended"]); + AsStr(pair["mistaken"]); + } + + parsed.Add(new Diagnostic( + code, + AsStr(fields["domain"]), + AsStr(fields["severity"]), + AsStr(fields["summary"]), + AsStr(fields["cause"]), + AsStr(fields["next_step"]), + fields)); + } + + return parsed; + } + + private static List SortedHashes(Json? value, int limit) + { + var hashes = new List(); + foreach (Json item in AsArray(value)) + { + hashes.Add(Hash.FromJson(item)); + } + + if (hashes.Count > limit) + { + throw ProtocolException.Exceeded("effect set exceeds selected limits"); + } + + if (!StrictlySorted(hashes)) + { + throw ProtocolException.Malformed("effects must be a strictly sorted set"); + } + + return hashes; + } + + private static bool StrictlySorted(IReadOnlyList hashes) + { + for (int i = 1; i < hashes.Count; i++) + { + if (hashes[i].CompareTo(hashes[i - 1]) <= 0) + { + return false; + } + } + + return true; + } + + private static JsonArray HashList(IEnumerable hashes) => Json.Of(hashes.Select(hash => (Json)Json.Of(hash.Hex))); + + /// Validates and renders the single v0 invoke frame. + /// E1601, E1602, or E1604 when the invocation cannot be encoded within . + public static JsonObject InvokeFrame(Hash target, CallInterface spec, IReadOnlyList args, IReadOnlyList operations, Limits limits) + { + ArgumentNullException.ThrowIfNull(target); + ArgumentNullException.ThrowIfNull(spec); + ArgumentNullException.ThrowIfNull(args); + ArgumentNullException.ThrowIfNull(operations); + ArgumentNullException.ThrowIfNull(limits); + if (args.Count != spec.Parameters.Count) + { + throw ProtocolException.Malformed("argument count differs from parameter count"); + } + + var budget = new Budget(limits); + budget.Count(args.Count, true); + if (spec.Effects.Count > limits.MaxEffects) + { + throw ProtocolException.Exceeded("effect set exceeds selected limits"); + } + + if (!StrictlySorted(spec.Effects)) + { + throw ProtocolException.Malformed("effects must be a strictly sorted set"); + } + + if (operations.Count > limits.MaxOperations) + { + throw ProtocolException.Exceeded("registry exceeds max_operations"); + } + + OperationEntry[] entries = operations.ToArray(); + foreach (OperationEntry entry in entries) + { + ArgumentNullException.ThrowIfNull(entry, nameof(operations)); + } + + Array.Sort(entries); + var granted = new List(entries.Length); + for (int index = 0; index < entries.Length; index++) + { + OperationEntry entry = entries[index]; + if (!spec.Effects.Contains(entry.Effect)) + { + throw ProtocolException.Malformed("operation effect is outside the interface row"); + } + + if (index > 0 && entries[index - 1].Equals(entry)) + { + throw ProtocolException.Malformed("duplicate operation entry"); + } + + granted.Add(Json.Object(("effect", Json.Of(entry.Effect.Hex)), ("mode", Json.Of("once")), ("operation", Json.Of(entry.Operation.Hex)))); + } + + var arguments = new List(args.Count); + foreach (Value arg in args) + { + arguments.Add(EncodeValue(arg, budget)); + } + + var parameters = new List(spec.Parameters.Count); + foreach (BoundaryType parameter in spec.Parameters) + { + parameters.Add(EncodeType(parameter, budget)); + } + + Json result = EncodeType(spec.Result, budget); + return Json.Object( + ("arguments", Json.Of(arguments)), + ("capabilities", Json.Object(("effects", HashList(spec.Effects)), ("operations", Json.Of(granted)))), + ("interface", Json.Object(("effects", HashList(spec.Effects)), ("parameters", Json.Of(parameters)), ("result", result))), + ("invocation_id", Json.Of(InvocationId)), + ("kind", Json.Of("invoke")), + ("protocol", Json.Of(Compat.Protocol)), + ("target", Json.Object(("callable", Json.Of(target.Hex)), ("kind", Json.Of("store-term-v0"))))); + } + + private static void Hashes(JsonObject fields, params string[] keys) + { + foreach (string key in keys) + { + Hash.FromJson(fields[key]); + } + } + + private static (JsonObject Evidence, string Terminal) ValidateEvidence(Json? value, Limits limits, Budget budget) + { + JsonObject evidence = Exact(value, "core", "host_observations"); + JsonObject core = Exact(evidence["core"], "capabilities", "effect_requests", "interface", "invocation_id", "schema", "target", "terminal"); + string? terminal = core["terminal"].AsString(); + if (terminal is null || !Terminals.Contains(terminal)) + { + throw ProtocolException.Malformed("unknown core evidence schema or terminal"); + } + + if (!Json.Of("jacquard-host-core-evidence-v0").Equals(core["schema"])) + { + throw ProtocolException.Malformed("unknown core evidence schema or terminal"); + } + + if (!Json.Of(InvocationId).Equals(core["invocation_id"])) + { + throw ProtocolException.Order("wrong evidence invocation"); + } + + JsonObject target = Exact(core["target"], "callable", "kind"); + Hash.FromJson(target["callable"]); + if (!Json.Of("store-term-v0").Equals(target["kind"])) + { + throw ProtocolException.Malformed("unknown target kind"); + } + + JsonObject spec = Exact(core["interface"], "effects", "parameters", "result"); + SortedHashes(spec["effects"], limits.MaxEffects); + IReadOnlyList parameters = AsArray(spec["parameters"]); + budget.Count(parameters.Count, true); + foreach (Json parameter in parameters) + { + DecodeType(parameter, budget); + } + + DecodeType(spec["result"], budget); + JsonObject capabilities = Exact(core["capabilities"], "effects", "operations"); + SortedHashes(capabilities["effects"], limits.MaxEffects); + IReadOnlyList operations = AsArray(capabilities["operations"]); + if (operations.Count > limits.MaxOperations) + { + throw ProtocolException.Exceeded("too many evidence operations"); + } + + foreach (Json operation in operations) + { + JsonObject entry = Exact(operation, "effect", "mode", "operation"); + Hashes(entry, "effect", "operation"); + if (!Json.Of("once").Equals(entry["mode"])) + { + throw ProtocolException.Malformed("unknown operation mode"); + } + } + + IReadOnlyList requests = AsArray(core["effect_requests"]); + if (requests.Count > limits.MaxEffectRequests) + { + throw ProtocolException.Exceeded("too many evidence requests"); + } + + for (int index = 0; index < requests.Count; index++) + { + JsonObject entry = Exact(requests[index], "effect", "operation", "ordinal"); + Hashes(entry, "effect", "operation"); + if (!Json.Of(index + 1L).Equals(entry["ordinal"])) + { + throw ProtocolException.Malformed("evidence ordinals must be consecutive"); + } + } + + JsonObject observations = Exact(evidence["host_observations"], "responses", "schema"); + if (!Json.Of("jacquard-host-observations-v0").Equals(observations["schema"])) + { + throw ProtocolException.Malformed("unknown host observations schema"); + } + + foreach (Json response in AsArray(observations["responses"])) + { + JsonObject entry = Exact(response, "category", "completion", "ordinal"); + string? category = entry["category"].AsString(); + if (category is null || !Categories.Contains(category)) + { + throw ProtocolException.Malformed("unknown observation category"); + } + + string? completion = entry["completion"].AsString(); + if (completion is null || !Completions.Contains(completion)) + { + throw ProtocolException.Malformed("unknown observation completion"); + } + + long? ordinal = entry["ordinal"].AsInt(); + if (ordinal is null || ordinal < 1 || ordinal > requests.Count) + { + throw ProtocolException.Malformed("invalid observation ordinal"); + } + } + + return (evidence, terminal); + } + + private static void Invocation(JsonObject fields) + { + if (!string.Equals(AsStr(fields["invocation_id"]), InvocationId, StringComparison.Ordinal)) + { + throw ProtocolException.Order("frame names the wrong invocation"); + } + } + + /// Validates one Core frame after core_hello. + /// The frozen code Core's frame earns. + public static CoreFrame ParseCoreFrame(JsonObject frame, Limits limits) + { + ArgumentNullException.ThrowIfNull(frame); + ArgumentNullException.ThrowIfNull(limits); + string kind = frame.Get("kind")?.AsString() ?? string.Empty; + if (kind == "core_hello") + { + throw ProtocolException.Order("core_hello may only be the first frame"); + } + + if (!Json.Of(Compat.Protocol).Equals(frame.Get("protocol"))) + { + throw ProtocolException.Malformed("unknown protocol"); + } + + switch (kind) + { + case "fatal": + Exact(frame, "diagnostics", "kind", "protocol"); + return new Fatal(Diagnostics(frame["diagnostics"], limits), frame); + case "shutdown_ack": + Exact(frame, "kind", "protocol"); + return new ShutdownAck(frame); + case "effect_request": + return ParseRequest(frame, limits); + case "outcome": + return ParseOutcome(frame, limits); + default: + throw ProtocolException.Malformed("unknown Core frame kind"); + } + } + + private static EffectRequest ParseRequest(JsonObject frame, Limits limits) + { + Exact(frame, "arguments", "effect", "invocation_id", "kind", "mode", "operation", "protocol", "request_id"); + Invocation(frame); + if (!Json.Of("once").Equals(frame["mode"])) + { + throw ProtocolException.Malformed("unknown operation mode"); + } + + string id = AsStr(frame["request_id"]); + if (!Hash.LowerHex(id, 16)) + { + throw ProtocolException.Malformed("request_id must have 16 lowercase hexadecimal digits"); + } + + ulong ordinal = ulong.Parse(id, NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture); + if (ordinal == 0) + { + throw ProtocolException.Order("request ordinals start at one"); + } + + if (ordinal > Limits.MaxSafeInteger) + { + throw ProtocolException.Order("request ordinal is not consecutive"); + } + + var budget = new Budget(limits); + IReadOnlyList items = Counted(frame["arguments"], budget, true); + Hash effect = Hash.FromJson(frame["effect"]); + Hash operation = Hash.FromJson(frame["operation"]); + var arguments = new List(items.Count); + foreach (Json item in items) + { + arguments.Add(DecodeValue(item, budget)); + } + + return new EffectRequest((long)ordinal, effect, operation, arguments, frame); + } + + private static Outcome ParseOutcome(JsonObject frame, Limits limits) + { + Exact(frame, "evidence", "invocation_id", "kind", "protocol", "result"); + Invocation(frame); + + // Core encodes evidence types and the result value against one budget. + var budget = new Budget(limits); + (JsonObject evidence, string terminal) = ValidateEvidence(frame["evidence"], limits, budget); + JsonObject result = AsObject(frame["result"]); + string? name = result.Get("kind")?.AsString(); + if (name == "ok") + { + Exact(result, "kind", "value"); + if (terminal != "ok") + { + throw ProtocolException.Malformed("ok result has a non-ok terminal"); + } + + Value decoded = DecodeValue(result["value"], budget); + return new Outcome(decoded, Array.Empty(), terminal, evidence, frame); + } + + if (name == "error") + { + Exact(result, "diagnostics", "kind"); + if (terminal == "ok") + { + throw ProtocolException.Malformed("error result has an ok terminal"); + } + + List parsed = Diagnostics(result["diagnostics"], limits); + return new Outcome(null, parsed, terminal, evidence, frame); + } + + throw ProtocolException.Malformed("unknown result kind"); + } +} diff --git a/dotnet/src/JacquardHost/Protocol/Frames.cs b/dotnet/src/JacquardHost/Protocol/Frames.cs new file mode 100644 index 0000000..f4f7a37 --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Frames.cs @@ -0,0 +1,218 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace JacquardLang.Host.Protocol; + +/// A parsed Core frame after core_hello: an or a . +public abstract class CoreFrame +{ + private protected CoreFrame(JsonObject raw) + { + Raw = raw; + } + + /// The exact frame. + public JsonObject Raw { get; } +} + +/// Core asking the host to perform one operation. +public sealed class EffectRequest : CoreFrame +{ + internal EffectRequest(long ordinal, Hash effect, Hash operation, IReadOnlyList arguments, JsonObject raw) + : base(raw) + { + Ordinal = ordinal; + Effect = effect; + Operation = operation; + Arguments = arguments.ToArray(); + } + + /// The request ordinal, from one. + public long Ordinal { get; } + + /// The effect identity. + public Hash Effect { get; } + + /// The operation identity. + public Hash Operation { get; } + + /// The decoded arguments. + public IReadOnlyList Arguments { get; } +} + +/// Core's final frame: an , a , or a . +public abstract class Terminal : CoreFrame +{ + private protected Terminal(JsonObject raw) + : base(raw) + { + } +} + +/// +/// An invocation outcome: a value (null for an error result), or +/// diagnostics, with Core's evidence and terminal name. +/// +public sealed class Outcome : Terminal +{ + internal Outcome(Value? value, IReadOnlyList diagnostics, string terminalName, JsonObject evidence, JsonObject raw) + : base(raw) + { + Value = value; + Diagnostics = diagnostics.ToArray(); + TerminalName = terminalName; + Evidence = evidence; + } + + /// The value of an ok result, else null. + public Value? Value { get; } + + /// The diagnostics of an error result. + public IReadOnlyList Diagnostics { get; } + + /// Core's evidence terminal name, such as "ok". + public string TerminalName { get; } + + /// Core's evidence. + public JsonObject Evidence { get; } +} + +/// A session-level failure. +public sealed class Fatal : Terminal +{ + internal Fatal(IReadOnlyList diagnostics, JsonObject raw) + : base(raw) + { + Diagnostics = diagnostics.ToArray(); + } + + /// Core's diagnostics. + public IReadOnlyList Diagnostics { get; } +} + +/// The acknowledgement of a host shutdown. +public sealed class ShutdownAck : Terminal +{ + internal ShutdownAck(JsonObject raw) + : base(raw) + { + } +} + +/// +/// One jacquard-diagnostic-v1 entry; keeps the exact +/// object. is empty when Core sent null. +/// +/// The diagnostic code, or empty. +/// The domain. +/// The severity. +/// The summary (non-normative prose). +/// The cause (non-normative prose). +/// The next step (non-normative prose). +/// The exact object. +public sealed record Diagnostic(string Code, string Domain, string Severity, string Summary, string Cause, string NextStep, JsonObject Raw); + +/// The category of a host effect failure. +public enum FailureCategory +{ + /// The host does not provide the operation (not started). + UnsupportedOperation, + + /// The host refused to authorize the operation (not started). + RefusedAuthority, + + /// An outside action failed without effect. + OutsideFailure, + + /// An outside action's completion is unknown. + CompletionUnknown, +} + +/// The reason a pending request is withdrawn. +public enum CancelReason +{ + /// The caller cancelled. + Cancelled, + + /// A deadline passed. + Timeout, + + /// The host is shutting down. + HostShutdown, +} + +/// The host session state. +public enum State +{ + /// Awaiting core_hello. + Hello, + + /// Hello received; awaiting the host's selection. + Negotiating, + + /// Limits selected. + Selected, + + /// Shutdown sent. + ShuttingDown, + + /// The invocation is running. + Running, + + /// An effect request awaits its response. + Waiting, + + /// A terminal response was sent; awaiting the outcome. + Finishing, + + /// Core's terminal was received. + Closed, +} + +/// The wire names of the protocol enumerations. +public static class WireNames +{ + /// The wire name, such as "refused_authority". + public static string WireName(this FailureCategory category) => category switch + { + FailureCategory.UnsupportedOperation => "unsupported_operation", + FailureCategory.RefusedAuthority => "refused_authority", + FailureCategory.OutsideFailure => "outside_failure", + FailureCategory.CompletionUnknown => "completion_unknown", + _ => throw new ArgumentOutOfRangeException(nameof(category)), + }; + + /// The one completion the protocol permits with the category. + public static string Completion(this FailureCategory category) => category switch + { + FailureCategory.UnsupportedOperation => "not_started", + FailureCategory.RefusedAuthority => "not_started", + FailureCategory.OutsideFailure => "failed", + FailureCategory.CompletionUnknown => "unknown", + _ => throw new ArgumentOutOfRangeException(nameof(category)), + }; + + /// The wire name, such as "host_shutdown". + public static string WireName(this CancelReason reason) => reason switch + { + CancelReason.Cancelled => "cancelled", + CancelReason.Timeout => "timeout", + CancelReason.HostShutdown => "host_shutdown", + _ => throw new ArgumentOutOfRangeException(nameof(reason)), + }; + + /// The state's protocol name, such as "shutting_down". + public static string WireName(this State state) => state switch + { + State.Hello => "hello", + State.Negotiating => "negotiating", + State.Selected => "selected", + State.ShuttingDown => "shutting_down", + State.Running => "running", + State.Waiting => "waiting", + State.Finishing => "finishing", + State.Closed => "closed", + _ => throw new ArgumentOutOfRangeException(nameof(state)), + }; +} diff --git a/dotnet/src/JacquardHost/Protocol/Hash.cs b/dotnet/src/JacquardHost/Protocol/Hash.cs new file mode 100644 index 0000000..6ed84b8 --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Hash.cs @@ -0,0 +1,68 @@ +using System; + +namespace JacquardLang.Host.Protocol; + +/// A HASH_V0 identity: exactly 64 lowercase hexadecimal digits. +public sealed record Hash : IComparable +{ + private const string Message = "HASH_V0 must be exactly 64 lowercase hexadecimal digits"; + + /// Validates; prefer , which reports E1601. + /// is not 64 lowercase hexadecimal digits. + public Hash(string hex) + { + if (!LowerHex(hex, 64)) + { + throw new ArgumentException(Message, nameof(hex)); + } + + Hex = hex; + } + + /// The 64 lowercase hexadecimal digits. + public string Hex { get; } + + /// Validates a HASH_V0 identity. + /// E1601 when is not 64 lowercase hexadecimal digits. + public static Hash Parse(string? text) + { + if (!LowerHex(text, 64)) + { + throw ProtocolException.Malformed(Message); + } + + return new Hash(text!); + } + + internal static Hash FromJson(Json? value) => Parse(value?.AsString()); + + internal static bool LowerHex(string? text, int length) + { + if (text is null || text.Length != length) + { + return false; + } + + foreach (char c in text) + { + if (!((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f'))) + { + return false; + } + } + + return true; + } + + /// + public int CompareTo(Hash? other) => other is null ? 1 : string.CompareOrdinal(Hex, other.Hex); + + /// + public bool Equals(Hash? other) => other is not null && string.Equals(Hex, other.Hex, StringComparison.Ordinal); + + /// + public override int GetHashCode() => StringComparer.Ordinal.GetHashCode(Hex); + + /// + public override string ToString() => Hex; +} diff --git a/dotnet/src/JacquardHost/Protocol/Json.cs b/dotnet/src/JacquardHost/Protocol/Json.cs new file mode 100644 index 0000000..4ca946a --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Json.cs @@ -0,0 +1,395 @@ +using System; +using System.Collections; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using System.Linq; + +namespace JacquardLang.Host.Protocol; + +/// +/// A decoded JSON value: , , +/// , , , or +/// . Values are immutable and compare structurally; +/// objects keep their keys in code-point order, the order the canonical +/// encoding uses. +/// +public abstract class Json : IEquatable +{ + /// The smallest Core Int and JSON integer: -2^62. + public const long IntMin = -(1L << 62); + + /// The largest Core Int and JSON integer: 2^62 - 1. + public const long IntMax = (1L << 62) - 1; + + private protected Json() + { + } + + /// JSON null. + public static JsonNull Null { get; } = new(); + + /// Orders strings by Unicode code point, as Rust, Go, and Python do. + public static IComparer CodePointOrder { get; } = new CodePointComparer(); + + /// A string value. + public static JsonString Of(string value) => new(value); + + /// An integer value; the encoder refuses one outside the 63-bit range. + public static JsonInt Of(long value) => new(value); + + /// A boolean value. + public static JsonBool Of(bool value) => value ? JsonBool.True : JsonBool.False; + + /// An array value holding a copy of . + public static JsonArray Of(IEnumerable items) => new(items); + + /// An object from key/value pairs, such as Json.Object(("kind", Json.Of("int"))). + /// A key repeats. + [SuppressMessage("Naming", "CA1720", Justification = "JSON's own name for the value.")] + public static JsonObject Object(params (string Key, Json Value)[] fields) + { + ArgumentNullException.ThrowIfNull(fields); + return new JsonObject(fields.Select(field => new KeyValuePair(field.Key, field.Value))); + } + + /// An object holding a sorted copy of . + /// A key repeats. + [SuppressMessage("Naming", "CA1720", Justification = "JSON's own name for the value.")] + public static JsonObject Object(IEnumerable> fields) => new(fields); + + /// The object, or null if this is not an object. + public JsonObject? AsObject() => this as JsonObject; + + /// The string, or null if this is not a string. + public string? AsString() => (this as JsonString)?.Value; + + /// The items, or null if this is not an array. + public IReadOnlyList? AsArray() => (this as JsonArray)?.Items; + + /// The integer, or null if this is not an integer. + public long? AsInt() => (this as JsonInt)?.Value; + + /// + public abstract bool Equals(Json? other); + + /// + public override bool Equals(object? obj) => Equals(obj as Json); + + /// + public abstract override int GetHashCode(); + + /// The canonical encoding, or a placeholder for a value that has none. + public override string ToString() + { + try + { + return JsonCodec.Stringify(this, 256); + } + catch (ProtocolException error) + { + return ""; + } + } + + private sealed class CodePointComparer : IComparer + { + public int Compare(string? x, string? y) + { + if (ReferenceEquals(x, y)) + { + return 0; + } + + if (x is null) + { + return -1; + } + + if (y is null) + { + return 1; + } + + int i = 0; + int j = 0; + while (i < x.Length && j < y.Length) + { + int a = CodePoint(x, i, out int aLength); + int b = CodePoint(y, j, out int bLength); + if (a != b) + { + return a.CompareTo(b); + } + + i += aLength; + j += bLength; + } + + return (i < x.Length).CompareTo(j < y.Length); + } + + /// The code point at ; a lone surrogate is its own unit. + private static int CodePoint(string text, int index, out int length) + { + char c = text[index]; + if (char.IsHighSurrogate(c) && index + 1 < text.Length && char.IsLowSurrogate(text[index + 1])) + { + length = 2; + return char.ConvertToUtf32(c, text[index + 1]); + } + + length = 1; + return c; + } + } +} + +/// JSON null. +public sealed class JsonNull : Json +{ + internal JsonNull() + { + } + + /// + public override bool Equals(Json? other) => other is JsonNull; + + /// + public override int GetHashCode() => 0; +} + +/// A JSON boolean. +public sealed class JsonBool : Json +{ + internal static readonly JsonBool True = new(true); + internal static readonly JsonBool False = new(false); + + private JsonBool(bool value) + { + Value = value; + } + + /// The boolean. + public bool Value { get; } + + /// + public override bool Equals(Json? other) => other is JsonBool flag && flag.Value == Value; + + /// + public override int GetHashCode() => Value ? 1 : 2; +} + +/// A JSON integer. Decoding admits only Core's 63-bit range. +public sealed class JsonInt : Json +{ + internal JsonInt(long value) + { + Value = value; + } + + /// The integer. + public long Value { get; } + + /// + public override bool Equals(Json? other) => other is JsonInt number && number.Value == Value; + + /// + public override int GetHashCode() => Value.GetHashCode(); +} + +/// A JSON string. +public sealed class JsonString : Json +{ + internal JsonString(string value) + { + ArgumentNullException.ThrowIfNull(value); + Value = value; + } + + /// The string. + public string Value { get; } + + /// + public override bool Equals(Json? other) => other is JsonString text && string.Equals(text.Value, Value, StringComparison.Ordinal); + + /// + public override int GetHashCode() => StringComparer.Ordinal.GetHashCode(Value); +} + +/// A JSON array. +public sealed class JsonArray : Json +{ + private readonly Json[] items; + + internal JsonArray(IEnumerable items) + { + ArgumentNullException.ThrowIfNull(items); + this.items = items.ToArray(); + foreach (Json item in this.items) + { + ArgumentNullException.ThrowIfNull(item, nameof(items)); + } + } + + /// The items. + public IReadOnlyList Items => items; + + /// + public override bool Equals(Json? other) + { + if (other is not JsonArray array || array.items.Length != items.Length) + { + return false; + } + + for (int i = 0; i < items.Length; i++) + { + if (!items[i].Equals(array.items[i])) + { + return false; + } + } + + return true; + } + + /// + public override int GetHashCode() + { + HashCode hash = default; + hash.Add(items.Length); + foreach (Json item in items) + { + hash.Add(item); + } + + return hash.ToHashCode(); + } +} + +/// A JSON object; it enumerates its fields in code-point order of their keys. +public sealed class JsonObject : Json, IReadOnlyDictionary +{ + private readonly string[] keys; + private readonly Json[] values; + private readonly Dictionary index; + + internal JsonObject(IEnumerable> fields) + { + ArgumentNullException.ThrowIfNull(fields); + var keyList = new List(); + var valueList = new List(); + index = new Dictionary(StringComparer.Ordinal); + foreach (KeyValuePair field in fields) + { + ArgumentNullException.ThrowIfNull(field.Key, nameof(fields)); + ArgumentNullException.ThrowIfNull(field.Value, nameof(fields)); + if (!index.TryAdd(field.Key, keyList.Count)) + { + throw new ArgumentException("duplicate JSON field name", nameof(fields)); + } + + keyList.Add(field.Key); + valueList.Add(field.Value); + } + + keys = keyList.ToArray(); + values = valueList.ToArray(); + Array.Sort(keys, values, CodePointOrder); + for (int i = 0; i < keys.Length; i++) + { + index[keys[i]] = i; + } + } + + /// The empty object. + public static JsonObject Empty { get; } = new(Array.Empty>()); + + /// + public int Count => keys.Length; + + /// + public IEnumerable Keys => keys; + + /// + public IEnumerable Values => values; + + /// + public Json this[string key] => values[index[key]]; + + /// The value for , or null. + public Json? Get(string key) => index.TryGetValue(key, out int at) ? values[at] : null; + + /// A copy with set to . + public JsonObject With(string key, Json value) + { + var copy = new Dictionary(this, StringComparer.Ordinal) + { + [key] = value, + }; + return new JsonObject(copy); + } + + /// A copy without . + public JsonObject Without(string key) => new(this.Where(field => !string.Equals(field.Key, key, StringComparison.Ordinal))); + + /// + public bool ContainsKey(string key) => index.ContainsKey(key); + + /// + public bool TryGetValue(string key, out Json value) + { + if (index.TryGetValue(key, out int at)) + { + value = values[at]; + return true; + } + + value = Null; + return false; + } + + /// + public IEnumerator> GetEnumerator() + { + for (int i = 0; i < keys.Length; i++) + { + yield return new KeyValuePair(keys[i], values[i]); + } + } + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + + /// + public override bool Equals(Json? other) + { + if (other is not JsonObject obj || obj.keys.Length != keys.Length) + { + return false; + } + + for (int i = 0; i < keys.Length; i++) + { + if (!string.Equals(keys[i], obj.keys[i], StringComparison.Ordinal) || !values[i].Equals(obj.values[i])) + { + return false; + } + } + + return true; + } + + /// + public override int GetHashCode() + { + HashCode hash = default; + hash.Add(keys.Length); + for (int i = 0; i < keys.Length; i++) + { + hash.Add(keys[i], StringComparer.Ordinal); + hash.Add(values[i]); + } + + return hash.ToHashCode(); + } +} diff --git a/dotnet/src/JacquardHost/Protocol/JsonCodec.cs b/dotnet/src/JacquardHost/Protocol/JsonCodec.cs new file mode 100644 index 0000000..fe34b6f --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/JsonCodec.cs @@ -0,0 +1,695 @@ +using System; +using System.Buffers.Binary; +using System.Collections.Generic; +using System.Globalization; +using System.Text; + +namespace JacquardLang.Host.Protocol; + +/// +/// Strict UTF-8 JSON for the carrier. Decoding refuses invalid UTF-8, +/// duplicate keys, lone surrogates, non-integer numbers, integers outside +/// Core's 63-bit range, trailing data, and nesting deeper than the selected +/// limit. Encoding is canonical: object keys sorted by code point, no +/// whitespace, and the same string escaping as JavaScript's +/// JSON.stringify. +/// +/// +/// The parser is written by hand rather than on System.Text.Json's +/// reader, so that every rule above, and the order in which they are checked, +/// is visibly the same as in the other adapters. Integers are parsed from +/// their decimal text as a , never through a +/// or a . +/// +public static class JsonCodec +{ + private static readonly UTF8Encoding StrictUtf8 = new(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true); + private static readonly byte[] Hex = "0123456789abcdef"u8.ToArray(); + + /// Decodes one JSON value under the strict rules. + /// E1601 for malformed input, E1602 for nesting deeper than . + public static Json Decode(ReadOnlySpan payload, int maxDepth) + { + string text; + try + { + text = StrictUtf8.GetString(payload); + } + catch (DecoderFallbackException) + { + throw ProtocolException.Malformed("payload is not valid UTF-8"); + } + + var decoder = new Decoder(text, maxDepth); + Json value = decoder.Value(1); + decoder.Whitespace(); + if (decoder.Position != text.Length) + { + throw ProtocolException.Malformed("payload is not one JSON value"); + } + + return value; + } + + /// Decodes one JSON object. + /// As , and E1601 for any other value. + public static JsonObject DecodeObject(ReadOnlySpan payload, int maxDepth) => + Decode(payload, maxDepth) as JsonObject ?? throw ProtocolException.Malformed("expected a JSON object"); + + private sealed class Decoder + { + private readonly string text; + private readonly int maxDepth; + + internal Decoder(string text, int maxDepth) + { + this.text = text; + this.maxDepth = maxDepth; + } + + internal int Position { get; private set; } + + private int Peek() => Position < text.Length ? text[Position] : -1; + + private bool StartsWith(string literal) => string.CompareOrdinal(text, Position, literal, 0, literal.Length) == 0; + + internal void Whitespace() + { + while (Position < text.Length) + { + char c = text[Position]; + if (c != ' ' && c != '\t' && c != '\r' && c != '\n') + { + return; + } + + Position++; + } + } + + internal Json Value(int depth) + { + Whitespace(); + int next = Peek(); + if (next == '"') + { + return Json.Of(Quoted()); + } + + if (next == '{' || next == '[') + { + if (depth > maxDepth) + { + throw ProtocolException.Exceeded("JSON nesting exceeds max_json_depth"); + } + + return Container((char)next, depth); + } + + if (StartsWith("true")) + { + Position += 4; + return Json.Of(true); + } + + if (StartsWith("false")) + { + Position += 5; + return Json.Of(false); + } + + if (StartsWith("null")) + { + Position += 4; + return Json.Null; + } + + int start = Position; + if (Peek() == '-') + { + Position++; + } + + int first = Peek(); + if (first == '0') + { + Position++; + } + else if (first >= '1' && first <= '9') + { + while (Peek() >= '0' && Peek() <= '9') + { + Position++; + } + } + else + { + throw ProtocolException.Malformed("invalid JSON scalar"); + } + + if (Position - start > 20) + { + throw ProtocolException.Malformed("invalid JSON scalar"); + } + + // The token is an optional '-' and ASCII digits, so this parses + // exactly; a fraction or exponent is left unread and refused by + // the caller as trailing data. + if (!long.TryParse(text.AsSpan(start, Position - start), NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out long number) + || number < Json.IntMin || number > Json.IntMax) + { + throw ProtocolException.Malformed("JSON integer is outside the 63-bit range"); + } + + return Json.Of(number); + } + + private Json Container(char open, int depth) + { + Position++; + Whitespace(); + char close = open == '{' ? '}' : ']'; + var items = new List(); + var fields = new Dictionary(StringComparer.Ordinal); + if (Peek() == close) + { + Position++; + return Finish(open, items, fields); + } + + while (Position < text.Length) + { + if (open == '[') + { + items.Add(Value(depth + 1)); + } + else + { + if (Peek() != '"') + { + throw ProtocolException.Malformed("object key must be a string"); + } + + string key = Quoted(); + Whitespace(); + if (fields.ContainsKey(key)) + { + throw ProtocolException.Malformed("duplicate JSON field name"); + } + + if (Peek() != ':') + { + throw ProtocolException.Malformed("expected a colon"); + } + + Position++; + fields.Add(key, Value(depth + 1)); + } + + Whitespace(); + if (Peek() == close) + { + Position++; + return Finish(open, items, fields); + } + + if (Peek() != ',') + { + throw ProtocolException.Malformed("expected a comma"); + } + + Position++; + Whitespace(); + } + + throw ProtocolException.Malformed("unterminated JSON container"); + } + + private static Json Finish(char open, List items, Dictionary fields) => + open == '{' ? Json.Object(fields) : Json.Of(items); + + private int Hex4() + { + if (Position + 4 > text.Length) + { + return -1; + } + + int value = 0; + for (int i = 0; i < 4; i++) + { + char c = text[Position + i]; + int digit = c >= '0' && c <= '9' ? c - '0' : c >= 'a' && c <= 'f' ? c - 'a' + 10 : c >= 'A' && c <= 'F' ? c - 'A' + 10 : -1; + if (digit < 0) + { + return -1; + } + + value = (value * 16) + digit; + } + + Position += 4; + return value; + } + + private string Quoted() + { + Position++; + var output = new StringBuilder(); + while (true) + { + if (Position >= text.Length) + { + throw ProtocolException.Malformed("unterminated JSON string"); + } + + char c = text[Position]; + if (c == '"') + { + Position++; + return output.ToString(); + } + + if (c < 0x20) + { + throw ProtocolException.Malformed("invalid JSON string"); + } + + if (c != '\\') + { + // The payload was validated as UTF-8, so surrogates arrive paired. + output.Append(c); + Position++; + continue; + } + + Position++; + if (Position >= text.Length) + { + throw ProtocolException.Malformed("unterminated JSON string"); + } + + char escape = text[Position++]; + switch (escape) + { + case '"': + output.Append('"'); + break; + case '\\': + output.Append('\\'); + break; + case '/': + output.Append('/'); + break; + case 'b': + output.Append('\b'); + break; + case 'f': + output.Append('\f'); + break; + case 'n': + output.Append('\n'); + break; + case 'r': + output.Append('\r'); + break; + case 't': + output.Append('\t'); + break; + case 'u': + Unicode(output); + break; + default: + throw ProtocolException.Malformed("invalid JSON string"); + } + } + } + + private void Unicode(StringBuilder output) + { + int unit = Hex4(); + if (unit < 0) + { + throw ProtocolException.Malformed("invalid JSON string"); + } + + if (unit >= 0xd800 && unit < 0xdc00) + { + if (!StartsWith("\\u")) + { + throw ProtocolException.Malformed("a JSON string contains a lone surrogate"); + } + + Position += 2; + int low = Hex4(); + if (low < 0) + { + throw ProtocolException.Malformed("invalid JSON string"); + } + + if (low < 0xdc00 || low >= 0xe000) + { + throw ProtocolException.Malformed("a JSON string contains a lone surrogate"); + } + + output.Append((char)unit).Append((char)low); + } + else if (unit >= 0xdc00 && unit < 0xe000) + { + throw ProtocolException.Malformed("a JSON string contains a lone surrogate"); + } + else + { + output.Append((char)unit); + } + } + } + + /// Encodes a JSON value canonically, without a size bound. + /// E1601 for an integer outside the 63-bit range or a lone surrogate, E1602 for nesting deeper than . + public static string Stringify(Json value, int maxDepth) + { + ArgumentNullException.ThrowIfNull(value); + var sink = new Sink(Array.MaxLength, 0); + Write(sink, value, maxDepth, 1); + return Encoding.UTF8.GetString(sink.Payload); + } + + /// + /// Encodes one carrier frame: a big-endian u32 length and the payload, + /// refusing payloads outside 1..=. + /// The encoder counts UTF-8 bytes as it writes them and stops as soon as + /// the count passes , so a value whose encoding + /// is far larger than a frame costs at most about one frame of memory and + /// time. A that is not a + /// is bounded by its field count and key lengths before it is copied or + /// sorted. + /// + /// E1602 when the payload exceeds or nesting exceeds ; E1601 as for . + public static byte[] FrameBytes(IReadOnlyDictionary frame, int maxBytes, int maxDepth) + { + ArgumentNullException.ThrowIfNull(frame); + JsonObject obj = Bounded(frame, maxBytes); + var sink = new Sink(maxBytes, 4); + Write(sink, obj, maxDepth, 1); + ReadOnlySpan payload = sink.Payload; + if (payload.Length == 0 || payload.Length > maxBytes) + { + throw ProtocolException.Exceeded("payload exceeds max_frame_bytes"); + } + + byte[] bytes = sink.Frame(); + BinaryPrimitives.WriteUInt32BigEndian(bytes, (uint)payload.Length); + return bytes; + } + + /// + /// The frame as a sorted object. A caller's own map is refused before it + /// is copied when its fields cannot fit: each field encodes to at least + /// its key's length in chars plus four bytes (two quotes, a colon, and a + /// one-byte value), and every field but one needs a comma. + /// + internal static JsonObject Bounded(IReadOnlyDictionary frame, int maxBytes) + { + if (frame is JsonObject sorted) + { + return sorted; + } + + long total = 2; + if (total + (frame.Count * 5L) - 1 > maxBytes) + { + throw ProtocolException.Exceeded("payload exceeds max_frame_bytes"); + } + + var fields = new List>(frame.Count); + foreach (KeyValuePair field in frame) + { + total += (field.Key?.Length ?? 0) + 5L; + if (total - 1 > maxBytes) + { + throw ProtocolException.Exceeded("payload exceeds max_frame_bytes"); + } + + fields.Add(field); + } + + return Json.Object(fields); + } + + /// Whether is a sequence of Unicode scalars (no lone surrogate). + public static bool IsScalar(string text) + { + ArgumentNullException.ThrowIfNull(text); + for (int i = 0; i < text.Length; i++) + { + char c = text[i]; + if (char.IsHighSurrogate(c)) + { + if (i + 1 >= text.Length || !char.IsLowSurrogate(text[i + 1])) + { + return false; + } + + i++; + } + else if (char.IsLowSurrogate(c)) + { + return false; + } + } + + return true; + } + + /// The UTF-8 length of a scalar string. + public static long Utf8Length(string text) + { + ArgumentNullException.ThrowIfNull(text); + long length = 0; + for (int i = 0; i < text.Length; i++) + { + char c = text[i]; + if (c < 0x80) + { + length += 1; + } + else if (c < 0x800) + { + length += 2; + } + else if (char.IsHighSurrogate(c) && i + 1 < text.Length && char.IsLowSurrogate(text[i + 1])) + { + length += 4; + i++; + } + else + { + length += 3; + } + } + + return length; + } + + private static void Write(Sink output, Json value, int maxDepth, int depth) + { + switch (value) + { + case JsonNull: + output.Ascii("null"); + break; + case JsonBool flag: + output.Ascii(flag.Value ? "true" : "false"); + break; + case JsonInt number: + if (number.Value < Json.IntMin || number.Value > Json.IntMax) + { + throw ProtocolException.Malformed("JSON integer is outside the 63-bit range"); + } + + output.Ascii(number.Value.ToString(CultureInfo.InvariantCulture)); + break; + case JsonString text: + Quote(output, text.Value); + break; + case JsonArray array: + if (depth > maxDepth) + { + throw ProtocolException.Exceeded("JSON nesting exceeds max_json_depth"); + } + + output.Byte((byte)'['); + for (int i = 0; i < array.Items.Count; i++) + { + if (i > 0) + { + output.Byte((byte)','); + } + + Write(output, array.Items[i], maxDepth, depth + 1); + } + + output.Byte((byte)']'); + break; + case JsonObject obj: + if (depth > maxDepth) + { + throw ProtocolException.Exceeded("JSON nesting exceeds max_json_depth"); + } + + output.Byte((byte)'{'); + bool first = true; + foreach (KeyValuePair field in obj) + { + if (!first) + { + output.Byte((byte)','); + } + + first = false; + Quote(output, field.Key); + output.Byte((byte)':'); + Write(output, field.Value, maxDepth, depth + 1); + } + + output.Byte((byte)'}'); + break; + default: + throw new ArgumentException("unknown JSON value", nameof(value)); + } + } + + private static void Quote(Sink output, string text) + { + // Each char is at least one UTF-8 byte, so a text that cannot fit is + // refused before it is scanned. + if (output.Length + (long)text.Length + 2 > output.Limit) + { + throw ProtocolException.Exceeded("payload exceeds max_frame_bytes"); + } + + if (!IsScalar(text)) + { + throw ProtocolException.Malformed("a JSON string contains a lone surrogate"); + } + + output.Byte((byte)'"'); + for (int i = 0; i < text.Length; i++) + { + char c = text[i]; + switch (c) + { + case '"': + output.Ascii("\\\""); + break; + case '\\': + output.Ascii("\\\\"); + break; + case '\b': + output.Ascii("\\b"); + break; + case '\f': + output.Ascii("\\f"); + break; + case '\n': + output.Ascii("\\n"); + break; + case '\r': + output.Ascii("\\r"); + break; + case '\t': + output.Ascii("\\t"); + break; + default: + if (c < 0x20) + { + output.Ascii("\\u00"); + output.Byte(Hex[c >> 4]); + output.Byte(Hex[c & 0xf]); + } + else if (c < 0x80) + { + output.Byte((byte)c); + } + else if (c < 0x800) + { + output.Byte((byte)(0xc0 | (c >> 6))); + output.Byte((byte)(0x80 | (c & 0x3f))); + } + else if (char.IsHighSurrogate(c)) + { + // IsScalar guarantees the low half follows. + int point = char.ConvertToUtf32(c, text[++i]); + output.Byte((byte)(0xf0 | (point >> 18))); + output.Byte((byte)(0x80 | ((point >> 12) & 0x3f))); + output.Byte((byte)(0x80 | ((point >> 6) & 0x3f))); + output.Byte((byte)(0x80 | (point & 0x3f))); + } + else + { + output.Byte((byte)(0xe0 | (c >> 12))); + output.Byte((byte)(0x80 | ((c >> 6) & 0x3f))); + output.Byte((byte)(0x80 | (c & 0x3f))); + } + + break; + } + } + + output.Byte((byte)'"'); + } + + /// + /// The encoding so far, as UTF-8 bytes after a reserved prefix. Every + /// byte is counted as it is appended, and the encoding stops once it + /// passes ; the buffer never grows past the limit. + /// + private sealed class Sink + { + private readonly int prefix; + private byte[] buffer; + private int used; + + internal Sink(int limit, int prefix) + { + Limit = limit; + this.prefix = prefix; + buffer = new byte[prefix + Math.Min(limit, 256)]; + used = prefix; + } + + internal int Limit { get; } + + internal int Length => used - prefix; + + internal ReadOnlySpan Payload => buffer.AsSpan(prefix, used - prefix); + + /// The prefix and payload, exactly sized. + internal byte[] Frame() => buffer.AsSpan(0, used).ToArray(); + + internal void Byte(byte b) + { + if (used - prefix >= Limit) + { + throw ProtocolException.Exceeded("payload exceeds max_frame_bytes"); + } + + if (used == buffer.Length) + { + long grown = Math.Min((long)buffer.Length * 2, (long)prefix + Limit); + Array.Resize(ref buffer, (int)Math.Min(grown, Array.MaxLength)); + } + + buffer[used++] = b; + } + + internal void Ascii(string text) + { + foreach (char c in text) + { + Byte((byte)c); + } + } + } +} diff --git a/dotnet/src/JacquardHost/Protocol/Limits.cs b/dotnet/src/JacquardHost/Protocol/Limits.cs new file mode 100644 index 0000000..de8c52e --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Limits.cs @@ -0,0 +1,118 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace JacquardLang.Host.Protocol; + +/// The v0 ceilings a host selects; each must lie in 1..= its hard value. +public sealed record Limits( + int MaxArguments, + int MaxCollectionItems, + int MaxDiagnosticBytes, + int MaxDiagnostics, + int MaxEffectRequests, + int MaxEffects, + int MaxFrameBytes, + int MaxHostMessageBytes, + int MaxJsonDepth, + int MaxOperations, + int MaxStderrBytes, + int MaxTextBytes, + int MaxValueNodes) +{ + /// The largest integer the other adapters accept exactly in limits and ordinals. + internal const long MaxSafeInteger = (1L << 53) - 1; + + /// The frozen v0 ceilings Core advertises in core_hello. + public static Limits Hard { get; } = new(64, 1024, 65536, 32, 1024, 64, 1_048_576, 4096, 64, 256, 65536, 262_144, 4096); + + /// The wire names, in protocol (sorted) order. + public static IReadOnlyList Names { get; } = new[] + { + "max_arguments", + "max_collection_items", + "max_diagnostic_bytes", + "max_diagnostics", + "max_effect_requests", + "max_effects", + "max_frame_bytes", + "max_host_message_bytes", + "max_json_depth", + "max_operations", + "max_stderr_bytes", + "max_text_bytes", + "max_value_nodes", + }; + + private int[] Values() => new[] + { + MaxArguments, MaxCollectionItems, MaxDiagnosticBytes, MaxDiagnostics, MaxEffectRequests, MaxEffects, + MaxFrameBytes, MaxHostMessageBytes, MaxJsonDepth, MaxOperations, MaxStderrBytes, MaxTextBytes, MaxValueNodes, + }; + + private static Limits Of(int[] v) => new(v[0], v[1], v[2], v[3], v[4], v[5], v[6], v[7], v[8], v[9], v[10], v[11], v[12]); + + /// These limits with the one named by its wire name, such as "max_frame_bytes", replaced. + /// is not a limit's wire name. + public Limits With(string name, int value) + { + int index = Names.ToList().IndexOf(name); + if (index < 0) + { + throw new ArgumentException("unknown limit " + name, nameof(name)); + } + + int[] v = Values(); + v[index] = value; + return Of(v); + } + + /// Refuses a limit outside 1..= its hard ceiling. + /// E1602 for the first limit out of bounds. + public void Validate() + { + int[] v = Values(); + int[] hard = Hard.Values(); + for (int i = 0; i < v.Length; i++) + { + if (v[i] < 1 || v[i] > hard[i]) + { + throw ProtocolException.Exceeded("limit " + Names[i] + " exceeds its bounds"); + } + } + } + + /// The wire object for host_select. + public JsonObject ToJson() + { + int[] v = Values(); + return Json.Object(Names.Select((name, i) => new KeyValuePair(name, Json.Of(v[i])))); + } + + /// Parses and validates an exact limits object. + /// E1601 for another key set or a non-integer, E1602 for a limit out of bounds. + public static Limits FromJson(Json value) + { + JsonObject fields = Codec.Exact(value, Names.ToArray()); + int[] hard = Hard.Values(); + int[] parsed = new int[hard.Length]; + for (int i = 0; i < hard.Length; i++) + { + string name = Names[i]; + long? number = fields[name].AsInt(); + if (number is null || number < -MaxSafeInteger || number > MaxSafeInteger) + { + throw ProtocolException.Malformed("limit " + name + " must be an integer"); + } + + if (number < 1 || number > hard[i]) + { + throw ProtocolException.Exceeded("limit " + name + " exceeds its bounds"); + } + + parsed[i] = (int)number.Value; + } + + return Of(parsed); + } +} diff --git a/dotnet/src/JacquardHost/Protocol/Session.cs b/dotnet/src/JacquardHost/Protocol/Session.cs new file mode 100644 index 0000000..e52e58c --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Session.cs @@ -0,0 +1,271 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.Linq; + +namespace JacquardLang.Host.Protocol; + +/// +/// The carrier-independent v0 host state machine: one hello, one selection, +/// one invocation or shutdown, consecutive requests, and one terminal. A +/// session is used from one thread at a time. +/// +public sealed class Session +{ + private readonly List requests = new(); + private readonly List responses = new(); + + /// The current state. + public State State { get; private set; } = State.Hello; + + /// The selected limits; the hard limits until selection. + public Limits Limits { get; private set; } = Limits.Hard; + + /// The effect requests received, in order. + public IReadOnlyList Requests => requests; + + /// The responses rendered, in order. + public IReadOnlyList Responses => responses; + + /// Core's terminal frame, once received. + public Terminal? Terminal { get; private set; } + + /// The ordinal of the request awaiting a response, or null. + public long? Pending { get; private set; } + + /// Validates the next Core frame. Returns null for core_hello. + /// The frozen code the frame earns in this state. + public CoreFrame? Receive(JsonObject frame) + { + ArgumentNullException.ThrowIfNull(frame); + if (State == State.Closed) + { + throw ProtocolException.Order("Core sent a frame after its terminal"); + } + + if (State == State.Hello) + { + Hello(frame); + return null; + } + + CoreFrame parsed = Codec.ParseCoreFrame(frame, Limits); + switch (parsed) + { + case EffectRequest request: + if (State != State.Running) + { + throw ProtocolException.Order("effect request without an available invocation slot"); + } + + if (request.Ordinal != requests.Count + 1L) + { + throw ProtocolException.Order("request ordinal is not consecutive"); + } + + if (request.Ordinal > Limits.MaxEffectRequests) + { + throw ProtocolException.Exceeded("Core exceeded max_effect_requests"); + } + + requests.Add(request); + Pending = request.Ordinal; + State = State.Waiting; + break; + case Terminal end: + bool allowed = end switch + { + Outcome => State is State.Running or State.Waiting or State.Finishing, + Fatal => State is State.Negotiating or State.Selected or State.ShuttingDown or State.Running or State.Finishing, + _ => State == State.ShuttingDown, + }; + if (!allowed) + { + throw ProtocolException.Order(end switch + { + Outcome => "outcome without a live invocation", + Fatal => "fatal while a response is outstanding", + _ => "shutdown_ack without shutdown", + }); + } + + Pending = null; + Terminal = end; + State = State.Closed; + break; + } + + return parsed; + } + + private void Hello(JsonObject frame) + { + if (!Json.Of("core_hello").Equals(frame.Get("kind"))) + { + throw ProtocolException.Order("first Core frame must be core_hello"); + } + + Codec.Exact(frame, "carrier", "kind", "limits", "versions"); + if (!Json.Of(Compat.Carrier).Equals(frame["carrier"])) + { + throw ProtocolException.Malformed("unknown carrier"); + } + + IReadOnlyList versions = frame["versions"].AsArray() ?? throw ProtocolException.Malformed("expected an array"); + if (versions.Count == 0) + { + throw ProtocolException.Malformed("versions must be nonempty"); + } + + bool offered = false; + foreach (Json version in versions) + { + string name = version.AsString() ?? throw ProtocolException.Malformed("expected a string"); + offered |= name == Compat.Protocol; + } + + if (!Limits.FromJson(frame["limits"]).Equals(Limits.Hard)) + { + throw ProtocolException.Malformed("hello must advertise exactly the frozen limits"); + } + + if (!offered) + { + throw new ProtocolException("E1600", "Core does not offer " + Compat.Protocol); + } + + State = State.Negotiating; + } + + private void Require(params State[] states) + { + if (!states.Contains(State)) + { + throw ProtocolException.Order("message is not permitted in state " + State.WireName()); + } + } + + /// Chooses the session limits and renders host_select. + /// E1608 out of order; E1602 for a limit out of bounds. + public JsonObject Select(Limits selected) + { + ArgumentNullException.ThrowIfNull(selected); + Require(State.Negotiating); + selected.Validate(); + Limits = selected; + State = State.Selected; + return Json.Object(("kind", Json.Of("host_select")), ("limits", selected.ToJson()), ("protocol", Json.Of(Compat.Protocol))); + } + + /// Renders the session's single invocation. + /// E1608 out of order; otherwise as . + public JsonObject Invoke(Hash target, CallInterface spec, IReadOnlyList args, IReadOnlyList operations) + { + Require(State.Selected); + JsonObject frame = Codec.InvokeFrame(target, spec, args, operations, Limits); + State = State.Running; + return frame; + } + + /// Ends a selected session without invoking. + /// E1608 out of order. + public JsonObject Shutdown() + { + Require(State.Selected); + State = State.ShuttingDown; + return Json.Object(("kind", Json.Of("shutdown")), ("protocol", Json.Of(Compat.Protocol))); + } + + private static string RequestId(long ordinal) => ordinal.ToString("x16", CultureInfo.InvariantCulture); + + private JsonObject Respond(IEnumerable<(string Key, Json Value)> fields, bool terminalResponse) + { + long ordinal = Pending!.Value; + Pending = null; + var frame = new Dictionary(StringComparer.Ordinal); + foreach ((string key, Json value) in fields) + { + frame[key] = value; + } + + frame["protocol"] = Json.Of(Compat.Protocol); + frame["invocation_id"] = Json.Of(Codec.InvocationId); + frame["request_id"] = Json.Of(RequestId(ordinal)); + JsonObject rendered = Json.Object(frame); + responses.Add(rendered); + State = terminalResponse ? State.Finishing : State.Running; + return rendered; + } + + /// The effect_ok frame for a value, validated without changing state. + /// E1608 out of order; otherwise as . + public JsonObject OkFrame(Value? value) + { + Require(State.Waiting); + Json encoded = Codec.EncodeValue(value, new Budget(Limits)); + JsonObject frame = Json.Object( + ("invocation_id", Json.Of(Codec.InvocationId)), + ("kind", Json.Of("effect_ok")), + ("protocol", Json.Of(Compat.Protocol)), + ("request_id", Json.Of(RequestId(Pending!.Value))), + ("value", encoded)); + JsonCodec.FrameBytes(frame, Limits.MaxFrameBytes, Limits.MaxJsonDepth); + return frame; + } + + /// Answers the pending request with a value. + /// E1608 out of order; otherwise as . + public JsonObject RespondOk(Value? value) + { + Require(State.Waiting); + Json encoded = Codec.EncodeValue(value, new Budget(Limits)); + return Respond(new[] { ("kind", (Json)Json.Of("effect_ok")), ("value", encoded) }, false); + } + + /// Answers the pending request with a permitted failure. + /// E1608 out of order; E1601 for a lone surrogate; E1602 for a message over max_host_message_bytes. + public JsonObject RespondFailure(FailureCategory category, string message) + { + ArgumentNullException.ThrowIfNull(message); + Require(State.Waiting); + if (message.Length > Limits.MaxHostMessageBytes) + { + throw ProtocolException.Exceeded("host message exceeds max_host_message_bytes"); + } + + if (!JsonCodec.IsScalar(message)) + { + throw ProtocolException.Malformed("a JSON string contains a lone surrogate"); + } + + if (JsonCodec.Utf8Length(message) > Limits.MaxHostMessageBytes) + { + throw ProtocolException.Exceeded("host message exceeds max_host_message_bytes"); + } + + return Respond( + new[] + { + ("category", (Json)Json.Of(category.WireName())), + ("completion", Json.Of(category.Completion())), + ("kind", Json.Of("effect_failure")), + ("message", Json.Of(message)), + }, + true); + } + + /// Withdraws the pending request. + /// E1608 out of order; E1601 for an unknown completion. + public JsonObject Cancel(CancelReason reason, string completion) + { + Require(State.Waiting); + if (completion is not ("not_started" or "failed" or "unknown")) + { + throw ProtocolException.Malformed("unknown cancellation enum"); + } + + return Respond( + new[] { ("completion", (Json)Json.Of(completion)), ("kind", Json.Of("cancel")), ("reason", Json.Of(reason.WireName())) }, + true); + } +} diff --git a/dotnet/src/JacquardHost/Protocol/Value.cs b/dotnet/src/JacquardHost/Protocol/Value.cs new file mode 100644 index 0000000..2ff6615 --- /dev/null +++ b/dotnet/src/JacquardHost/Protocol/Value.cs @@ -0,0 +1,169 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace JacquardLang.Host.Protocol; + +/// +/// One v0 boundary value: , , +/// , , , +/// or . Values compare structurally. +/// +public abstract record Value +{ + private protected Value() + { + } + + /// The empty tuple, Jacquard's unit. + public static TupleValue Unit { get; } = new(Array.Empty()); + + internal static bool SameItems(Value[] a, Value[] b) + { + if (a.Length != b.Length) + { + return false; + } + + for (int i = 0; i < a.Length; i++) + { + if (!a[i].Equals(b[i])) + { + return false; + } + } + + return true; + } + + internal static int HashItems(Value[] items) + { + HashCode hash = default; + hash.Add(items.Length); + foreach (Value item in items) + { + hash.Add(item); + } + + return hash.ToHashCode(); + } + + internal static Value[] Copy(IEnumerable items, string name) + { + ArgumentNullException.ThrowIfNull(items, name); + Value[] copy = items.ToArray(); + foreach (Value item in copy) + { + ArgumentNullException.ThrowIfNull(item, name); + } + + return copy; + } +} + +/// +/// Core's signed 63-bit integer, held exactly in a . A +/// outside -2^62..2^62-1 can be held but has no +/// encoding (E1604). Money crosses the boundary as an integer count of minor +/// units, never as a . +/// +/// The integer. +public sealed record IntValue(long Value) : Value; + +/// A Unicode scalar string; a lone surrogate has no encoding (E1601). +public sealed record TextValue : Value +{ + /// A text value. + public TextValue(string value) + { + ArgumentNullException.ThrowIfNull(value); + Value = value; + } + + /// The string. + public string Value { get; } + + /// + public bool Equals(TextValue? other) => other is not null && string.Equals(Value, other.Value, StringComparison.Ordinal); + + /// + public override int GetHashCode() => StringComparer.Ordinal.GetHashCode(Value); +} + +/// Binary64 bits, preserving NaN payloads and signed zero. +/// The IEEE 754 binary64 bits. +public sealed record RealValue(ulong Bits) : Value +{ + /// The with the exact bits of . + public static RealValue Of(double value) => new((ulong)BitConverter.DoubleToInt64Bits(value)); + + /// The bits as a . + public double ToDouble() => BitConverter.Int64BitsToDouble((long)Bits); +} + +/// An explicit Hash. +public sealed record HashValue : Value +{ + /// A hash value. + public HashValue(Hash hash) + { + ArgumentNullException.ThrowIfNull(hash); + Hash = hash; + } + + /// The hash. + public Hash Hash { get; } +} + +/// A positional product; the empty tuple is unit. +public sealed record TupleValue : Value +{ + private readonly Value[] items; + + /// A tuple holding a copy of . + public TupleValue(IEnumerable items) + { + this.items = Copy(items, nameof(items)); + } + + /// A tuple of . + public TupleValue(params Value[] items) + : this((IEnumerable)items) + { + } + + /// The items. + public IReadOnlyList Items => items; + + /// + public bool Equals(TupleValue? other) => other is not null && SameItems(items, other.items); + + /// + public override int GetHashCode() => HashItems(items); +} + +/// A constructor identity and its positional arguments. +public sealed record ConstructorValue : Value +{ + private readonly Value[] arguments; + + /// A constructor value holding a copy of . + public ConstructorValue(Hash identity, IEnumerable arguments) + { + ArgumentNullException.ThrowIfNull(identity); + Identity = identity; + this.arguments = Copy(arguments, nameof(arguments)); + } + + /// The constructor identity. + public Hash Identity { get; } + + /// The arguments. + public IReadOnlyList Arguments => arguments; + + /// + public bool Equals(ConstructorValue? other) => other is not null && Identity.Equals(other.Identity) && SameItems(arguments, other.arguments); + + /// + public override int GetHashCode() => HashCode.Combine(Identity, HashItems(arguments)); +} diff --git a/dotnet/src/JacquardHost/ProtocolException.cs b/dotnet/src/JacquardHost/ProtocolException.cs new file mode 100644 index 0000000..a64a0a8 --- /dev/null +++ b/dotnet/src/JacquardHost/ProtocolException.cs @@ -0,0 +1,33 @@ +using System; + +namespace JacquardLang.Host; + +/// A protocol refusal carrying its frozen v0 diagnostic code, such as E1601. +public sealed class ProtocolException : Exception +{ + /// A refusal with a frozen code and a message without the code. + public ProtocolException(string code, string detail) + : base(code + ": " + detail) + { + Code = code; + Detail = detail; + } + + /// The frozen diagnostic code, such as "E1601". + public string Code { get; } + + /// The message without the code. + public string Detail { get; } + + /// E1601: malformed input. + public static ProtocolException Malformed(string detail) => new("E1601", detail); + + /// E1602: a selected or hard limit is exceeded. + public static ProtocolException Exceeded(string detail) => new("E1602", detail); + + /// E1604: a value or type with no v0 encoding. + public static ProtocolException Unsupported(string detail) => new("E1604", detail); + + /// E1608: a message out of order for the session state. + public static ProtocolException Order(string detail) => new("E1608", detail); +} diff --git a/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/README.md b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/README.md new file mode 100644 index 0000000..789bd02 --- /dev/null +++ b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/README.md @@ -0,0 +1,96 @@ +# Host protocol v0 conformance kit + +This directory turns the frozen HB.1 schema/state vectors in +[`../vectors.json`](../vectors.json) into executable fixtures for the +`jacquard-host-v0` protocol over the `stdio-u32-json-v0` carrier. An adapter +in any language pins this kit, drives the installed `jacquard host worker`, +and must observe exactly the Core frames recorded here. + +## Files + +- `fixtures.jac`: first-party fixture declarations. Install them once with + the published recipe, `jacquard run fixtures.jac --store DIR`, using the + prelude shipped with the same Core release. +- `kit.json`: the manifest. It records the protocol and carrier names, the + Core version that produced the kit, the recipe, the exact identities of + every fixture member, the binding from each synthetic 64-digit vector + identity to a kit member, the hard limits, the fields an adapter owns, and + every pending decision. +- `transcripts.json`: one entry per positive and hostile vector. Each entry + lists the host frames in order (or the raw bytes for wire cases), the Core + frames the worker emitted, the exit status, the observed primary code and + terminal classification, the host's local refusal for the terminal-phase + case, and a status of `conforms` or `diverges`. + +## Binding + +The vectors use synthetic identities so that they can be frozen without a +store. The kit binds them to real content-addressed members: + +| Vector identity | Kit member | +|---|---| +| `aaaa…` | `kit-ready : () ->{} ()` | +| `dddd…` | `kit-echo : (Text) ->{KitWorld} Text` | +| `bbbb…` | the `KitWorld` effect | +| `cccc…` | its `send : (Text) -> Text` operation | +| `1111…` | the `Text` type | + +Every other synthetic identity in a hostile mutation (for example the extra +capability `eeee…` or the uppercase target) is left as written, because the +case exists to prove that Core rejects it. + +## How an adapter uses the kit + +1. Pin `protocol`, `carrier`, `core_version`, and the SHA-256 of `kit.json` + and `transcripts.json` in the adapter's compatibility record. +2. Build the store with the recipe and check that the identities in + `kit.json` resolve, so the adapter is talking to the same fixtures. +3. For each transcript, start one worker and play the recorded steps in + order: read one Core frame for each `core_to_host` step and write one host + frame (or the raw bytes) for each `host_to_core` step. The two negotiation + frames, `host_select` and `invoke`, are consecutive host steps. Never write + after a terminal frame has been read: a recorded host step that follows + the terminal is counted under `host_inputs_skipped_after_terminal`, and the + `after_terminal` step of the terminal-phase case must be refused locally + as E1608. Compare the Core frames, exit status, and observed summary with + the transcript. +4. Compare semantic JSON: key order and insignificant whitespace do not + matter. Diagnostic prose (`summary`, `cause`, `next_step`, `contrast`) is + Core-authored human text and is not normative; the schema, domain, code, + severity, and span are, and an `effect_failure` message must end the first + diagnostic's cause exactly. +5. Treat any divergence as a protocol or implementation finding. An adapter + must never normalise its own frames to make a transcript pass. + +Fields an adapter owns and may add outside the Core frames are named under +`host_owned` in `kit.json`; nothing inside a Core frame is host-owned. + +## The fake host + +`test/host_kit.ml` contains the deterministic fake host used to produce and +replay these transcripts. It spawns the installed worker over pipes, writes +host frames only when it is the host's turn, refuses to send after it has +observed a terminal frame (recording the local E1608 the protocol requires), +closes its input after raw wire cases, and records every Core frame, the exit +status, and the bounded operator text. It is evidence tooling, not an adapter +or embedding contract. + +## Regeneration and drift + +`dune exec test/gen_host_kit.exe` from the repository root rebuilds the store +with the recipe, replays every vector, and rewrites `kit.json` and +`transcripts.json`. `test/test_host_kit.ml` fails when a regenerated +transcript differs from the checked-in one, when a positive sequence stops +matching its bound HB.1 template, when a hostile case stops producing its +expected primary code, or when a divergence appears that is not listed under +`pending_decisions`. + +Changing a vector's expected code or shape remains a reviewed protocol +decision, never a mechanical refresh. Two kinds of drift are recorded rather +than resolved here: + +- `pending_decisions` lists `noncanonical-target-hash`, whose vector expects + E1603 while the shipped preflight classifies an uppercase hash as a + malformed scalar (E1601) under the frozen fail-fast order. +- `diagnostic_prose.drift` lists the outcome diagnostics whose vector prose + differs from the prose the shipped session layer emits for the same code. diff --git a/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/fixtures.jac b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/fixtures.jac new file mode 100644 index 0000000..bea80b0 --- /dev/null +++ b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/fixtures.jac @@ -0,0 +1,21 @@ +-- Conformance kit fixtures for jacquard-host-v0. +-- Install once with `jacquard run fixtures.jac --store DIR`; kit.json records +-- the resulting identities and binds them to the synthetic HB.1 vectors. +once effect KitWorld where { + send : (Text) -> Text +} + +kit-ready : () ->{} () +kit-ready() = () + +kit-echo : (Text) ->{KitWorld} Text +kit-echo(body) = send(body) + +kit-twice : (Text) ->{KitWorld} Text +kit-twice(body) = send(send(body)) + +kit-double : (Int) ->{} Int +kit-double(n) = mul(n, 2) + +kit-fail : (Int) ->{} Int +kit-fail(n) = div(n, 0) diff --git a/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/kit.json b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/kit.json new file mode 100644 index 0000000..6927d67 --- /dev/null +++ b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/kit.json @@ -0,0 +1,106 @@ +{ + "bindings": { + "1111111111111111111111111111111111111111111111111111111111111111": { + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "member": "text" + }, + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": { + "identity": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "member": "kit-ready" + }, + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb": { + "identity": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "member": "kit-world" + }, + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc": { + "identity": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "member": "send" + }, + "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd": { + "identity": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "member": "kit-echo" + } + }, + "carrier": "stdio-u32-json-v0", + "core_version": "0.3.0", + "diagnostic_prose": { + "cause_suffix_rule": "an effect_failure message must end the first diagnostic's cause", + "compared": [ "schema", "domain", "code", "severity", "span" ], + "drift": [ + { + "path": "refused_outcome/result/diagnostics/0/summary", + "transcript": "refused-authority" + }, + { + "path": "refused_outcome/result/diagnostics/0/cause", + "transcript": "refused-authority" + }, + { + "path": "refused_outcome/result/diagnostics/0/next_step", + "transcript": "refused-authority" + }, + { + "path": "timeout_unknown_outcome/result/diagnostics/0/summary", + "transcript": "timeout-with-unknown-completion" + }, + { + "path": "timeout_unknown_outcome/result/diagnostics/0/cause", + "transcript": "timeout-with-unknown-completion" + }, + { + "path": "timeout_unknown_outcome/result/diagnostics/0/next_step", + "transcript": "timeout-with-unknown-completion" + } + ], + "normative": false + }, + "hard_limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "host_owned": [ + "operator text on the worker's standard error", + "the exact moment a carrier loss is observed", + "any run identifier, timestamp, peer identity, or receipt an adapter records outside the Core frames" + ], + "identities": { + "int": "907085f5670ab5835e5356feb10ae729496e3816b863ddbb21cfe289e7d34f0d", + "kit-double": "1e5b4939f0d7b13b0502fbb04d75659a7e01ee4927ad8a5f1c64637c90357c8f", + "kit-echo": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kit-fail": "d337c5d51261e63407923b4aaaf6c411bee7185711358523e9c77626d9152a36", + "kit-ready": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kit-twice": "2451609ed1af76f59a8ec47a9f3bd792e1e888e0ffaa6518e3f097a66339cd23", + "kit-world": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "send": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "text": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3" + }, + "pending_decisions": [ + { + "expected": "E1603", + "name": "noncanonical-target-hash", + "observed": "E1601" + } + ], + "protocol": "jacquard-host-v0", + "recipe": { + "command": "jacquard run fixtures.jac --store DIR", + "fixtures": "fixtures.jac", + "prelude": "the prelude shipped with the same Core release", + "worker": "jacquard host worker --store DIR" + }, + "schema": "jacquard-host-kit-v0", + "transcript_count": 21, + "transcripts": "transcripts.json", + "vectors": "../vectors.json" +} diff --git a/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/transcripts.json b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/transcripts.json new file mode 100644 index 0000000..a3ed58e --- /dev/null +++ b/dotnet/src/JacquardHost/kit/host-protocol-v0/kit/transcripts.json @@ -0,0 +1,3066 @@ +[ + { + "name": "pure-success", + "kind": "positive", + "base": null, + "phase": null, + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { "terminal": "ok" }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "evidence": { + "core": { + "capabilities": { "effects": [], "operations": [] }, + "effect_requests": [], + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + }, + "terminal": "ok" + }, + "host_observations": { + "responses": [], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { "kind": "ok", "value": { "items": [], "kind": "tuple" } } + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "outcome", + "primary_code": null, + "terminal": "ok", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "one-once-effect-success", + "kind": "positive", + "base": null, + "phase": null, + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { "terminal": "ok" }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "arguments": [ { "kind": "text", "value": "ping" } ], + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "invocation_id": "0000000000000000", + "kind": "effect_request", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + }, + { + "evidence": { + "core": { + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "effect_requests": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "ordinal": 1 + } + ], + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + }, + "terminal": "ok" + }, + "host_observations": { + "responses": [ + { "category": "ok", "completion": "completed", "ordinal": 1 } + ], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "kind": "ok", + "value": { "kind": "text", "value": "pong" } + } + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "outcome", + "primary_code": null, + "terminal": "ok", + "effect_requests": 1, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "refused-authority", + "kind": "positive", + "base": null, + "phase": null, + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_refused", "refused_outcome" + ], + "expect": { "terminal": "host_failure" }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_refused", + "frame": { + "category": "refused_authority", + "completion": "not_started", + "invocation_id": "0000000000000000", + "kind": "effect_failure", + "message": "policy denied", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "refused_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "arguments": [ { "kind": "text", "value": "ping" } ], + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "invocation_id": "0000000000000000", + "kind": "effect_request", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + }, + { + "evidence": { + "core": { + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "effect_requests": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "ordinal": 1 + } + ], + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + }, + "terminal": "host_failure" + }, + "host_observations": { + "responses": [ + { + "category": "refused_authority", + "completion": "not_started", + "ordinal": 1 + } + ], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "diagnostics": [ + { + "cause": "The host reported an outside failure. Host detail: policy denied", + "code": "E1607", + "domain": "process", + "next_step": "Review the host's authority policy before starting a new invocation.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host refused authority before starting the outside action." + } + ], + "kind": "error" + } + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "outcome", + "primary_code": "E1607", + "terminal": "host_failure", + "effect_requests": 1, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [ + "refused_outcome/result/diagnostics/0/summary", + "refused_outcome/result/diagnostics/0/cause", + "refused_outcome/result/diagnostics/0/next_step" + ], + "status": "conforms" + }, + { + "name": "timeout-with-unknown-completion", + "kind": "positive", + "base": null, + "phase": null, + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "timeout_unknown_cancel", "timeout_unknown_outcome" + ], + "expect": { "terminal": "completion_unknown" }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "timeout_unknown_cancel", + "frame": { + "completion": "unknown", + "invocation_id": "0000000000000000", + "kind": "cancel", + "protocol": "jacquard-host-v0", + "reason": "timeout", + "request_id": "0000000000000001" + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "timeout_unknown_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "arguments": [ { "kind": "text", "value": "ping" } ], + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "invocation_id": "0000000000000000", + "kind": "effect_request", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + }, + { + "evidence": { + "core": { + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "effect_requests": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "ordinal": 1 + } + ], + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + }, + "terminal": "completion_unknown" + }, + "host_observations": { + "responses": [ + { + "category": "timeout", + "completion": "unknown", + "ordinal": 1 + } + ], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "diagnostics": [ + { + "cause": "The host ended the response slot without a successful value.", + "code": "E1612", + "domain": "process", + "next_step": "Reconcile host-owned evidence; do not automatically retry the action or invocation.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The outside action's completion is unknown." + } + ], + "kind": "error" + } + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "outcome", + "primary_code": "E1612", + "terminal": "completion_unknown", + "effect_requests": 1, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [ + "timeout_unknown_outcome/result/diagnostics/0/summary", + "timeout_unknown_outcome/result/diagnostics/0/cause", + "timeout_unknown_outcome/result/diagnostics/0/next_step" + ], + "status": "conforms" + }, + { + "name": "selected-shutdown", + "kind": "positive", + "base": null, + "phase": null, + "sequence": [ "core_hello", "host_select", "shutdown", "shutdown_ack" ], + "expect": { "terminal": "shutdown" }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "shutdown", + "frame": { "kind": "shutdown", "protocol": "jacquard-host-v0" } + }, + { "step": 3, "direction": "core_to_host", "template": "shutdown_ack" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { "kind": "shutdown_ack", "protocol": "jacquard-host-v0" } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "shutdown_ack", + "primary_code": null, + "terminal": "shutdown", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "unknown-version", + "kind": "hostile", + "base": "pure-success", + "phase": "host_select", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1600", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v1" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The selected protocol version is not advertised.", + "code": "E1600", + "domain": "process", + "next_step": "Select jacquard-host-v0 with the advertised stdio-u32-json-v0 carrier.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host selected an unsupported protocol version." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1600", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "unknown-invoke-field", + "kind": "hostile", + "base": "pure-success", + "phase": "invoke", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1601", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "display_name": "mutable-name", + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The protocol envelope has a missing or unknown field.", + "code": "E1601", + "domain": "process", + "next_step": "Send one exact UTF-8 JSON object using the frozen jacquard-host-v0 envelope.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host protocol frame is malformed." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1601", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "duplicate-json-key", + "kind": "hostile", + "base": "pure-success", + "phase": "framing", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1601", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "raw_hex": "000000267b226b696e64223a22686f73745f73656c656374222c226b696e64223a22696e766f6b65227d" + }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "A JSON object contains a duplicate field name.", + "code": "E1601", + "domain": "process", + "next_step": "Send one exact UTF-8 JSON object using the frozen jacquard-host-v0 envelope.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host protocol frame is malformed." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1601", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "invalid-utf8", + "kind": "hostile", + "base": "pure-success", + "phase": "framing", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1601", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { "step": 1, "direction": "host_to_core", "raw_hex": "000000037bff7d" }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The frame payload is not valid Unicode scalar UTF-8.", + "code": "E1601", + "domain": "process", + "next_step": "Send one exact UTF-8 JSON object using the frozen jacquard-host-v0 envelope.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host protocol frame is malformed." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1601", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "oversized-frame-prefix", + "kind": "hostile", + "base": "pure-success", + "phase": "framing", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1602", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { "step": 1, "direction": "host_to_core", "raw_hex": "00100001" }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The frame payload length exceeds max_frame_bytes.", + "code": "E1602", + "domain": "process", + "next_step": "Choose positive advertised limits and keep the frame within the selected ceilings.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host protocol limit was exceeded or cannot represent a mandatory result." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1602", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "nonpositive-selected-limit", + "kind": "hostile", + "base": "pure-success", + "phase": "host_select", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1602", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 0, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The selected max_frame_bytes is not positive and advertised.", + "code": "E1602", + "domain": "process", + "next_step": "Choose positive advertised limits and keep the frame within the selected ceilings.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host protocol limit was exceeded or cannot represent a mandatory result." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1602", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "noncanonical-target-hash", + "kind": "hostile", + "base": "pure-success", + "phase": "invoke", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1603", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The target callable identity must be exactly 64 lowercase hexadecimal digits.", + "code": "E1601", + "domain": "process", + "next_step": "Send one exact UTF-8 JSON object using the frozen jacquard-host-v0 envelope.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host protocol frame is malformed." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1601", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "diverges" + }, + { + "name": "interface-mismatch", + "kind": "hostile", + "base": "one-once-effect-success", + "phase": "invoke", + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { + "code": "E1603", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ { "items": [], "kind": "tuple" } ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The pinned interface disagrees with the checked target arrow.", + "code": "E1603", + "domain": "process", + "next_step": "Select one complete checked store-term closure and use its exact first-order interface.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The selected Jacquard target or pinned interface is invalid." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1603", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 1 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "unsupported-secret-value", + "kind": "hostile", + "base": "one-once-effect-success", + "phase": "invoke", + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { + "code": "E1604", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "secret", "value": "must-not-cross" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "Boundary value kind \"secret\" is not supported in jacquard-host-v0.", + "code": "E1604", + "domain": "process", + "next_step": "Use only the frozen closed first-order type and lossless value descriptor subset.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "A Jacquard type or value is unsupported at the v0 host boundary." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1604", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 1 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "extra-capability", + "kind": "hostile", + "base": "one-once-effect-success", + "phase": "invoke", + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { + "code": "E1605", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "Capability effects do not exactly equal the pinned interface row.", + "code": "E1605", + "domain": "process", + "next_step": "Use the exact checked effect set and a sorted unique registry of its public once operations.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The selected host capability or operation registry is invalid." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1605", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 1 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "multi-operation-registry-entry", + "kind": "hostile", + "base": "one-once-effect-success", + "phase": "invoke", + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { + "code": "E1605", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "multi", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "A registry entry does not select mode once.", + "code": "E1605", + "domain": "process", + "next_step": "Use the exact checked effect set and a sorted unique registry of its public once operations.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The selected host capability or operation registry is invalid." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1605", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 1 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "unconfigured-root-operation", + "kind": "hostile", + "base": "one-once-effect-success", + "phase": "evaluation", + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { + "code": "E1606", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "evidence": { + "core": { + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [] + }, + "effect_requests": [], + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + }, + "terminal": "diagnostic" + }, + "host_observations": { + "responses": [], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "diagnostics": [ + { + "cause": "The root operation is absent from the configured registry.", + "code": "E1606", + "domain": "process", + "next_step": "Configure the exact operation before starting a new invocation.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The root operation has no configured host implementation." + } + ], + "kind": "error" + } + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "outcome", + "primary_code": "E1606", + "terminal": "diagnostic", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 1 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "wrong-response-id", + "kind": "hostile", + "base": "one-once-effect-success", + "phase": "effect_response", + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { + "code": "E1608", + "effect_requests_before_failure": 1, + "forbid_outside_action": false, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000002", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "arguments": [ { "kind": "text", "value": "ping" } ], + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "invocation_id": "0000000000000000", + "kind": "effect_request", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + }, + { + "evidence": { + "core": { + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "effect_requests": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "ordinal": 1 + } + ], + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + }, + "terminal": "diagnostic" + }, + "host_observations": { + "responses": [], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "diagnostics": [ + { + "cause": "The host response is malformed or disagrees with its checked result type.", + "code": "E1608", + "domain": "process", + "next_step": "Send only the next message permitted by the serial jacquard-host-v0 state machine.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host protocol message is invalid in the current state." + } + ], + "kind": "error" + } + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "outcome", + "primary_code": "E1608", + "terminal": "diagnostic", + "effect_requests": 1, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "message-after-outcome", + "kind": "hostile", + "base": "one-once-effect-success", + "phase": "terminal", + "sequence": [ + "core_hello", "host_select", "effect_invoke", "effect_request", + "effect_ok", "effect_outcome" + ], + "expect": { + "code": "E1608", + "effect_requests_before_failure": 1, + "forbid_outside_action": false, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { + "step": 1, + "direction": "host_to_core", + "template": "host_select", + "frame": { + "kind": "host_select", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "protocol": "jacquard-host-v0" + } + }, + { + "step": 2, + "direction": "host_to_core", + "template": "effect_invoke", + "frame": { + "arguments": [ { "kind": "text", "value": "ping" } ], + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + } + } + }, + { + "step": 3, + "direction": "core_to_host", + "template": "effect_request" + }, + { + "step": 4, + "direction": "host_to_core", + "template": "effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "pong" } + } + }, + { + "step": 5, + "direction": "core_to_host", + "template": "effect_outcome" + }, + { + "step": 6, + "direction": "host_to_core", + "template": "late_effect_ok", + "frame": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": { "kind": "text", "value": "late" } + }, + "after_terminal": true + } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "arguments": [ { "kind": "text", "value": "ping" } ], + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "invocation_id": "0000000000000000", + "kind": "effect_request", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + }, + { + "evidence": { + "core": { + "capabilities": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "operations": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "mode": "once", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722" + } + ] + }, + "effect_requests": [ + { + "effect": "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0", + "operation": "3120a126f44773feba44f437ef0d2514e5e55764e1c0667f6434561f430c5722", + "ordinal": 1 + } + ], + "interface": { + "effects": [ + "44b60d80136d8f1366f32889d56ced087381a1ef31646d7ff173d5c36bbe4db0" + ], + "parameters": [ + { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "ca456459bd7fbe8c9694388e1246c4333221817248d24707f05b7423828516e3", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "91d93eb6a5a3390035a17eb4d4ce24cf8e18369e458d0382f17328fd9df303d2", + "kind": "store-term-v0" + }, + "terminal": "ok" + }, + "host_observations": { + "responses": [ + { "category": "ok", "completion": "completed", "ordinal": 1 } + ], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "kind": "ok", + "value": { "kind": "text", "value": "pong" } + } + } + ], + "exit_status": 0, + "observed": { + "terminal_kind": "outcome", + "primary_code": null, + "terminal": "ok", + "effect_requests": 1, + "terminal_frames": 1, + "host_local_refusal": "E1608", + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "truncated-length-prefix", + "kind": "hostile", + "base": "pure-success", + "phase": "framing", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1611", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { "step": 1, "direction": "host_to_core", "raw_hex": "0000" }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The carrier ended before the declared frame completed.", + "code": "E1611", + "domain": "process", + "next_step": "Treat the missing terminal exchange as host-owned carrier-failure evidence.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host carrier was lost before a trustworthy frame completed." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 74, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1611", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + }, + { + "name": "truncated-frame-payload", + "kind": "hostile", + "base": "pure-success", + "phase": "framing", + "sequence": [ + "core_hello", "host_select", "pure_invoke", "pure_outcome" + ], + "expect": { + "code": "E1611", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "host_input": [ + { "step": 0, "direction": "core_to_host", "template": "core_hello" }, + { "step": 1, "direction": "host_to_core", "raw_hex": "000000057b7d" }, + { + "step": 2, + "direction": "host_to_core", + "template": "pure_invoke", + "frame": { + "arguments": [], + "capabilities": { "effects": [], "operations": [] }, + "interface": { + "effects": [], + "parameters": [], + "result": { "items": [], "kind": "tuple" } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "2af2f2e66770cf0597374a97b4c3988dc0c3f9b42d87f6f430be53525ce6a7ec", + "kind": "store-term-v0" + } + } + }, + { "step": 3, "direction": "core_to_host", "template": "pure_outcome" } + ], + "core_output": [ + { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "versions": [ "jacquard-host-v0" ] + }, + { + "diagnostics": [ + { + "cause": "The carrier ended before the declared frame completed.", + "code": "E1611", + "domain": "process", + "next_step": "Treat the missing terminal exchange as host-owned carrier-failure evidence.", + "schema": "jacquard-diagnostic-v1", + "severity": "error", + "span": null, + "summary": "The host carrier was lost before a trustworthy frame completed." + } + ], + "kind": "fatal", + "protocol": "jacquard-host-v0" + } + ], + "exit_status": 74, + "observed": { + "terminal_kind": "fatal", + "primary_code": "E1611", + "terminal": "fatal", + "effect_requests": 0, + "terminal_frames": 1, + "host_local_refusal": null, + "host_inputs_skipped_after_terminal": 0 + }, + "prose_drift": [], + "status": "conforms" + } +] diff --git a/dotnet/src/JacquardHost/kit/host-protocol-v0/vectors.json b/dotnet/src/JacquardHost/kit/host-protocol-v0/vectors.json new file mode 100644 index 0000000..78591d3 --- /dev/null +++ b/dotnet/src/JacquardHost/kit/host-protocol-v0/vectors.json @@ -0,0 +1,775 @@ +{ + "schema": "jacquard-host-vectors-v0", + "hard_limits": { + "max_arguments": 64, + "max_collection_items": 1024, + "max_diagnostic_bytes": 65536, + "max_diagnostics": 32, + "max_effect_requests": 1024, + "max_effects": 64, + "max_frame_bytes": 1048576, + "max_host_message_bytes": 4096, + "max_json_depth": 64, + "max_operations": 256, + "max_stderr_bytes": 65536, + "max_text_bytes": 262144, + "max_value_nodes": 4096 + }, + "templates": { + "core_hello": { + "direction": "core_to_host", + "message": { + "carrier": "stdio-u32-json-v0", + "kind": "core_hello", + "limits": {"$ref": "hard_limits"}, + "versions": ["jacquard-host-v0"] + } + }, + "host_select": { + "direction": "host_to_core", + "message": { + "kind": "host_select", + "limits": {"$ref": "hard_limits"}, + "protocol": "jacquard-host-v0" + } + }, + "pure_invoke": { + "direction": "host_to_core", + "message": { + "arguments": [], + "capabilities": {"effects": [], "operations": []}, + "interface": { + "effects": [], + "parameters": [], + "result": {"items": [], "kind": "tuple"} + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": "store-term-v0" + } + } + }, + "pure_outcome": { + "direction": "core_to_host", + "message": { + "evidence": { + "core": { + "capabilities": {"effects": [], "operations": []}, + "effect_requests": [], + "interface": { + "effects": [], + "parameters": [], + "result": {"items": [], "kind": "tuple"} + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": "store-term-v0" + }, + "terminal": "ok" + }, + "host_observations": { + "responses": [], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": {"kind": "ok", "value": {"items": [], "kind": "tuple"}} + } + }, + "effect_invoke": { + "direction": "host_to_core", + "message": { + "arguments": [{"kind": "text", "value": "ping"}], + "capabilities": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "operations": [ + { + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "mode": "once", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc" + } + ] + }, + "interface": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "parameters": [ + { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "kind": "invoke", + "protocol": "jacquard-host-v0", + "target": { + "callable": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "kind": "store-term-v0" + } + } + }, + "effect_request": { + "direction": "core_to_host", + "message": { + "arguments": [{"kind": "text", "value": "ping"}], + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "invocation_id": "0000000000000000", + "kind": "effect_request", + "mode": "once", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + } + }, + "effect_ok": { + "direction": "host_to_core", + "message": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": {"kind": "text", "value": "pong"} + } + }, + "effect_outcome": { + "direction": "core_to_host", + "message": { + "evidence": { + "core": { + "capabilities": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "operations": [ + { + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "mode": "once", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc" + } + ] + }, + "effect_requests": [ + { + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "ordinal": 1 + } + ], + "interface": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "parameters": [ + { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "kind": "store-term-v0" + }, + "terminal": "ok" + }, + "host_observations": { + "responses": [{"category": "ok", "completion": "completed", "ordinal": 1}], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": {"kind": "ok", "value": {"kind": "text", "value": "pong"}} + } + }, + "effect_refused": { + "direction": "host_to_core", + "message": { + "category": "refused_authority", + "completion": "not_started", + "invocation_id": "0000000000000000", + "kind": "effect_failure", + "message": "policy denied", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001" + } + }, + "refused_outcome": { + "direction": "core_to_host", + "message": { + "evidence": { + "core": { + "capabilities": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "operations": [ + { + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "mode": "once", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc" + } + ] + }, + "effect_requests": [ + { + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "ordinal": 1 + } + ], + "interface": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "parameters": [ + { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "kind": "store-term-v0" + }, + "terminal": "host_failure" + }, + "host_observations": { + "responses": [ + {"category": "refused_authority", "completion": "not_started", "ordinal": 1} + ], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "diagnostics": [ + { + "schema": "jacquard-diagnostic-v1", + "domain": "process", + "code": "E1607", + "severity": "error", + "span": null, + "summary": "The host refused the requested authority.", + "cause": "The host refused operation cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc before starting an outside action: policy denied.", + "next_step": "Grant the required host authority or choose an invocation that does not request this operation." + } + ], + "kind": "error" + } + } + }, + "timeout_unknown_cancel": { + "direction": "host_to_core", + "message": { + "completion": "unknown", + "invocation_id": "0000000000000000", + "kind": "cancel", + "protocol": "jacquard-host-v0", + "reason": "timeout", + "request_id": "0000000000000001" + } + }, + "timeout_unknown_outcome": { + "direction": "core_to_host", + "message": { + "evidence": { + "core": { + "capabilities": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "operations": [ + { + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "mode": "once", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc" + } + ] + }, + "effect_requests": [ + { + "effect": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "operation": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "ordinal": 1 + } + ], + "interface": { + "effects": ["bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "parameters": [ + { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + ], + "result": { + "arguments": [], + "identity": "1111111111111111111111111111111111111111111111111111111111111111", + "kind": "nominal" + } + }, + "invocation_id": "0000000000000000", + "schema": "jacquard-host-core-evidence-v0", + "target": { + "callable": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "kind": "store-term-v0" + }, + "terminal": "completion_unknown" + }, + "host_observations": { + "responses": [{"category": "timeout", "completion": "unknown", "ordinal": 1}], + "schema": "jacquard-host-observations-v0" + } + }, + "invocation_id": "0000000000000000", + "kind": "outcome", + "protocol": "jacquard-host-v0", + "result": { + "diagnostics": [ + { + "schema": "jacquard-diagnostic-v1", + "domain": "process", + "code": "E1612", + "severity": "error", + "span": null, + "summary": "The outside action may have completed.", + "cause": "The host reported that operation cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc may have completed before its timeout cancellation was observed.", + "next_step": "Do not retry automatically; reconcile the outside system before deciding what to do next." + } + ], + "kind": "error" + } + } + }, + "shutdown": { + "direction": "host_to_core", + "message": {"kind": "shutdown", "protocol": "jacquard-host-v0"} + }, + "shutdown_ack": { + "direction": "core_to_host", + "message": {"kind": "shutdown_ack", "protocol": "jacquard-host-v0"} + }, + "late_effect_ok": { + "direction": "host_to_core", + "message": { + "invocation_id": "0000000000000000", + "kind": "effect_ok", + "protocol": "jacquard-host-v0", + "request_id": "0000000000000001", + "value": {"kind": "text", "value": "late"} + } + } + }, + "positive": [ + { + "name": "pure-success", + "sequence": ["core_hello", "host_select", "pure_invoke", "pure_outcome"], + "terminal": "ok" + }, + { + "name": "one-once-effect-success", + "sequence": [ + "core_hello", + "host_select", + "effect_invoke", + "effect_request", + "effect_ok", + "effect_outcome" + ], + "terminal": "ok" + }, + { + "name": "refused-authority", + "sequence": [ + "core_hello", + "host_select", + "effect_invoke", + "effect_request", + "effect_refused", + "refused_outcome" + ], + "terminal": "host_failure" + }, + { + "name": "timeout-with-unknown-completion", + "sequence": [ + "core_hello", + "host_select", + "effect_invoke", + "effect_request", + "timeout_unknown_cancel", + "timeout_unknown_outcome" + ], + "terminal": "completion_unknown" + }, + { + "name": "selected-shutdown", + "sequence": ["core_hello", "host_select", "shutdown", "shutdown_ack"], + "terminal": "shutdown" + } + ], + "terminal_mappings": [ + { + "category": "unsupported_operation", + "completion": "not_started", + "kind": "effect_failure", + "primary_code": "E1606", + "terminal": "host_failure" + }, + { + "category": "refused_authority", + "completion": "not_started", + "kind": "effect_failure", + "primary_code": "E1607", + "terminal": "host_failure" + }, + { + "category": "outside_failure", + "completion": "failed", + "kind": "effect_failure", + "primary_code": "E1613", + "terminal": "host_failure" + }, + { + "category": "completion_unknown", + "completion": "unknown", + "kind": "effect_failure", + "primary_code": "E1612", + "terminal": "completion_unknown" + }, + { + "completion": "not_started", + "kind": "cancel", + "primary_code": "E1614", + "reason": "cancelled", + "terminal": "cancelled" + }, + { + "completion": "failed", + "kind": "cancel", + "primary_code": "E1614", + "reason": "cancelled", + "terminal": "cancelled" + }, + { + "completion": "unknown", + "kind": "cancel", + "primary_code": "E1612", + "reason": "cancelled", + "terminal": "completion_unknown" + }, + { + "completion": "not_started", + "kind": "cancel", + "primary_code": "E1609", + "reason": "timeout", + "terminal": "cancelled" + }, + { + "completion": "failed", + "kind": "cancel", + "primary_code": "E1609", + "reason": "timeout", + "terminal": "cancelled" + }, + { + "completion": "unknown", + "kind": "cancel", + "primary_code": "E1612", + "reason": "timeout", + "terminal": "completion_unknown" + }, + { + "completion": "not_started", + "kind": "cancel", + "primary_code": "E1610", + "reason": "host_shutdown", + "terminal": "cancelled" + }, + { + "completion": "failed", + "kind": "cancel", + "primary_code": "E1610", + "reason": "host_shutdown", + "terminal": "cancelled" + }, + { + "completion": "unknown", + "kind": "cancel", + "primary_code": "E1612", + "reason": "host_shutdown", + "terminal": "completion_unknown" + } + ], + "hostile": [ + { + "base": "pure-success", + "expect": { + "code": "E1600", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 1, + "op": "replace", + "path": "/protocol", + "value": "jacquard-host-v1" + }, + "name": "unknown-version", + "phase": "host_select" + }, + { + "base": "pure-success", + "expect": { + "code": "E1601", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 2, + "op": "add", + "path": "/display_name", + "value": "mutable-name" + }, + "name": "unknown-invoke-field", + "phase": "invoke" + }, + { + "base": "pure-success", + "expect": { + "code": "E1601", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 1, + "op": "raw_wire", + "wire_hex": "000000267b226b696e64223a22686f73745f73656c656374222c226b696e64223a22696e766f6b65227d" + }, + "name": "duplicate-json-key", + "phase": "framing" + }, + { + "base": "pure-success", + "expect": { + "code": "E1601", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": {"frame": 1, "op": "raw_wire", "wire_hex": "000000037bff7d"}, + "name": "invalid-utf8", + "phase": "framing" + }, + { + "base": "pure-success", + "expect": { + "code": "E1602", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": {"frame": 1, "op": "raw_wire", "wire_hex": "00100001"}, + "name": "oversized-frame-prefix", + "phase": "framing" + }, + { + "base": "pure-success", + "expect": { + "code": "E1602", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 1, + "op": "replace", + "path": "/limits/max_frame_bytes", + "value": 0 + }, + "name": "nonpositive-selected-limit", + "phase": "host_select" + }, + { + "base": "pure-success", + "expect": { + "code": "E1603", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 2, + "op": "replace", + "path": "/target/callable", + "value": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" + }, + "name": "noncanonical-target-hash", + "phase": "invoke" + }, + { + "base": "one-once-effect-success", + "expect": { + "code": "E1603", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 2, + "op": "replace", + "path": "/interface/parameters/0", + "value": {"items": [], "kind": "tuple"} + }, + "name": "interface-mismatch", + "phase": "invoke" + }, + { + "base": "one-once-effect-success", + "expect": { + "code": "E1604", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 2, + "op": "replace", + "path": "/arguments/0", + "value": {"kind": "secret", "value": "must-not-cross"} + }, + "name": "unsupported-secret-value", + "phase": "invoke" + }, + { + "base": "one-once-effect-success", + "expect": { + "code": "E1605", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 2, + "op": "add", + "path": "/capabilities/effects/1", + "value": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee" + }, + "name": "extra-capability", + "phase": "invoke" + }, + { + "base": "one-once-effect-success", + "expect": { + "code": "E1605", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 2, + "op": "replace", + "path": "/capabilities/operations/0/mode", + "value": "multi" + }, + "name": "multi-operation-registry-entry", + "phase": "invoke" + }, + { + "base": "one-once-effect-success", + "expect": { + "code": "E1606", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 2, + "op": "remove", + "path": "/capabilities/operations/0" + }, + "name": "unconfigured-root-operation", + "phase": "evaluation" + }, + { + "base": "one-once-effect-success", + "expect": { + "code": "E1608", + "effect_requests_before_failure": 1, + "forbid_outside_action": false, + "terminal_frames_max": 1 + }, + "mutation": { + "frame": 4, + "op": "replace", + "path": "/request_id", + "value": "0000000000000002" + }, + "name": "wrong-response-id", + "phase": "effect_response" + }, + { + "base": "one-once-effect-success", + "expect": { + "code": "E1608", + "effect_requests_before_failure": 1, + "forbid_outside_action": false, + "terminal_frames_max": 1 + }, + "mutation": {"op": "append", "template": "late_effect_ok"}, + "name": "message-after-outcome", + "phase": "terminal" + }, + { + "base": "pure-success", + "expect": { + "code": "E1611", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": {"frame": 1, "op": "raw_wire", "wire_hex": "0000"}, + "name": "truncated-length-prefix", + "phase": "framing" + }, + { + "base": "pure-success", + "expect": { + "code": "E1611", + "effect_requests_before_failure": 0, + "forbid_outside_action": true, + "terminal_frames_max": 1 + }, + "mutation": {"frame": 1, "op": "raw_wire", "wire_hex": "000000057b7d"}, + "name": "truncated-frame-payload", + "phase": "framing" + } + ] +} diff --git a/go/NOTICE b/go/NOTICE index 09bc7d2..8de2b4f 100644 --- a/go/NOTICE +++ b/go/NOTICE @@ -5,9 +5,10 @@ This product includes conformance artifacts from Jacquard Core, https://github.com/jbwinters/jacquard-lang, licensed under Apache License 2.0. The artifacts under src/jacquard_host/data/conformance/host-protocol-v0/, and their byte-identical copies under go/kit/data/host-protocol-v0/, -rust/kit/host-protocol-v0/, and -java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, are -copied verbatim from the commits recorded in docs/COMPATIBILITY.md. +rust/kit/host-protocol-v0/, +java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, and +dotnet/src/JacquardHost/kit/host-protocol-v0/, are copied verbatim from the +commits recorded in docs/COMPATIBILITY.md. Upstream notice: Jacquard diff --git a/java/NOTICE b/java/NOTICE index 09bc7d2..8de2b4f 100644 --- a/java/NOTICE +++ b/java/NOTICE @@ -5,9 +5,10 @@ This product includes conformance artifacts from Jacquard Core, https://github.com/jbwinters/jacquard-lang, licensed under Apache License 2.0. The artifacts under src/jacquard_host/data/conformance/host-protocol-v0/, and their byte-identical copies under go/kit/data/host-protocol-v0/, -rust/kit/host-protocol-v0/, and -java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, are -copied verbatim from the commits recorded in docs/COMPATIBILITY.md. +rust/kit/host-protocol-v0/, +java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, and +dotnet/src/JacquardHost/kit/host-protocol-v0/, are copied verbatim from the +commits recorded in docs/COMPATIBILITY.md. Upstream notice: Jacquard diff --git a/rust/NOTICE b/rust/NOTICE index 09bc7d2..8de2b4f 100644 --- a/rust/NOTICE +++ b/rust/NOTICE @@ -5,9 +5,10 @@ This product includes conformance artifacts from Jacquard Core, https://github.com/jbwinters/jacquard-lang, licensed under Apache License 2.0. The artifacts under src/jacquard_host/data/conformance/host-protocol-v0/, and their byte-identical copies under go/kit/data/host-protocol-v0/, -rust/kit/host-protocol-v0/, and -java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, are -copied verbatim from the commits recorded in docs/COMPATIBILITY.md. +rust/kit/host-protocol-v0/, +java/src/main/resources/org/jacquardlang/host/kit/host-protocol-v0/, and +dotnet/src/JacquardHost/kit/host-protocol-v0/, are copied verbatim from the +commits recorded in docs/COMPATIBILITY.md. Upstream notice: Jacquard diff --git a/scripts/core_pin.py b/scripts/core_pin.py index 3abf994..32124f1 100644 --- a/scripts/core_pin.py +++ b/scripts/core_pin.py @@ -38,11 +38,13 @@ PIN = REPO / "core-pin.json" SCHEMA = "jacquard-host-core-pin-v1" KIT_DIR = REPO / "src" / "jacquard_host" / "data" / "conformance" / "host-protocol-v0" -# Go, Rust, and Java embed their own byte-identical copies; go:embed cannot -# reach outside the module, a published crate contains only its own directory, -# and a jar carries its kit as class-path resources. +# Go, Rust, Java, and .NET embed their own byte-identical copies; go:embed +# cannot reach outside the module, a published crate contains only its own +# directory, a jar carries its kit as class-path resources, and a .NET +# assembly carries it as embedded resources. KIT_DIRS = (KIT_DIR, REPO / "go" / "kit" / "data" / "host-protocol-v0", REPO / "rust" / "kit" / "host-protocol-v0", - REPO / "java" / "src" / "main" / "resources" / "org" / "jacquardlang" / "host" / "kit" / "host-protocol-v0") + REPO / "java" / "src" / "main" / "resources" / "org" / "jacquardlang" / "host" / "kit" / "host-protocol-v0", + REPO / "dotnet" / "src" / "JacquardHost" / "kit" / "host-protocol-v0") KIT_FILES = ("kit/kit.json", "kit/transcripts.json", "kit/fixtures.jac", "vectors.json") # Copied with the kit but not part of the digest pin; prose may be refreshed. KIT_EXTRAS = ("kit/README.md",) @@ -263,12 +265,52 @@ def render_java(pin: dict) -> str: """ +def render_dotnet(pin: dict) -> str: + release, kit = pin["release"], pin["kit"] + digests = "".join(f' new("{name}", "{kit["digests"][name]}"),\n' for name in KIT_FILES) + pending = ", ".join(f'"{name}"' for name in kit["pending_decisions"]) + return f"""// Generated from core-pin.json by scripts/core_pin.py; do not edit by hand. + +using System.Collections.Generic; + +namespace JacquardLang.Host; + +/// The Jacquard Core release and host protocol this adapter is pinned to. +public static class Compat +{{ + /// The host protocol. + public const string Protocol = "{pin["protocol"]}"; + + /// The carrier. + public const string Carrier = "{pin["carrier"]}"; + + /// The pinned Core version. + public const string CoreVersion = "{release["version"]}"; + + /// The pinned Core release tag. + public const string CoreTag = "{release["tag"]}"; + + /// The pinned Core commit. + public const string CoreCommit = "{release["commit"]}"; + + /// SHA-256 digests of the pinned conformance kit files. + public static IReadOnlyList> KitDigests {{ get; }} = new KeyValuePair[] + {{ +{digests} }}; + + /// Decisions Core recorded as pending; replay reproduces them. + public static IReadOnlyList PendingDecisions {{ get; }} = new[] {{ {pending} }}; +}} +""" + + GENERATED = { "src/jacquard_host/compat.py": render_python, "typescript/src/compat.ts": render_typescript, "go/protocol/compat.go": render_go, "rust/src/compat.rs": render_rust, "java/src/main/java/org/jacquardlang/host/Compat.java": render_java, + "dotnet/src/JacquardHost/Compat.cs": render_dotnet, } BLOCKS = { "docs/COMPATIBILITY.md": render_compatibility, diff --git a/tests/test_core_pin.py b/tests/test_core_pin.py index 3032389..d6a040f 100644 --- a/tests/test_core_pin.py +++ b/tests/test_core_pin.py @@ -74,6 +74,16 @@ def test_hand_edits_and_kit_tampering_are_reported(self) -> None: found = core_pin.problems(self.pin, root) self.assertTrue(any("java/src/main/resources" in item and "transcripts.json" in item for item in found), found) self.assertTrue(any("Compat.java" in item for item in found), found) + dotnet_kit = root / core_pin.KIT_DIRS[4].relative_to(REPO) / "kit/fixtures.jac" + dotnet_kit.write_bytes(dotnet_kit.read_bytes() + b" ") + dotnet_readme = root / core_pin.KIT_DIRS[4].relative_to(REPO) / "kit/README.md" + dotnet_readme.write_text("drifted") + compat_cs = root / "dotnet/src/JacquardHost/Compat.cs" + compat_cs.write_text(compat_cs.read_text().replace(self.pin["kit"]["digests"]["vectors.json"], "0" * 64)) + found = core_pin.problems(self.pin, root) + self.assertTrue(any("dotnet/src/JacquardHost/kit" in item and "fixtures.jac" in item for item in found), found) + self.assertTrue(any("dotnet/src/JacquardHost/kit" in item and "README.md" in item for item in found), found) + self.assertTrue(any("Compat.cs" in item for item in found), found) def test_sync_rewrites_only_generated_regions(self) -> None: root = self.copy() @@ -221,6 +231,9 @@ def setUp(self) -> None: self.java_kit = Path(tempfile.mkdtemp(prefix="core-java-kit-")) self.addCleanup(shutil.rmtree, self.java_kit) shutil.copytree(core_pin.KIT_DIR, self.java_kit, dirs_exist_ok=True) + self.dotnet_kit = Path(tempfile.mkdtemp(prefix="core-dotnet-kit-")) + self.addCleanup(shutil.rmtree, self.dotnet_kit) + shutil.copytree(core_pin.KIT_DIR, self.dotnet_kit, dirs_exist_ok=True) self.files = {name: (core_pin.KIT_DIR / name).read_bytes() for name in (*core_pin.KIT_FILES, *core_pin.KIT_EXTRAS)} def run_update(self, manifest_changes: dict, assets: dict) -> dict: @@ -231,7 +244,7 @@ def run_update(self, manifest_changes: dict, assets: dict) -> dict: with patch.object(core_pin, "tag_commit", return_value="e" * 40), \ patch.object(core_pin, "kit_file", side_effect=lambda repo, commit, path, checkout: files[path.split("/", 2)[2]]), \ patch.object(core_pin, "release_assets", return_value=assets) as self.fetched: - return core_pin.update(self.pin, "jacquard-core-9.9.9", None, (self.kit, self.go_kit, self.rust_kit, self.java_kit)) + return core_pin.update(self.pin, "jacquard-core-9.9.9", None, (self.kit, self.go_kit, self.rust_kit, self.java_kit, self.dotnet_kit)) def test_a_release_is_adopted_with_its_kit_and_assets(self) -> None: assets = {"linux-x86_64": {"name": "jacquard-linux-x86_64.tar.gz", "sha256": "c" * 64}} @@ -243,6 +256,7 @@ def test_a_release_is_adopted_with_its_kit_and_assets(self) -> None: self.assertEqual((self.go_kit / "kit/kit.json").read_bytes(), written) self.assertEqual((self.rust_kit / "kit/kit.json").read_bytes(), written) self.assertEqual((self.java_kit / "kit/kit.json").read_bytes(), written) + self.assertEqual((self.dotnet_kit / "kit/kit.json").read_bytes(), written) self.assertEqual([path.name for path in self.kit.rglob(".*.core-pin")], []) def test_a_failed_replacement_restores_every_kit_file(self) -> None: From d9039dbfb78125348d32a7f337ff9c42cbf5cca8 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 01:22:23 -0400 Subject: [PATCH 02/33] Test the .NET adapter offline and against Core NUnit tests on .NET 8 and 10 cover the strict codecs, exact integers, limits, Core-order descriptor checks, the session state machine, and bounded encoding and validation. A stand-in worker program replays all 21 recordings and simulates faulty and hostile workers: malformed, extra, and oversized frames; silent, stalled, truncated, and nonzero workers; a worker that never reads its input; descendants that hold the pipes, daemonize, or flood stdout or stderr. Test seams (internal, null outside tests) stop a call right after a request is decoded and right before the handler thread's check. Linux tests count descriptors and handler threads across successes and failures. Live tests replay the kit against Core and run the README example, failure pairs, and handler deadlines; with JACQUARD_REQUIRE_LIVE set they fail rather than skip. The README documents the API, exact integers and money as integer minor units, cancellation, and the carrier; a test keeps its example identical to the compiled one. --- dotnet/JacquardHost.sln | 71 +++ dotnet/README.md | 207 ++++++++ .../JacquardHost.StandIn.csproj | 16 + dotnet/test/JacquardHost.StandIn/Program.cs | 363 ++++++++++++++ dotnet/test/JacquardHost.Tests/CodecTests.cs | 454 ++++++++++++++++++ .../JacquardHost.Tests/DescriptorTests.cs | 183 +++++++ .../test/JacquardHost.Tests/DocumentTests.cs | 45 ++ dotnet/test/JacquardHost.Tests/HostTests.cs | 429 +++++++++++++++++ .../JacquardHost.Tests.csproj | 34 ++ dotnet/test/JacquardHost.Tests/LiveTests.cs | 217 +++++++++ .../JacquardHost.Tests/ProcessTreeTests.cs | 191 ++++++++ dotnet/test/JacquardHost.Tests/Readme/Echo.cs | 33 ++ dotnet/test/JacquardHost.Tests/Support.cs | 290 +++++++++++ 13 files changed, 2533 insertions(+) create mode 100644 dotnet/JacquardHost.sln create mode 100644 dotnet/README.md create mode 100644 dotnet/test/JacquardHost.StandIn/JacquardHost.StandIn.csproj create mode 100644 dotnet/test/JacquardHost.StandIn/Program.cs create mode 100644 dotnet/test/JacquardHost.Tests/CodecTests.cs create mode 100644 dotnet/test/JacquardHost.Tests/DescriptorTests.cs create mode 100644 dotnet/test/JacquardHost.Tests/DocumentTests.cs create mode 100644 dotnet/test/JacquardHost.Tests/HostTests.cs create mode 100644 dotnet/test/JacquardHost.Tests/JacquardHost.Tests.csproj create mode 100644 dotnet/test/JacquardHost.Tests/LiveTests.cs create mode 100644 dotnet/test/JacquardHost.Tests/ProcessTreeTests.cs create mode 100644 dotnet/test/JacquardHost.Tests/Readme/Echo.cs create mode 100644 dotnet/test/JacquardHost.Tests/Support.cs diff --git a/dotnet/JacquardHost.sln b/dotnet/JacquardHost.sln new file mode 100644 index 0000000..bc797ee --- /dev/null +++ b/dotnet/JacquardHost.sln @@ -0,0 +1,71 @@ + +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio Version 17 +VisualStudioVersion = 17.0.31903.59 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{827E0CD3-B72D-47B6-A68D-7590B98EB39B}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "JacquardHost", "src\JacquardHost\JacquardHost.csproj", "{3C269BF6-7938-464F-B385-6997B38B63B9}" +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "test", "test", "{0C88DD14-F956-CE84-757C-A364CCF449FC}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "JacquardHost.StandIn", "test\JacquardHost.StandIn\JacquardHost.StandIn.csproj", "{CB695A56-2929-47FB-B93C-3E74883663E2}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "JacquardHost.Tests", "test\JacquardHost.Tests\JacquardHost.Tests.csproj", "{3ED29B8E-500E-4D0D-B6AD-F987037225AD}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Any CPU = Debug|Any CPU + Debug|x64 = Debug|x64 + Debug|x86 = Debug|x86 + Release|Any CPU = Release|Any CPU + Release|x64 = Release|x64 + Release|x86 = Release|x86 + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {3C269BF6-7938-464F-B385-6997B38B63B9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Debug|Any CPU.Build.0 = Debug|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Debug|x64.ActiveCfg = Debug|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Debug|x64.Build.0 = Debug|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Debug|x86.ActiveCfg = Debug|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Debug|x86.Build.0 = Debug|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Release|Any CPU.ActiveCfg = Release|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Release|Any CPU.Build.0 = Release|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Release|x64.ActiveCfg = Release|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Release|x64.Build.0 = Release|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Release|x86.ActiveCfg = Release|Any CPU + {3C269BF6-7938-464F-B385-6997B38B63B9}.Release|x86.Build.0 = Release|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Debug|Any CPU.Build.0 = Debug|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Debug|x64.ActiveCfg = Debug|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Debug|x64.Build.0 = Debug|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Debug|x86.ActiveCfg = Debug|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Debug|x86.Build.0 = Debug|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Release|Any CPU.ActiveCfg = Release|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Release|Any CPU.Build.0 = Release|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Release|x64.ActiveCfg = Release|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Release|x64.Build.0 = Release|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Release|x86.ActiveCfg = Release|Any CPU + {CB695A56-2929-47FB-B93C-3E74883663E2}.Release|x86.Build.0 = Release|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Debug|Any CPU.Build.0 = Debug|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Debug|x64.ActiveCfg = Debug|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Debug|x64.Build.0 = Debug|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Debug|x86.ActiveCfg = Debug|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Debug|x86.Build.0 = Debug|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Release|Any CPU.ActiveCfg = Release|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Release|Any CPU.Build.0 = Release|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Release|x64.ActiveCfg = Release|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Release|x64.Build.0 = Release|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Release|x86.ActiveCfg = Release|Any CPU + {3ED29B8E-500E-4D0D-B6AD-F987037225AD}.Release|x86.Build.0 = Release|Any CPU + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection + GlobalSection(NestedProjects) = preSolution + {3C269BF6-7938-464F-B385-6997B38B63B9} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} + {CB695A56-2929-47FB-B93C-3E74883663E2} = {0C88DD14-F956-CE84-757C-A364CCF449FC} + {3ED29B8E-500E-4D0D-B6AD-F987037225AD} = {0C88DD14-F956-CE84-757C-A364CCF449FC} + EndGlobalSection +EndGlobal diff --git a/dotnet/README.md b/dotnet/README.md new file mode 100644 index 0000000..418f97a --- /dev/null +++ b/dotnet/README.md @@ -0,0 +1,207 @@ +# Jacquard Host for .NET + +An independent .NET adapter for Jacquard Core's experimental `jacquard-host-v0` +protocol. It implements the pinned protocol on its own, without calling +Python, Node.js, Go, Rust, or Java, and has no runtime dependencies beyond +the .NET base class library. It targets .NET 8 and runs on .NET 8 and 10. It +runs workers on Linux and macOS: the carrier calls the C library's `poll(2)` +(see "Process carrier" below). + +This is the protocol-level adapter, comparable to the Go `host` package, the +Rust `host` module, and the Java adapter. It includes strict codecs, the host +session state machine, and a bounded process carrier for `jacquard host +worker`. It also provides a `Host` with a closed operation registry and a +replayer for Core's pinned conformance kit. There is no SDK layer yet. It does +not compile programs, map native types, or cache stores. You prepare a store +with the Core CLI and pass exact identities and boundary types. + +**Experimental:** the protocol is Core's pinned `jacquard-host-v0`, and Core is +installed separately. Each invocation runs one fresh, serial worker. The host +and operating system are trusted. Linux is exercised; macOS should work but has +no CI; Windows is not supported (the worker reports a spawn failure). See +[limits](../docs/LIMITS.md). + +## Use + +The package is not published to a registry. Build it from a clone +(`dotnet pack src/JacquardHost -c Release` in this directory) and reference +the `JacquardLang.Host` package or the project. The example below invokes a +checked function whose single operation is served by a .NET handler: + +```csharp +using System; +using JacquardLang.Host; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Protocol; + +public static class Echo +{ + public static Value Run(string store, Hash target, Hash text, Hash world, Hash send) + { + // The command defaults to $JACQUARD_BIN, then `jacquard`. + var options = new HostOptions { Carrier = new WorkerOptions { Deadline = TimeSpan.FromSeconds(5) } }; + var host = new Host(store, options); + OperationRegistry registry = new OperationRegistry().Bind(world, send, (context, args) => + { + if (context.IsCancelled) + { + throw HandlerException.CompletionUnknown("cancelled"); + } + + if (args is [TextValue input]) + { + return new TextValue(input.Value + "!"); + } + + throw HandlerException.OutsideFailed("expected one text argument"); + }); + var textType = new NominalType(text); + var spec = new CallInterface(new[] { textType }, textType, new[] { world }); + return host.Invoke(target, spec, new[] { new TextValue("ping") }, registry).ValueOrThrow(); + } +} +``` + +`Invoke` returns an `InvocationResult` in every case; only a null argument +throws (`ArgumentNullException`), before any worker starts. The result keeps +Core's frames, diagnostics, and evidence. It also keeps the host's frames, +exit status, bounded stderr, and host evidence +(`jacquard-host-dotnet-evidence-v0`). `ValueOrThrow` returns the value, or +throws an `InvocationException` whose `Failure` is one of `DeadlineExceeded`, +`WorkerLost`, `OperationFailed`, or `Other`. + +## Values and numbers + +Values are `IntValue`, `TextValue`, `RealValue`, `HashValue`, `TupleValue` +(`Value.Unit` is the empty tuple), and `ConstructorValue`; they compare +structurally. + +- **Integers are exact.** `IntValue` holds Core's signed 63-bit integer in a + `long`. On the wire an `Int` is decimal text, and JSON integers in frames + are decimal tokens; both are parsed from their text as a `long`, never + through a `double` or a `decimal`, so every value in `-2^62..2^62-1` + crosses unchanged. A non-canonical text (`"01"`, `"-0"`, `"+1"`), a + fraction or exponent, and anything outside the range are refused (E1601), + as the Rust and Java adapters refuse them. A `long` outside the range has + no encoding and is refused with E1604. +- **Money crosses as integer minor units**: an amount of 12.34 in a currency + with two decimal places is `IntValue(1234)`. Do not use `RealValue` for + money. +- **Reals are binary64.** `RealValue` keeps the exact IEEE 754 bits, so NaN + payloads and signed zero survive; `RealValue.Of(double)` and `ToDouble()` + convert. +- **Text is Unicode scalars.** A string with a lone surrogate, which .NET + strings can hold, is refused with E1601, unless it is already too long for + `max_text_bytes` or the frame, which is refused with E1602 before it is + scanned. + +## Handlers and cancellation + +A handler returns a value or throws a `HandlerException`. The four structured +categories are `Unsupported`, `Refused`, `OutsideFailed`, and +`CompletionUnknown`. Any other exception a handler throws is reported to Core +as completion unknown, and its message is never sent. A returned value with +no encoding within the selected limits, including `null`, is reported as an +outside failure. A message longer than `max_host_message_bytes` is cut at a +code point, and a lone surrogate in it becomes U+FFFD. + +Each handler runs on its own background thread, bounded by the worker's +lifetime deadline. No handler starts once the call is cancelled or past its +lifetime: the host checks after decoding the request, before it creates the +handler thread, and the thread checks again just before calling the handler. +A stop that lands after that check is seen by the handler through its +context. At the deadline, or when the caller's `CancellationToken` is +cancelled, the host kills the worker's process tree and sends no late +response. Cancellation is cooperative: `HandlerContext.IsCancelled` turns true +as soon as the call is cancelled or its lifetime passes, and stays true once +the invocation ends, and `HandlerContext.CancellationToken` is cancelled when +the caller's token is, at the deadline, and when the invocation ends. A +handler that ignores both keeps running on its thread until it returns. +Nothing is retried. + +## Process carrier + +.NET cannot start a process in its own process group, and its pipe streams +offer no read or write that a deadline or a cancellation can interrupt: on +Unix an anonymous pipe's asynchronous read is a blocking read on a pool +thread. The carrier is built around both: + +- **No read or write ever blocks.** The caller's thread asks the C library's + `poll(2)` (the one native call the adapter makes) whether a pipe is ready, + in waits of at most 5 ms, and only then reads the bytes already in the pipe, + or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a + pipe that poll reports writable accepts at once. Every wait, whether for a + frame, a write, a handler, or the exit, rechecks the one lifetime deadline, + the `CancellationToken`, and disposal (`Worker.Dispose` may come from + another thread). No thread reads or writes the worker's pipes, so none can + be left blocked on them, and a worker that never reads its input cannot + hold a write past the deadline. +- **Nothing drains the pipes behind the host's back.** .NET's `Process` reads + redirected output on its own only when asked to (`BeginOutputReadLine`), + which the carrier never does. stderr is read whenever the caller waits, at + most 1 MiB per read pass, and only a bounded prefix + (`max_stderr_bytes`) is kept. +- **Expiry kills a process tree, not a group.** At expiry, cancellation, or + disposal of a running worker, the host calls `Process.Kill(true)`: .NET + stops the worker, lists its children from the process table, kills it, and + does the same for each child. A descendant that was already reparented is + not found and keeps running: one whose parent exited, or one that + daemonized. So is one started after its parent was listed. +- **A descendant holding the pipes after the exit is detected.** The runtime + reaps the worker when it exits. If stdout or stderr still has a writer + after that, a descendant holds it: the host waits up to one second for it to + close, then reports a carrier loss ("worker descendants kept its pipes open + after exit"), as the Go and Rust hosts do. It cannot stop that descendant; + closing its ends of the pipes at disposal makes the descendant's next write + fail. +- **Signals are inferred.** .NET reports a process ended by signal N as exit + status 128 + N. The host reports `Signal == "SIGKILL"` only when it killed + the worker itself and the status is 137; any other value is reported as the + exit status. + +What this guarantees: an invocation returns at its deadline plus bounded +cleanup. Each cleanup wait is capped at one second: for a killed worker's exit +to be recorded, and for a still-running worker to exit at disposal. So in the +worst case `Invoke` returns about two seconds after the deadline. This holds +while a descendant that escaped the kill floods stdout or stderr, or holds +stdin without reading, and the flood does not reach the heap. The carrier +leaves no thread or pipe descriptor behind; a handler that ignores +cancellation keeps its own thread until it returns (see above). Linux tests +count descriptors and adapter threads across successful and failed +invocations, and check that a reparented descendant flooding stdout or +stderr holds neither the caller nor the heap, at a worker's own exit, at +expiry, and at cancellation. + +## Conformance kit + +`Kit.Load()` verifies the embedded copy of the kit against the pinned digests. +`Replayer.ReplayKit` replays all 21 recordings, including the hostile host +inputs, against a worker. It reports any semantic differences and any drift in +non-normative diagnostic prose. It keeps at most one frame past each +recording's, so a worker that keeps sending cannot fill memory. + +## Development + +From this directory, with the .NET 8 or 10 SDK: + +```sh +dotnet test test/JacquardHost.Tests -f net8.0 +# With a .NET 10 SDK, also: +dotnet test test/JacquardHost.Tests -f net10.0 +# With the pinned Core configured (eval "$(python3 ../scripts/core_pin.py install)"): +JACQUARD_REQUIRE_LIVE=1 dotnet test test/JacquardHost.Tests -f net8.0 +``` + +The live tests in `LiveTests` are ignored without `JACQUARD_BIN` and +`JACQUARD_PRELUDE` (set `JACQUARD_REQUIRE_LIVE` to make them fail instead). +The offline tests drive `test/JacquardHost.StandIn`, a console program that +replays recordings or simulates faulty workers in a fresh process. Its +descendant modes use `sh` and `sleep`, so those tests run on Unix only, and +the descriptor tests run on Linux only. The build treats warnings as errors, +with nullable reference types and the `8.0-recommended` analyzers. + +`src/JacquardHost/Compat.cs` is generated, and +`src/JacquardHost/kit/host-protocol-v0/` is a byte copy of the pinned kit. +`scripts/core_pin.py` maintains both; do not edit them by hand. `NOTICE` must +stay identical to the repository's `NOTICE`, and the README example must stay +identical to `test/JacquardHost.Tests/Readme/Echo.cs`; tests check both. diff --git a/dotnet/test/JacquardHost.StandIn/JacquardHost.StandIn.csproj b/dotnet/test/JacquardHost.StandIn/JacquardHost.StandIn.csproj new file mode 100644 index 0000000..f7c78ac --- /dev/null +++ b/dotnet/test/JacquardHost.StandIn/JacquardHost.StandIn.csproj @@ -0,0 +1,16 @@ + + + + Exe + net8.0 + JacquardLang.Host.StandIn + JacquardLang.Host.StandIn + false + true + + + + + + + diff --git a/dotnet/test/JacquardHost.StandIn/Program.cs b/dotnet/test/JacquardHost.StandIn/Program.cs new file mode 100644 index 0000000..457c8fd --- /dev/null +++ b/dotnet/test/JacquardHost.StandIn/Program.cs @@ -0,0 +1,363 @@ +using System; +using System.Buffers.Binary; +using System.Diagnostics; +using System.Globalization; +using System.IO; +using System.Linq; +using System.Runtime.InteropServices; +using System.Text; +using System.Threading; +using JacquardLang.Host.Conformance; +using JacquardLang.Host.Protocol; +using Microsoft.Win32.SafeHandles; + +namespace JacquardLang.Host.StandIn; + +/// +/// A recorded or faulty worker for the offline tests. It implements no Core +/// semantics and is not part of the adapter's interface. The descendant modes +/// use sh and sleep, so they are Unix-only. +/// +public static partial class Program +{ + /// Names the stand-in behaviour. + public const string ModeVariable = "JACQUARD_DOTNET_STANDIN"; + + /// Names the transcript replayed in replay mode. + public const string CaseVariable = "JACQUARD_DOTNET_STANDIN_CASE"; + + // Descriptors 0, 1, and 2 themselves, so closing stdout really closes it. + private static readonly FileStream Out = new(new SafeFileHandle(1, ownsHandle: true), FileAccess.Write, 1); + private static readonly FileStream Err = new(new SafeFileHandle(2, ownsHandle: false), FileAccess.Write, 1); + private static readonly FileStream In = new(new SafeFileHandle(0, ownsHandle: false), FileAccess.Read, 1); + + /// Runs the mode named by . + public static int Main() + { + string mode = Env(ModeVariable); + switch (mode) + { + case "exit": + return 0; + case "silent": + return Block(); + case "truncated": + Out.Write(new byte[] { 0, 0, 0 }); + Out.Flush(); + return 0; + case "descendant": + // Leave a descendant holding stdout, like a launcher that does not exec. + Shell("sleep 30 /dev/null & echo $! >&2", wait: true); + return 0; + case "child-silent": + // A child holding stdout and stderr while the worker stays alive. + Report(Shell("exec sleep 30 &2; wait", wait: false); + return Block(); + case "daemon-silent": + // A descendant reparented while the worker is alive: its shell exits at once. + Shell("sleep 30 /dev/null & echo $! >&2", wait: true); + return Block(); + case "stderr-descendant": + case "stderr-descendant-nonzero": + // Answer normally, but leave a child holding only stderr. + Report(Shell("exec sleep 30 /dev/null", wait: false)); + return Fault(Kit.Load(), mode == "stderr-descendant" ? "ok" : "nonzero"); + case "escaped-descendant": + case "escaped-descendant-unknown-field": + // A daemonized descendant holds stderr; it is not the worker's + // descendant by the time the host could look for it. + Shell("sleep 30 /dev/null & echo $! >&2", wait: true); + return Fault(Kit.Load(), mode == "escaped-descendant" ? "ok" : "unknown-field"); + case "stdin-descendant": + case "stdin-child-alive": + { + // A child holds stdin and never reads. The worker reads the + // selection, then exits (or stays alive without reading), so + // the invoke write can only wait. + Report(Shell("exec sleep 30 >/dev/null 2>/dev/null", wait: false)); + Send(Kit.Load().Transcripts[0].CoreOutput[0]); + ReadFrame(); + return mode == "stdin-descendant" ? 0 : Block(); + } + + case "replay": + return Replay(Kit.Load(), Env(CaseVariable)); + case "large-request": + { + // An effect request whose one argument is a text at the text + // ceiling, so the host spends a while decoding it. + Transcript effect = Kit.Load().Find("one-once-effect-success")!; + Send(effect.CoreOutput[0]); + ReadFrame(); + ReadFrame(); + Json text = Json.Object(("kind", Json.Of("text")), ("value", Json.Of(new string('x', 262_144)))); + Send(effect.CoreOutput[1].With("arguments", Json.Of(new[] { text }))); + return Block(); + } + + case "flood-after-terminal": + { + // The pure recording, then valid frames of about 600 KB each. + Kit kit = Kit.Load(); + Replay(kit, kit.Transcripts[0].Name); + JsonObject padding = Json.Object(("padding", Json.Of(Enumerable.Range(0, 100_000).Select(i => (Json)Json.Of(i))))); + for (int i = 0; i < 300; i++) + { + Send(padding); + } + + return 0; + } + + case "": + return 2; + default: + return Fault(Kit.Load(), mode); + } + } + + private static string Env(string name) => Environment.GetEnvironmentVariable(name) ?? string.Empty; + + /// Starts sh -c script sharing this process's descriptors; returns its pid. + private static int Shell(string script, bool wait) + { + var info = new ProcessStartInfo("sh") { UseShellExecute = false }; + info.ArgumentList.Add("-c"); + info.ArgumentList.Add(script); + using Process shell = Process.Start(info) ?? throw new InvalidOperationException("sh did not start"); + if (wait) + { + shell.WaitForExit(); + } + + return shell.Id; + } + + private static void Report(int pid) + { + Err.Write(Encoding.ASCII.GetBytes(pid.ToString(CultureInfo.InvariantCulture))); + Err.Flush(); + } + + private static void Send(JsonObject frame) + { + try + { + byte[] payload = Encoding.UTF8.GetBytes(JsonCodec.Stringify(frame, 256)); + byte[] prefix = new byte[4]; + BinaryPrimitives.WriteUInt32BigEndian(prefix, (uint)payload.Length); + Out.Write(prefix); + Out.Write(payload); + Out.Flush(); + } + catch (IOException) + { + // The host may have closed the pipe. + } + } + + private static bool Discard(long count) + { + byte[] chunk = new byte[8192]; + try + { + for (long left = count; left > 0;) + { + int n = In.Read(chunk, 0, (int)Math.Min(chunk.Length, left)); + if (n <= 0) + { + return false; + } + + left -= n; + } + + return true; + } + catch (IOException) + { + return false; + } + } + + private static bool ReadFrame() + { + byte[] prefix = new byte[4]; + try + { + for (int filled = 0; filled < 4;) + { + int n = In.Read(prefix, filled, 4 - filled); + if (n <= 0) + { + return false; + } + + filled += n; + } + } + catch (IOException) + { + return false; + } + + return Discard(BinaryPrimitives.ReadUInt32BigEndian(prefix)); + } + + private static int Replay(Kit kit, string name) + { + Transcript? transcript = kit.Find(name); + if (transcript is null) + { + return 3; + } + + int index = 0; + bool raw = false; + foreach (TranscriptStep step in transcript.HostInput) + { + if (step.Direction == "core_to_host") + { + JsonObject frame = transcript.CoreOutput[index++]; + Send(frame); + if (frame.Get("kind")?.AsString() is "outcome" or "fatal" or "shutdown_ack") + { + return (int)transcript.ExitStatus; + } + } + else if (step.RawHex is not null) + { + Discard(step.RawHex.Length / 2); + raw = true; + } + else if (!step.AfterTerminal && !raw) + { + ReadFrame(); + } + } + + return (int)transcript.ExitStatus; + } + + /// + /// Waits without reading or writing. A minute bounds it, so a stand-in + /// left behind by a crashed test run still ends on its own. + /// + private static int Block() + { + Thread.Sleep(TimeSpan.FromMinutes(1)); + return 0; + } + + private static int Fault(Kit kit, string mode) + { + Transcript pure = kit.Transcripts[0]; + JsonObject hello = pure.CoreOutput[0]; + JsonObject outcome = pure.CoreOutput[1]; + if (mode == "stderr") + { + byte[] noise = new byte[2 * 1024 * 1024]; + Array.Fill(noise, (byte)'x'); + Err.Write(noise); + Err.Flush(); + } + + if (mode == "unknown-field") + { + hello = hello.With("extra", Json.Of(1)); + } + + Send(hello); + if (mode == "oversized-early") + { + // Without waiting for host_select, send a frame larger than the + // selected max_frame_bytes but within the hard limit. + Send(Json.Object(("padding", Json.Of(new string('x', 4096))))); + } + + if (mode == "no-read") + { + return Block(); + } + + int count = 1; + while (ReadFrame()) + { + if (count == 2) + { + if (mode == "invalid-hash") + { + outcome = WithCallable(outcome, new string('A', 64)); + } + + Send(outcome); + if (mode == "extra") + { + Send(outcome); + } + + if (mode == "stall") + { + return Block(); + } + + // Close stdout so the host sees end of stream before the exit. + CloseStdout(); + if (mode == "slow-exit") + { + Thread.Sleep(1500); + } + + if (mode == "hang-after-close") + { + return Block(); + } + + return mode == "nonzero" ? 74 : 0; + } + + count++; + } + + return 0; + } + + [LibraryImport("libc", EntryPoint = "close", SetLastError = true)] + private static partial int Close(int fd); + + /// + /// Closes descriptor 1 and every other descriptor on the same pipe: the + /// .NET runtime keeps a duplicate of stdout, which would hold the pipe + /// open after descriptor 1 is closed. + /// + private static void CloseStdout() + { + Out.Flush(); + if (Directory.Exists("/proc/self/fd")) + { + string pipe = new FileInfo("/proc/self/fd/1").LinkTarget ?? string.Empty; + foreach (string entry in Directory.GetFiles("/proc/self/fd")) + { + if (int.TryParse(Path.GetFileName(entry), NumberStyles.None, CultureInfo.InvariantCulture, out int fd) && fd != 1 + && new FileInfo(entry).LinkTarget == pipe) + { + Close(fd); + } + } + } + + Out.Dispose(); + } + + private static JsonObject WithCallable(JsonObject outcome, string callable) + { + JsonObject evidence = (JsonObject)outcome["evidence"]; + JsonObject core = (JsonObject)evidence["core"]; + JsonObject target = (JsonObject)core["target"]; + return outcome.With("evidence", evidence.With("core", core.With("target", target.With("callable", Json.Of(callable))))); + } +} diff --git a/dotnet/test/JacquardHost.Tests/CodecTests.cs b/dotnet/test/JacquardHost.Tests/CodecTests.cs new file mode 100644 index 0000000..9e014d1 --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/CodecTests.cs @@ -0,0 +1,454 @@ +using System; +using System.Collections; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using JacquardLang.Host.Conformance; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// Strict JSON, exact numbers, value and type codecs, limits, and the session state machine. +[TestFixture] +public sealed class CodecTests +{ + private static readonly Limits Hard = Limits.Hard; + + private static Budget NewBudget() => new(Hard); + + [Test] + public void StrictJsonRefusals() + { + foreach (string text in new[] + { + "{\"a\":1,\"a\":2}", + "{\"x\":{\"a\":1,\"a\":2}}", + "{\"x\":1.0}", + "{\"x\":1e0}", + "{\"x\":1E5}", + "{\"x\":NaN}", + "{\"x\":Infinity}", + "{\"x\":-Infinity}", + "{\"x\":\"\\ud800\"}", + "{\"x\":\"\\udc00\"}", + "{\"x\":\"\\ud800A\"}", + "{}{}", + "[]", + "{\"x\":4611686018427387904}", + "{\"x\":-4611686018427387905}", + "{\"x\":9223372036854775808}", + "{\"x\":123456789012345678901}", + "{\"x\":01}", + "{\"x\":+1}", + "{\"x\":.5}", + "{\"x\":true,}", + "{\"x\":[1,]}", + "{\"x\":\"\u0001\"}", + "{\"x\":\"\\q\"}", + "{\"x\":-}", + "{\"x\":\"\\u+abc\"}", + "{\"x\":\"\\u\uff10\uff10\uff10\uff10\"}", + "\ufeff{}", + }) + { + Code("E1601", text, () => JsonCodec.DecodeObject(Encoding.UTF8.GetBytes(text), 64)); + } + + Code("E1601", "invalid UTF-8", () => JsonCodec.DecodeObject(new byte[] { 0x7b, 0x22, 0xff, 0x22, 0x3a, 0x30, 0x7d }, 64)); + + // An encoded surrogate (CESU-8) and an overlong encoding are not UTF-8. + Code("E1601", "encoded surrogate", () => JsonCodec.DecodeObject(new byte[] { 0x7b, 0x22, 0xed, 0xa0, 0x80, 0x22, 0x3a, 0x30, 0x7d }, 64)); + Code("E1601", "overlong", () => JsonCodec.DecodeObject(new byte[] { 0x7b, 0x22, 0xc0, 0xaf, 0x22, 0x3a, 0x30, 0x7d }, 64)); + JsonObject parsed = JsonCodec.DecodeObject(Encoding.UTF8.GetBytes("{\"a\":4611686018427387903,\"b\":-4611686018427387904,\"c\":\"😀é\"}"), 64); + Assert.That(parsed["a"], Is.EqualTo(Json.Of(Json.IntMax))); + Assert.That(parsed["b"], Is.EqualTo(Json.Of(Json.IntMin))); + Assert.That(parsed["c"], Is.EqualTo(Json.Of("😀é"))); + Assert.That(JsonCodec.DecodeObject(Encoding.UTF8.GetBytes("{\"x\":\"\\ud83d\\ude00\"}"), 64)["x"], Is.EqualTo(Json.Of("😀"))); + Json value = Json.Object( + ("z", Json.Of(new Json[] { Json.Null, Json.Of(true), Json.Of(-1) })), + ("a", Json.Of("q\"\\\n\u0001😀")), + ("__proto__", JsonObject.Empty)); + Assert.That(JsonCodec.Stringify(value, 64), Is.EqualTo("{\"__proto__\":{},\"a\":\"q\\\"\\\\\\n\\u0001😀\",\"z\":[null,true,-1]}")); + Code("E1601", "63-bit encode", () => JsonCodec.Stringify(Json.Object(("x", Json.Of(Json.IntMax + 1))), 64)); + + // Keys sort by code point, not by UTF-16 unit: U+FFFF precedes U+1F600. + Assert.That(JsonCodec.Stringify(Json.Object(("😀", Json.Of(2)), ("\uffff", Json.Of(1))), 64), Is.EqualTo("{\"\uffff\":1,\"😀\":2}")); + Code("E1601", "lone surrogate encode", () => JsonCodec.Stringify(Json.Of("\ud800"), 64)); + Throws(() => Json.Object(("a", Json.Of(1)), ("a", Json.Of(2)))); + } + + [Test] + public void IntegersAreParsedExactlyFromTheirText() + { + // Each of these differs from its nearest double: parsing through a + // double or a decimal would change or refuse them. + foreach (long exact in new[] { 9_007_199_254_740_993L, 4_611_686_018_427_387_903L, -4_611_686_018_427_387_904L, 4_611_686_018_427_387_901L, 0L }) + { + string text = "{\"x\":" + exact.ToString(System.Globalization.CultureInfo.InvariantCulture) + "}"; + Assert.That(JsonCodec.DecodeObject(Encoding.UTF8.GetBytes(text), 64)["x"].AsInt(), Is.EqualTo(exact), text); + Assert.That(JsonCodec.Stringify(JsonCodec.DecodeObject(Encoding.UTF8.GetBytes(text), 64), 64), Is.EqualTo(text)); + } + + // As in the Rust and Java adapters, "-0" is the integer 0, and a + // token longer than 20 characters is refused before it is parsed. + Assert.That(JsonCodec.DecodeObject(Encoding.UTF8.GetBytes("{\"x\":-0}"), 64)["x"], Is.EqualTo(Json.Of(0))); + ProtocolException error = Code("E1601", "21 digits", () => JsonCodec.DecodeObject(Encoding.UTF8.GetBytes("{\"x\":-12345678901234567890}"), 64)); + Assert.That(error.Detail, Is.EqualTo("invalid JSON scalar")); + error = Code("E1601", "2^62", () => JsonCodec.DecodeObject(Encoding.UTF8.GetBytes("{\"x\":4611686018427387904}"), 64)); + Assert.That(error.Detail, Is.EqualTo("JSON integer is outside the 63-bit range")); + + // Core's Int crosses as canonical decimal text and is parsed exactly too. + foreach (long exact in new[] { 9_007_199_254_740_993L, Json.IntMax, Json.IntMin, -1L }) + { + Json wire = Json.Object(("kind", Json.Of("int")), ("value", Json.Of(exact.ToString(System.Globalization.CultureInfo.InvariantCulture)))); + Assert.That(Codec.DecodeValue(wire, NewBudget()), Is.EqualTo(new IntValue(exact))); + Assert.That(Codec.EncodeValue(new IntValue(exact), NewBudget()), Is.EqualTo(wire)); + } + } + + [Test] + public void DepthFrameAndLimits() + { + Code("E1602", "decode depth", () => JsonCodec.DecodeObject(Encoding.UTF8.GetBytes("{\"a\":[[]]}"), 2)); + Code("E1602", "frame bytes", () => JsonCodec.FrameBytes(Json.Object(("a", Json.Of("xx"))), 4, 64)); + Assert.That(JsonCodec.FrameBytes(Json.Object(("a", Json.Of(1))), 64, 64), Is.EqualTo(Encoding.UTF8.GetBytes("\0\0\0\u0007{\"a\":1}"))); + Code("E1602", "zero limit", () => Hard.With("max_arguments", 0).Validate()); + Code("E1601", "extra limit", () => Limits.FromJson(Hard.ToJson().With("extra", Json.Of(1)))); + Code("E1601", "boolean limit", () => Limits.FromJson(Hard.ToJson().With("max_arguments", Json.Of(true)))); + Code("E1601", "limit beyond the exact integer range", () => Limits.FromJson(Hard.ToJson().With("max_arguments", Json.Of(1L << 60)))); + Assert.That(Limits.FromJson(Hard.ToJson()), Is.EqualTo(Hard)); + } + + [Test] + public void ValuesRoundtripExactly() + { + Value[] values = + { + new IntValue(0), + new IntValue(Json.IntMin), + new IntValue(Json.IntMax), + new TextValue("hé😀"), + new HashValue(HashOf('a')), + new TupleValue(new IntValue(3), new TextValue("x")), + new ConstructorValue(HashOf('b'), new[] { new IntValue(3) }), + new RealValue(0x7ff0_0000_0000_0001UL), + RealValue.Of(-0.0), + RealValue.Of(double.PositiveInfinity), + Value.Unit, + }; + foreach (Value value in values) + { + Json first = Codec.EncodeValue(value, NewBudget()); + Value decoded = Codec.DecodeValue(first, NewBudget()); + Assert.That(decoded, Is.EqualTo(value)); + Json second = Codec.EncodeValue(decoded, NewBudget()); + Assert.That(JsonCodec.Stringify(second, 64), Is.EqualTo(JsonCodec.Stringify(first, 64))); + } + + Assert.That(RealValue.Of(-0.0), Is.EqualTo(new RealValue(0x8000_0000_0000_0000UL))); + Assert.That(JsonCodec.Stringify(Codec.EncodeValue(RealValue.Of(-0.0), NewBudget()), 64), Is.EqualTo("{\"bits\":\"8000000000000000\",\"kind\":\"real\"}")); + foreach (string text in new[] { "-0", "00", "+1", "4611686018427387904", "-4611686018427387905", " 1", "99999999999999999999", "1.0", "1e3", "１" }) + { + Json wire = Json.Object(("kind", Json.Of("int")), ("value", Json.Of(text))); + Code("E1601", text, () => Codec.DecodeValue(wire, NewBudget())); + } + + Code("E1604", "secret", () => Codec.DecodeValue(Parse("{\"kind\":\"secret\",\"value\":\"x\"}"), NewBudget())); + Code("E1604", "Int range", () => Codec.EncodeValue(new IntValue(Json.IntMax + 1), NewBudget())); + Code("E1604", "Int range below", () => Codec.EncodeValue(new IntValue(long.MinValue), NewBudget())); + Code("E1604", "null", () => Codec.EncodeValue(null, NewBudget())); + Code("E1601", "lone surrogate Text", () => Codec.EncodeValue(new TextValue("a\udc00"), NewBudget())); + Code("E1602", "Text bytes", () => Codec.EncodeValue(new TextValue(string.Concat(Enumerable.Repeat("é", 131_073))), NewBudget())); + string upper = "{\"kind\":\"nominal\",\"identity\":\"" + new string('A', 64) + "\",\"arguments\":[]}"; + Code("E1601", "uppercase hash", () => Codec.DecodeType(Parse(upper), NewBudget())); + Limits small = Hard.With("max_value_nodes", 1); + Code("E1602", "value nodes", () => Codec.DecodeValue(Parse("{\"kind\":\"tuple\",\"items\":[{\"kind\":\"int\",\"value\":\"1\"}]}"), new Budget(small))); + Code("E1601", "uppercase real", () => Codec.DecodeValue(Parse("{\"kind\":\"real\",\"bits\":\"7FF0000000000001\"}"), NewBudget())); + Code("E1601", "short hash", () => Hash.Parse("A")); + Throws(() => _ = new Hash("A")); + } + + [Test] + public void DecodersFollowCoresCheckOrder() + { + string item = "{\"kind\":\"int\",\"value\":\"1\"}"; + string many = string.Join(",", Enumerable.Repeat(item, 65)); + + // Core: exact fields, one node, identity, then the count. + string bad = "{\"kind\":\"constructor\",\"identity\":\"bad\",\"arguments\":[" + many + "]}"; + Code("E1601", "constructor identity before count", () => Codec.DecodeValue(Parse(bad), NewBudget())); + string good = "{\"kind\":\"constructor\",\"identity\":\"" + new string('a', 64) + "\",\"arguments\":[" + many + "]}"; + Code("E1602", "constructor argument count", () => Codec.DecodeValue(Parse(good), NewBudget())); + + // A malformed descriptor is refused as malformed even when the budget is spent. + var spent = new Budget(Hard.With("max_value_nodes", 1)); + Codec.DecodeValue(Parse(item), spent); + string nominal = "{\"kind\":\"nominal\",\"identity\":\"" + new string('a', 64) + "\"}"; + Code("E1601", "exact fields before the node", () => Codec.DecodeType(Parse(nominal), spent)); + Code("E1604", "kind before the node", () => Codec.DecodeValue(Parse("{\"kind\":\"secret\"}"), spent)); + Code("E1601", "missing kind", () => Codec.DecodeValue(Parse("{\"value\":\"1\"}"), NewBudget())); + } + + private static JsonObject Hello() => Json.Object( + ("carrier", Json.Of(Compat.Carrier)), + ("kind", Json.Of("core_hello")), + ("limits", Hard.ToJson()), + ("versions", Json.Of(new Json[] { Json.Of(Compat.Protocol) }))); + + [Test] + public void HelloAndSelection() + { + Code("E1601", "extra hello field", () => new Session().Receive(Hello().With("extra", Json.Of(1)))); + Code("E1600", "unknown version", () => new Session().Receive(Hello().With("versions", Json.Of(new Json[] { Json.Of("jacquard-host-v9") })))); + Code("E1601", "lowered hello limits", () => new Session().Receive(Hello().With("limits", Hard.With("max_arguments", 1).ToJson()))); + Code("E1608", "first frame", () => new Session().Receive(Json.Object(("kind", Json.Of("fatal")), ("protocol", Json.Of(Compat.Protocol))))); + var session = new Session(); + Assert.That(session.Receive(Hello()), Is.Null); + Code("E1608", "invoke before select", () => session.Invoke(HashOf('a'), Unit(), Array.Empty(), Array.Empty())); + Code("E1602", "select above ceiling", () => session.Select(Hard.With("max_frame_bytes", Hard.MaxFrameBytes + 1))); + } + + [Test] + public void InvokeFrameValidation() + { + Hash a = HashOf('a'); + Hash b = HashOf('b'); + var unsorted = new CallInterface(Array.Empty(), new TupleType(), new[] { b, a }); + Code("E1601", "unsorted effects", () => Codec.InvokeFrame(a, unsorted, Array.Empty(), Array.Empty(), Hard)); + var spec = new CallInterface(Array.Empty(), new TupleType(), new[] { a }); + Code("E1601", "operation outside row", () => Codec.InvokeFrame(a, spec, Array.Empty(), new[] { new OperationEntry(b, a) }, Hard)); + Code("E1601", "duplicate operation", () => Codec.InvokeFrame(a, spec, Array.Empty(), new[] { new OperationEntry(a, b), new OperationEntry(a, b) }, Hard)); + Code("E1601", "argument count", () => Codec.InvokeFrame(a, spec, new Value[] { new IntValue(1) }, Array.Empty(), Hard)); + JsonObject frame = Codec.InvokeFrame(a, spec, Array.Empty(), new[] { new OperationEntry(a, b) }, Hard); + Assert.That( + JsonCodec.Stringify(frame["capabilities"], 64), + Is.EqualTo("{\"effects\":[\"" + a + "\"],\"operations\":[{\"effect\":\"" + a + "\",\"mode\":\"once\",\"operation\":\"" + b + "\"}]}")); + } + + [Test] + public void JsStringMatchesJavascriptCoercion() + { + Assert.That(Replayer.JsString(Json.Of("x")), Is.EqualTo("x")); + Assert.That(Replayer.JsString(Json.Null), Is.EqualTo("null")); + Assert.That(Replayer.JsString(Json.Of(5)), Is.EqualTo("5")); + Assert.That(Replayer.JsString(Json.Of(true)), Is.EqualTo("true")); + Assert.That(Replayer.JsString(Parse("[1,null,[\"a\",\"b\"]]")), Is.EqualTo("1,,a,b")); + Assert.That(Replayer.JsString(Parse("{\"a\":1}")), Is.EqualTo("[object Object]")); + } + + [Test] + public void PinnedKitLoads() + { + Assert.That(Pinned.Transcripts, Has.Count.EqualTo(Kit.TranscriptCount)); + Assert.That(Pinned.Identities.ContainsKey("kit-echo"), Is.True); + Assert.That(Kit.File("kit/fixtures.jac"), Is.Not.Null); + Assert.That(Kit.File("kit/README.md"), Is.Not.Null); + Assert.That(Kit.File("../NOTICE"), Is.Null); + } + + [Test] + public void OutcomeEvidenceAndResultShareOneFrameBudget() + { + JsonObject outcome = Pinned.Transcripts[0].CoreOutput[1]; + Codec.ParseCoreFrame(outcome, Hard.With("max_value_nodes", 2)); + Code("E1602", "pure outcome", () => Codec.ParseCoreFrame(outcome, Hard.With("max_value_nodes", 1))); + + // An error outcome spends its two evidence types only. + IReadOnlyList refused = Pinned.Find("refused-authority")!.CoreOutput; + JsonObject errorOutcome = refused[^1]; + Codec.ParseCoreFrame(errorOutcome, Hard.With("max_value_nodes", 2)); + Code("E1602", "error outcome", () => Codec.ParseCoreFrame(errorOutcome, Hard.With("max_value_nodes", 1))); + } + + [Test] + public void SessionEnforcesOneInvocationAndLocalTerminalRefusal() + { + Transcript pure = Pinned.Transcripts[0]; + var session = new Session(); + session.Receive(pure.CoreOutput[0]); + session.Select(Hard); + session.Invoke(Pinned.Identity("kit-ready"), Unit(), Array.Empty(), Array.Empty()); + Code("E1608", "second invoke", () => session.Invoke(HashOf('a'), Unit(), Array.Empty(), Array.Empty())); + session.Receive(pure.CoreOutput[1]); + Code("E1608", "shutdown after terminal", () => session.Shutdown()); + Code("E1608", "frame after terminal", () => session.Receive(pure.CoreOutput[1])); + } + + [Test] + public void RequestOrderingAndOneResponseSlot() + { + Transcript transcript = Pinned.Find("one-once-effect-success")!; + var session = new Session(); + session.Receive(transcript.CoreOutput[0]); + session.Select(Hard); + JsonObject invoke = transcript.HostInput.Select(step => step.Frame).First(frame => Json.Of("invoke").Equals(frame.Get("kind"))); + JsonObject wire = (JsonObject)invoke["interface"]; + var spec = new CallInterface( + wire["parameters"].AsArray()!.Select(parameter => Codec.DecodeType(parameter, NewBudget())), + Codec.DecodeType(wire["result"], NewBudget()), + wire["effects"].AsArray()!.Select(effect => Hash.Parse(effect.AsString()))); + OperationEntry[] operations = ((JsonObject)invoke["capabilities"])["operations"].AsArray()! + .Select(entry => new OperationEntry(Hash.Parse(((JsonObject)entry)["effect"].AsString()), Hash.Parse(((JsonObject)entry)["operation"].AsString()))) + .ToArray(); + Hash callable = Hash.Parse(((JsonObject)invoke["target"])["callable"].AsString()); + session.Invoke(callable, spec, new Value[] { new TextValue("ping") }, operations); + JsonObject request = transcript.CoreOutput[1]; + Code("E1601", "numeric invocation id", () => session.Receive(request.With("invocation_id", Json.Of(0)))); + Code("E1608", "other invocation", () => session.Receive(request.With("invocation_id", Json.Of("ffffffffffffffff")))); + Code("E1608", "skipped ordinal", () => session.Receive(request.With("request_id", Json.Of("0000000000000002")))); + Code("E1608", "ordinal beyond the exact range", () => session.Receive(request.With("request_id", Json.Of("ffffffffffffffff")))); + session.Receive(request); + Code("E1608", "second request while waiting", () => session.Receive(request)); + session.RespondOk(new TextValue("pong")); + Code("E1608", "second response", () => session.RespondOk(new TextValue("again"))); + } + + [Test] + public void TheEncoderStopsAtTheFrameCeiling() + { + JsonObject frame = Json.Object(("k", Json.Of(new string('\u0001', 1000)))); + ProtocolException refused = Code("E1602", "one byte short", () => JsonCodec.FrameBytes(frame, 6007, 64)); + Assert.That(refused.Detail, Is.EqualTo("payload exceeds max_frame_bytes")); + + // {"k":"..."} is 8 bytes plus 6 per escaped U+0001: exactly 6008. + Assert.That(JsonCodec.FrameBytes(frame, 6008, 64), Has.Length.EqualTo(4 + 6008)); + Assert.That(JsonCodec.Stringify(Json.Of("\u0001\u001f"), 64), Is.EqualTo("\"\\u0001\\u001f\"")); + + // A text that cannot fit is refused by length, before the scan that + // would find its lone surrogate. + refused = Code("E1602", "long text", () => JsonCodec.FrameBytes(Json.Object(("k", Json.Of("\ud800" + new string('x', 7000)))), 6008, 64)); + Assert.That(refused.Detail, Is.EqualTo("payload exceeds max_frame_bytes")); + } + + /// + /// 256 references to one text at the text ceiling: within the node, + /// collection, and text limits, but over 384 MiB once escaped. + /// + internal static TupleValue HugeResult() => new(Enumerable.Repeat(new TextValue(new string('\u0001', 262_144)), 256)); + + [Test] + public void EncodingAFrameFarOverTheCeilingStopsAtTheCeiling() + { + // The encoded value as JSON, built without the frame budget. + JsonObject frame = Json.Object(("value", Json.Of(HugeResult().Items.Select(item => (Json)Json.Of(((TextValue)item).Value))))); + long before = GC.GetAllocatedBytesForCurrentThread(); + ProtocolException refused = Code("E1602", "huge frame", () => JsonCodec.FrameBytes(frame, Hard.MaxFrameBytes, Hard.MaxJsonDepth)); + long allocated = GC.GetAllocatedBytesForCurrentThread() - before; + Assert.That(refused.Detail, Is.EqualTo("payload exceeds max_frame_bytes")); + + // The sink grows to at most the 1 MiB ceiling (by doubling, so about + // 2 MiB in all); encoding everything first would allocate over 384 MiB. + Assert.That(allocated, Is.LessThan(8L << 20), "allocated " + allocated + " bytes"); + } + + [Test] + public void ValidatingManyReferencesToALongTextChargesTheFrame() + { + // 4 x 1022 references to one text of 65536 emoji (262144 UTF-8 + // bytes): within the node, collection, and text limits, and about + // 1 GiB of text to scan if each reference were checked in full. + // Validation charges every text against the frame and stops. + var text = new TextValue(string.Concat(Enumerable.Repeat("😀", 65_536))); + var value = new TupleValue(Enumerable.Repeat(new TupleValue(Enumerable.Repeat(text, 1022)), 4)); + ProtocolException refused = Code("E1602", "many references", () => Codec.EncodeValue(value, new Budget(Hard))); + Assert.That(refused.Detail, Is.EqualTo("payload exceeds max_frame_bytes")); + } + + [Test] + public void ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned() + { + // The lone surrogate at the end would be E1601 if the text were + // scanned; its length alone refuses it first. + string text = new string('é', 32 << 20) + "\ud800"; + ProtocolException refused = Code("E1602", "huge text", () => Codec.EncodeValue(new TextValue(text), new Budget(Hard))); + Assert.That(refused.Detail, Is.EqualTo("Text exceeds max_text_bytes")); + refused = Code("E1602", "host message", () => NewSessionWaiting().RespondFailure(FailureCategory.OutsideFailure, text)); + Assert.That(refused.Detail, Is.EqualTo("host message exceeds max_host_message_bytes")); + } + + private static Session NewSessionWaiting() + { + Transcript transcript = Pinned.Find("one-once-effect-success")!; + var session = new Session(); + session.Receive(transcript.CoreOutput[0]); + session.Select(Hard); + session.Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, new[] { new OperationEntry(Pinned.Identity("kit-world"), Pinned.Identity("send")) }); + session.Receive(transcript.CoreOutput[1]); + return session; + } + + /// A caller's map that counts how far it was enumerated. + private sealed class CountingMap : IReadOnlyDictionary + { + private readonly int count; + private readonly int keyLength; + + internal CountingMap(int count, int keyLength) + { + this.count = count; + this.keyLength = keyLength; + } + + internal int Enumerated { get; private set; } + + public int Count => count; + + public IEnumerable Keys => this.Select(field => field.Key); + + public IEnumerable Values => this.Select(field => field.Value); + + public Json this[string key] => throw new KeyNotFoundException(); + + public bool ContainsKey(string key) => false; + + public bool TryGetValue(string key, out Json value) + { + value = Json.Null; + return false; + } + + public IEnumerator> GetEnumerator() + { + string prefix = new('k', keyLength); + for (int i = 0; i < count; i++) + { + Enumerated++; + yield return new KeyValuePair(prefix + i.ToString(System.Globalization.CultureInfo.InvariantCulture), Json.Null); + } + } + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + } + + [Test] + public void ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted() + { + // 60,000 keys of 2,048 characters: far more than a frame can hold. + // The frame is refused after the keys that already pass the ceiling, + // without copying or sorting the rest. + var map = new CountingMap(60_000, 2048); + ProtocolException refused = Code("E1602", "long keys", () => JsonCodec.FrameBytes(map, Hard.MaxFrameBytes, Hard.MaxJsonDepth)); + Assert.That(refused.Detail, Is.EqualTo("payload exceeds max_frame_bytes")); + Assert.That(map.Enumerated, Is.LessThanOrEqualTo((Hard.MaxFrameBytes / 2048) + 1)); + + // A count that cannot fit is refused before any key is read. + var many = new CountingMap(Hard.MaxFrameBytes, 0); + Code("E1602", "many keys", () => JsonCodec.FrameBytes(many, Hard.MaxFrameBytes, Hard.MaxJsonDepth)); + Assert.That(many.Enumerated, Is.EqualTo(0)); + + // A small map is encoded in sorted order. + Assert.That( + JsonCodec.FrameBytes(new Dictionary { ["b"] = Json.Of(1), ["a"] = Json.Of(2) }, 64, 64), + Is.EqualTo(Encoding.UTF8.GetBytes("\0\0\0\u000d{\"a\":2,\"b\":1}"))); + } + + [Test] + public void BoundedMessagesCutAtACodePoint() + { + Assert.That(Host.Bounded("\ud800" + string.Concat(Enumerable.Repeat("é", 3000)), 4096), Is.EqualTo("\ufffd" + string.Concat(Enumerable.Repeat("é", 2046)))); + Assert.That(Host.Bounded("😀", 3), Is.Empty); + Assert.That(Host.Bounded("a😀", 5), Is.EqualTo("a😀")); + } +} diff --git a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs new file mode 100644 index 0000000..9a561d5 --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs @@ -0,0 +1,183 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Globalization; +using System.IO; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// +/// Open descriptors, adapter threads, and the heap across invocations and +/// failures, read from /proc. The adapter starts no thread of its own +/// except one per handler call, named jacquard-handler. +/// +[TestFixture] +[Platform("Linux")] +public sealed class DescriptorTests +{ + private static bool DescriptorsBackTo(int before) => Eventually(() => Descriptors() <= before); + + [Test] + public void InvocationsReleaseTheirPipesAndThreads() + { + Assert.That(InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))).Ok, Is.True); // warm up lazily opened files + int before = Descriptors(); + Assert.That(AdapterThreads(), Is.EqualTo(0)); + for (int i = 0; i < 5; i++) + { + InvocationResult result = InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))); + Assert.That(result.Ok, Is.True, result.ToString()); + } + + Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); + Assert.That(AdapterThreads(), Is.EqualTo(0), "adapter threads outlived their invocations"); + + // A daemonized descendant keeps holding stderr after the invocation + // returns; the host does not wait for it, and its pipes are closed. + InvocationResult escaped = InvokeReady(Faulty("escaped-descendant", TimeSpan.FromSeconds(10))); + int pid = Descendant("escaped-descendant", escaped.Stderr); + bool released = DescriptorsBackTo(before); + int threads = AdapterThreads(); + Kill(pid); + Assert.That(released, Is.True, "escaped descendant: descriptors grew from " + before + " to " + Descriptors()); + Assert.That(threads, Is.EqualTo(0), "escaped descendant: adapter threads remain"); + + // Handlers run on their own threads, which end with the handler; the + // count sees a handler thread while one runs. + int during = -1; + OperationRegistry registry = new OperationRegistry().Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), (context, args) => + { + during = AdapterThreads(); + return new TextValue("pong"); + }); + Host replaying = Replaying("one-once-effect-success"); + for (int i = 0; i < 3; i++) + { + InvocationResult result = replaying.Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, registry); + Assert.That(result.Ok, Is.True, result.ToString()); + } + + Assert.That(during, Is.EqualTo(1), "the handler thread was not seen while it ran"); + Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "handler threads remain"); + Assert.That(DescriptorsBackTo(before), Is.True, "handlers: descriptors grew from " + before + " to " + Descriptors()); + + // Failure paths release the same resources: expiry, cancellation, a + // truncated frame, and a spawn failure. + for (int i = 0; i < 3; i++) + { + Assert.That(Reason(InvokeReady(Faulty("silent", TimeSpan.FromMilliseconds(500)))), Is.EqualTo("deadline")); + using var cancelled = new CancellationTokenSource(); + After(TimeSpan.FromMilliseconds(200), cancelled.Cancel); + Assert.That(Reason(InvokeReady(Faulty("silent", TimeSpan.FromSeconds(10)), cancelled.Token)), Is.EqualTo("aborted")); + Assert.That(InvokeReady(Faulty("truncated", TimeSpan.FromSeconds(10))).Ok, Is.False); + Assert.That(Reason(InvokeReady(new Host("unused", new HostOptions { Carrier = new WorkerOptions { Command = new[] { "/no/such/executable" } } }))), Is.EqualTo("worker_spawn_failed")); + } + + Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "failure paths: adapter threads remain"); + Assert.That(DescriptorsBackTo(before), Is.True, "failure paths: descriptors grew from " + before + " to " + Descriptors()); + } + + [Test] + public void AnEscapedDescendantHoldingStdinLeavesNoThreadOrDescriptor() + { + // The worker exits after the selection; its reparented child holds + // stdin and never reads, so the invoke frame cannot be written. The + // call ends at the deadline with the descendant still alive, and + // nothing of the host's stays behind with it. + Assert.That(InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))).Ok, Is.True); // warm up + int before = Descriptors(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => Faulty("stdin-descendant", TimeSpan.FromSeconds(2)) + .Invoke(Pinned.Identity("kit-ready"), Echo(withWorld: false), new Value[] { new TextValue(new string('x', 220_000)) }, new OperationRegistry())); + int pid = Descendant("stdin-descendant", result.Stderr); + try + { + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(Alive(pid), Is.True); + Assert.That(AdapterThreads(), Is.EqualTo(0)); + Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); + } + finally + { + Kill(pid); + } + } + + [Test] + public void AStderrFloodAfterTheWorkerExitsDoesNotHoldTheCaller() => + AssertFloodDoesNotHold("(yes jacquard-flood >&2 & echo $! > \"$PIDFILE\"); sleep 0.3; exit 0", TimeSpan.FromSeconds(10), null); + + [Test] + public void AStderrFloodDoesNotHoldExpiry() => + AssertFloodDoesNotHold("(yes jacquard-flood >&2 & echo $! > \"$PIDFILE\"); exec sleep 30", TimeSpan.FromMilliseconds(500), null); + + [Test] + public void AStderrFloodDoesNotHoldCancellation() => + AssertFloodDoesNotHold("(yes jacquard-flood >&2 & echo $! > \"$PIDFILE\"); exec sleep 30", TimeSpan.FromSeconds(20), TimeSpan.FromMilliseconds(500)); + + [Test] + public void AStdoutFloodDoesNotHoldTheCaller() => + AssertFloodDoesNotHold("(yes jacquard-flood & echo $! > \"$PIDFILE\"); exec sleep 30", TimeSpan.FromSeconds(10), null); + + [Test] + public void ASilentDescendantHoldingStdoutAndStderrDoesNotHoldTheCaller() => + AssertFloodDoesNotHold("(sleep 30 & echo $! > \"$PIDFILE\"); exit 0", TimeSpan.FromSeconds(10), null); + + /// + /// The worker leaves a reparented descendant writing to its stdout or + /// stderr as fast as it can (or holding them silently); the kill cannot + /// find it. The invocation must return within its bound, keep no flood + /// in the heap, and leave the writer without a reader once it returns. + /// + private static void AssertFloodDoesNotHold(string script, TimeSpan deadline, TimeSpan? cancelAfter) + { + Assert.That(InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))).Ok, Is.True); // warm up + int before = Descriptors(); + string dir = Directory.CreateTempSubdirectory("jacquard-flood-").FullName; + string pidFile = Path.Combine(dir, "pid"); + var env = new Dictionary + { + ["PATH"] = Environment.GetEnvironmentVariable("PATH") ?? "/usr/bin:/bin", + ["PIDFILE"] = pidFile, + }; + var options = new WorkerOptions { Command = new[] { "sh", "-c", script }, Environment = env, Deadline = deadline }; + long heapBefore = GC.GetTotalMemory(forceFullCollection: true); + using var cancel = new CancellationTokenSource(); + if (cancelAfter is { } delay) + { + After(delay, cancel.Cancel); + } + + var watch = Stopwatch.StartNew(); + InvocationResult result = Within(TimeSpan.FromSeconds(30), () => new Host("unused", new HostOptions { Carrier = options }).ShutdownProbe(cancel.Token)); + TimeSpan elapsed = watch.Elapsed; + long heapAfter = GC.GetTotalMemory(forceFullCollection: false); + Assert.That(Eventually(() => File.Exists(pidFile) && File.ReadAllText(pidFile).Trim().Length > 0), Is.True, "the descendant never started"); + int pid = int.Parse(File.ReadAllText(pidFile).Trim(), CultureInfo.InvariantCulture); + try + { + Assert.That(result.Ok, Is.False, result.ToString()); + TimeSpan bound = (cancelAfter ?? (deadline < TimeSpan.FromSeconds(1) ? deadline : TimeSpan.FromSeconds(1))) + TimeSpan.FromSeconds(3); + Assert.That(elapsed, Is.LessThan(bound), "returned after " + elapsed); + Assert.That(heapAfter - heapBefore, Is.LessThan(64L << 20), "the heap grew by " + ((heapAfter - heapBefore) >> 20) + " MiB"); + if (script.Contains("yes", StringComparison.Ordinal)) + { + // The host closed its end, so the writer's next write fails. + Assert.That(Eventually(() => !Alive(pid)), Is.True, "the flood still has a reader"); + } + } + finally + { + Kill(pid); + File.Delete(pidFile); + Directory.Delete(dir); + } + + Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "adapter threads remain"); + Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); + } +} diff --git a/dotnet/test/JacquardHost.Tests/DocumentTests.cs b/dotnet/test/JacquardHost.Tests/DocumentTests.cs new file mode 100644 index 0000000..d860cf0 --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/DocumentTests.cs @@ -0,0 +1,45 @@ +using System; +using System.IO; +using System.Linq; +using System.Reflection; +using NUnit.Framework; + +namespace JacquardLang.Host.Tests; + +/// Files that must stay identical to their sources. +[TestFixture] +public sealed class DocumentTests +{ + /// The dotnet/ directory this test assembly was built from. + private static string DotnetRoot() => + typeof(DocumentTests).Assembly.GetCustomAttributes().Single(attribute => attribute.Key == "DotnetRoot").Value!; + + [Test] + public void TheReadmeExampleIsTheCompiledOne() + { + // The README's example is Readme/Echo.cs, which a live test runs + // against Core, minus its namespace line. + string readme = File.ReadAllText(Path.Combine(DotnetRoot(), "README.md")); + int start = readme.IndexOf("```csharp\n", StringComparison.Ordinal); + Assert.That(start, Is.GreaterThanOrEqualTo(0), "README.md has no csharp example"); + int end = readme.IndexOf("```", start + 10, StringComparison.Ordinal); + string example = readme[(start + 10)..end]; + string source = File.ReadAllText(Path.Combine(DotnetRoot(), "test", "JacquardHost.Tests", "Readme", "Echo.cs")); + string body = source[(source.IndexOf('\n', source.IndexOf("namespace ", StringComparison.Ordinal)) + 1)..].TrimStart(); + Assert.That(example, Is.EqualTo(body), "README.md's example differs from test/JacquardHost.Tests/Readme/Echo.cs"); + } + + [Test] + public void TheNoticeMatchesTheRepositoryNotice() + { + // A package or source copy of dotnet/ carries dotnet/NOTICE but not + // the repository's root NOTICE, so the two must stay identical. + string repository = Path.Combine(DotnetRoot(), "..", "NOTICE"); + if (!File.Exists(repository)) + { + Assert.Ignore("outside a checkout"); + } + + Assert.That(File.ReadAllBytes(Path.Combine(DotnetRoot(), "NOTICE")), Is.EqualTo(File.ReadAllBytes(repository)), "dotnet/NOTICE differs from the repository NOTICE; copy it again"); + } +} diff --git a/dotnet/test/JacquardHost.Tests/HostTests.cs b/dotnet/test/JacquardHost.Tests/HostTests.cs new file mode 100644 index 0000000..435f914 --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/HostTests.cs @@ -0,0 +1,429 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.IO; +using System.Linq; +using System.Text; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Conformance; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// Kit replay through the .NET carrier, and host fault handling against the stand-in worker. +[TestFixture] +public sealed class HostTests +{ + private static readonly Limits Hard = Limits.Hard; + + [TearDown] + public void ClearSeams() + { + Host.RequestDecodedHook = null; + Host.HandlerThreadStartedHook = null; + Host.HandlerCheckHook = null; + } + + [Test] + public void NullArgumentsAreRefusedBeforeAWorkerStarts() + { + Host host = Faulty("ok", TimeSpan.FromSeconds(10)); + Throws(() => host.Invoke(null!, Unit(), Array.Empty(), new OperationRegistry())); + Throws(() => host.Invoke(Pinned.Identity("kit-ready"), Unit(), Array.Empty(), null!)); + Throws(() => _ = new Host(null!)); + } + + private static Replay ReplayStandin(Transcript transcript, string original) => + Replayer.ReplayOne(transcript, "unused", Standin("replay", original, TimeSpan.FromSeconds(20))); + + [Test] + public void AllRecordingsReplayThroughTheDotnetCarrier() + { + foreach (Transcript transcript in Pinned.Transcripts) + { + Replay replay = ReplayStandin(transcript, transcript.Name); + Assert.That( + replay.Differences.Count == 0 && replay.ProseDrift.Count == 0 && replay.Status == transcript.Status, + Is.True, + transcript.Name + ": differences " + string.Join("; ", replay.Differences) + ", prose drift " + string.Join("; ", replay.ProseDrift) + ", status " + replay.Status); + } + } + + [Test] + public void ReplayTreatsAbsentExpectationsLikeTheReference() + { + Transcript missing = Pinned.Find("pure-success")! with { Expect = JsonObject.Empty }; + Assert.That(ReplayStandin(missing, "pure-success").Status, Is.EqualTo("diverges"), "absent terminal expectation conformed"); + Transcript unbounded = Pinned.Find("unknown-version")! with { Expect = Pinned.Find("unknown-version")!.Expect.Without("terminal_frames_max") }; + Assert.That(ReplayStandin(unbounded, "unknown-version").Status, Is.EqualTo("diverges"), "missing terminal_frames_max conformed"); + } + + private static Json SummaryOf(JsonObject frame) => ((JsonObject)((JsonObject)frame["result"])["diagnostics"].AsArray()![0])["summary"]; + + [Test] + public void ProseDriftIsReportedWithoutNormalizing() + { + Transcript original = Pinned.Find("refused-authority")!; + var output = original.CoreOutput.ToList(); + JsonObject last = output[^1]; + var result = (JsonObject)last["result"]; + var diagnostics = result["diagnostics"].AsArray()!.ToList(); + var diagnostic = (JsonObject)diagnostics[0]; + diagnostics[0] = diagnostic.With("summary", Json.Of(diagnostic["summary"].AsString() + " changed")); + output[^1] = last.With("result", result.With("diagnostics", Json.Of(diagnostics))); + Replay replay = ReplayStandin(original with { CoreOutput = output }, original.Name); + Assert.That(replay.Differences, Is.Empty); + Assert.That(replay.ProseDrift, Has.Count.EqualTo(1)); + Assert.That(SummaryOf(replay.CoreFrames[^1]), Is.EqualTo(SummaryOf(original.CoreOutput[^1]))); + } + + [Test] + public void ReplayStopsReadingAWorkerThatSendsPastTheRecording() + { + // 300 frames of about 600 KB follow the terminal. Replay keeps one + // frame past the recording's and stops reading; disposal kills the worker. + Transcript pure = Pinned.Transcripts[0]; + Replay replay = Within(TimeSpan.FromSeconds(30), () => Replayer.ReplayOne(pure, "unused", Standin("flood-after-terminal", string.Empty, TimeSpan.FromSeconds(10)))); + Assert.That(replay.MatchesRecording, Is.False); + Assert.That(replay.Differences, Does.Contain("Core sent more frames than the recording")); + Assert.That(replay.CoreFrames, Has.Count.EqualTo(pure.CoreOutput.Count + 1)); + } + + [Test] + public void MalformedExtraAndNoncanonicalCoreFramesFailClosed() + { + foreach ((string mode, string code) in new[] { ("unknown-field", "E1601"), ("invalid-hash", "E1601"), ("extra", "E1608") }) + { + InvocationResult result = InvokeReady(Faulty(mode, TimeSpan.FromSeconds(10))); + Assert.That(result.Kind == ResultKind.ProtocolError && result.Code == code, Is.True, mode + ": " + result); + } + } + + [Test] + public void CarrierFailuresRetainHostEvidence() + { + foreach (string mode in new[] { "exit", "truncated", "silent", "stall", "nonzero" }) + { + bool expired = mode is "silent" or "stall"; + InvocationResult result = InvokeReady(Faulty(mode, TimeSpan.FromSeconds(expired ? 2 : 10))); + Assert.That( + result.Kind == ResultKind.CarrierLoss && result.Code == "E1611" && Json.Of(Host.EvidenceSchema).Equals(result.HostEvidence.Get("schema")), + Is.True, + mode + ": " + result.HostEvidence); + bool sawOutcome = result.CoreFrames.Any(frame => Json.Of("outcome").Equals(frame.Get("kind"))); + Assert.That(sawOutcome, Is.EqualTo(mode is "stall" or "nonzero"), mode + ": outcome observed"); + Assert.That(Reason(result) == "deadline", Is.EqualTo(expired), mode + ": " + Reason(result)); + InvocationException error = Throws(() => result.ValueOrThrow())!; + Assert.That(error.Failure, Is.EqualTo(expired ? Failure.DeadlineExceeded : Failure.WorkerLost), mode); + } + } + + [Test] + public void FramesAfterSelectionAreReadUnderTheSelectedLimits() + { + InvocationResult result = InvokeReady(Faulty("oversized-early", TimeSpan.FromSeconds(10), Hard.With("max_frame_bytes", 1024))); + Assert.That(result.Kind == ResultKind.ProtocolError && result.Code == "E1602", Is.True, result.ToString()); + } + + [Test] + public void StderrIsDrainedAndRetentionObeysTheSelectedCeiling() + { + // The stand-in writes 2 MiB of stderr before its hello; the host + // drains it while it waits, and keeps the selected prefix. + InvocationResult result = InvokeReady(Faulty("stderr", TimeSpan.FromSeconds(10), Hard.With("max_stderr_bytes", 17))); + Assert.That(result.Ok, Is.True, result.ToString()); + Assert.That(Encoding.ASCII.GetString(result.Stderr.ToArray()), Is.EqualTo(new string('x', 17))); + } + + [Test] + public void AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline() + { + // The invoke frame (about 220 KB) is larger than a pipe holds, and the + // worker never reads it. A blocking write would never return. + var watch = Stopwatch.StartNew(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => Faulty("no-read", TimeSpan.FromSeconds(2)) + .Invoke(Pinned.Identity("kit-ready"), Echo(withWorld: false), new Value[] { new TextValue(new string('x', 220_000)) }, new OperationRegistry())); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(1), "the invoke frame was not the write in progress"); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + Assert.That(Throws(() => result.ValueOrThrow())!.Failure, Is.EqualTo(Failure.DeadlineExceeded)); + } + + [Test] + public void CancellationAbortsWithoutRetry() + { + using var cancel = new CancellationTokenSource(); + After(TimeSpan.FromMilliseconds(300), cancel.Cancel); + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeReady(Faulty("silent", TimeSpan.FromSeconds(20)), cancel.Token); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + Assert.That(Throws(() => result.ValueOrThrow())!.Failure, Is.EqualTo(Failure.WorkerLost)); + } + + [Test] + public void AStopBeforeTheSpawnStartsNoWorker() + { + using var cancel = new CancellationTokenSource(); + cancel.Cancel(); + InvocationResult result = InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10)), cancel.Token); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(result.CoreFrames, Is.Empty); + Assert.That(result.ExitStatus, Is.Null); + } + + [Test] + public void LocalRefusalStartsNoWorkerAndInvalidOptionsAreRejected() + { + var missing = new WorkerOptions { Command = new[] { "/no/such/executable" } }; + InvocationResult result = InvokeReady(new Host("unused", new HostOptions { Carrier = missing, Limits = Hard.With("max_frame_bytes", 20) })); + Assert.That(result.Kind == ResultKind.HostError && result.Code == "E1602" && result.ExitStatus is null, Is.True, result.ToString()); + Assert.That(Reason(result), Is.EqualTo("invoke_refused_locally")); + InvocationResult failed = InvokeReady(new Host("unused", new HostOptions { Carrier = missing })); + Assert.That(Reason(failed), Is.EqualTo("worker_spawn_failed"), failed.ToString()); + Assert.That(failed.Message, Does.Contain("/no/such/executable")); + foreach (HostOptions options in new[] + { + new HostOptions { Carrier = new WorkerOptions { Deadline = TimeSpan.Zero } }, + new HostOptions { Carrier = new WorkerOptions { Deadline = TimeSpan.FromSeconds(-1) } }, + new HostOptions { Carrier = new WorkerOptions { Command = new[] { string.Empty } } }, + new HostOptions { Limits = Hard.With("max_arguments", 0) }, + }) + { + Throws(() => _ = new Host("unused", options), "accepted " + options); + } + } + + [Test] + public void RegistryRefusesDuplicatesAndSortsEntries() + { + Hash a = HashOf('a'); + Hash b = HashOf('b'); + var registry = new OperationRegistry(); + registry.Bind(b, a, (context, args) => Value.Unit); + registry.Bind(a, b, (context, args) => Value.Unit); + Throws(() => registry.Bind(a, b, (context, args) => Value.Unit)); + IReadOnlyList entries = registry.Entries(); + Assert.That(entries[0].Effect, Is.EqualTo(a)); + Assert.That(entries[1].Effect, Is.EqualTo(b)); + } + + private static OperationRegistry Registry(Handler handler) => new OperationRegistry().Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), handler); + + private static InvocationResult InvokeEcho(Host host, Handler handler, CancellationToken cancellationToken = default) => + host.Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, Registry(handler), cancellationToken); + + /// Runs one handler against the stand-in replaying one-once-effect-success; returns the host's response. + private static JsonObject Response(Handler handler) + { + InvocationResult result = InvokeEcho(Replaying("one-once-effect-success"), handler); + + // The stand-in replays its recorded outcome whatever the host answers. + Assert.That(result.Ok, Is.True, result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(3)); + return result.HostFrames[2]; + } + + private static void AssertFailure(JsonObject frame, FailureCategory category, string message) + { + Assert.That(frame.Get("kind"), Is.EqualTo(Json.Of("effect_failure")), frame.ToString()); + Assert.That(frame.Get("category"), Is.EqualTo(Json.Of(category.WireName())), frame.ToString()); + Assert.That(frame.Get("completion"), Is.EqualTo(Json.Of(category.Completion())), frame.ToString()); + Assert.That(frame.Get("message"), Is.EqualTo(Json.Of(message)), frame.ToString()); + } + + [Test] + public void HandlerAnswersMapToTheirFailureCategories() + { + IReadOnlyList? seen = null; + JsonObject ok = Response((context, args) => + { + seen = args; + return new TextValue("pong"); + }); + Assert.That(seen, Is.EqualTo(new Value[] { new TextValue("ping") })); + Assert.That(ok.Get("kind"), Is.EqualTo(Json.Of("effect_ok"))); + AssertFailure(Response((context, args) => throw HandlerException.Refused("no authority")), FailureCategory.RefusedAuthority, "no authority"); + AssertFailure(Response((context, args) => throw new InvalidOperationException("private detail")), FailureCategory.CompletionUnknown, "handler threw an exception"); + AssertFailure(Response((context, args) => throw new IOException("private detail")), FailureCategory.CompletionUnknown, "handler threw an exception"); + AssertFailure(Response((context, args) => null!), FailureCategory.OutsideFailure, "handler result has no encoding within the selected limits"); + AssertFailure(Response((context, args) => new IntValue(long.MaxValue)), FailureCategory.OutsideFailure, "handler result has no encoding within the selected limits"); + + // Messages are cut to max_host_message_bytes at a code point, and a lone + // surrogate becomes U+FFFD. + AssertFailure( + Response((context, args) => throw HandlerException.OutsideFailed("\ud800" + string.Concat(Enumerable.Repeat("é", 3000)))), + FailureCategory.OutsideFailure, + "�" + string.Concat(Enumerable.Repeat("é", 2046))); + } + + [Test] + public void AResultFarOverTheFrameCeilingIsAnOutsideFailure() + { + var watch = Stopwatch.StartNew(); + JsonObject frame = Response((context, args) => CodecTests.HugeResult()); + AssertFailure(frame, FailureCategory.OutsideFailure, "handler result has no encoding within the selected limits"); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + } + + [Test] + public void ATextFarOverTheTextCeilingIsAnOutsideFailure() + { + var text = new TextValue(new string('é', 32 << 20)); + AssertFailure(Response((context, args) => text), FailureCategory.OutsideFailure, "handler result has no encoding within the selected limits"); + } + + [Test] + public void ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread() + { + // The seam runs on the caller's thread once the request is decoded, + // just before the caller's own check; the cancellation lands there. + using var cancel = new CancellationTokenSource(); + bool ran = false; + int started = 0; + Host.RequestDecodedHook = cancel.Cancel; + Host.HandlerThreadStartedHook = () => Interlocked.Increment(ref started); + InvocationResult result = InvokeEcho(Replaying("one-once-effect-success"), (context, args) => + { + ran = true; + return Value.Unit; + }, cancel.Token); + Assert.That(Volatile.Read(ref started), Is.EqualTo(0), "a handler thread started after the stop"); + Assert.That(ran, Is.False, "the handler ran after the stop"); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a response was sent after the stop"); + } + + [Test] + public void ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler() + { + // The seam runs on the handler thread just before its own check. + using var cancel = new CancellationTokenSource(); + bool ran = false; + bool stopped = false; + Host.HandlerCheckHook = () => + { + stopped = true; + cancel.Cancel(); + }; + InvocationResult result = InvokeEcho(Replaying("one-once-effect-success"), (context, args) => + { + ran = true; + return Value.Unit; + }, cancel.Token); + Assert.That(Volatile.Read(ref stopped), Is.True, "the seam never ran"); + Assert.That(Volatile.Read(ref ran), Is.False, "the handler ran after the stop"); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + } + + [Test] + public void ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler() + { + // The seam holds the handler thread until the lifetime has passed. + bool ran = false; + Host host = Replaying("one-once-effect-success", TimeSpan.FromSeconds(3)); + var watch = Stopwatch.StartNew(); + Host.HandlerCheckHook = () => + { + while (watch.Elapsed < TimeSpan.FromSeconds(4)) + { + Thread.Sleep(10); + } + }; + InvocationResult result = InvokeEcho(host, (context, args) => + { + ran = true; + return Value.Unit; + }); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(Eventually(() => watch.Elapsed > TimeSpan.FromSeconds(4.5), TimeSpan.FromSeconds(6)), Is.True); + Assert.That(Volatile.Read(ref ran), Is.False, "the handler ran after the deadline"); + } + + [Test] + public void AHandlerSeesTheStopThroughItsContext() + { + // A cooperative handler waits on its token; cancellation releases it, + // and the host sends no late response. + using var cancel = new CancellationTokenSource(); + using var finished = new ManualResetEventSlim(); + bool sawCancelled = false; + InvocationResult result = InvokeEcho(Replaying("one-once-effect-success"), (context, args) => + { + cancel.CancelAfter(200); + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + sawCancelled = context.IsCancelled && context.CancellationToken.IsCancellationRequested; + finished.Set(); + return new TextValue("late"); + }, cancel.Token); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a late response was sent"); + Assert.That(finished.Wait(TimeSpan.FromSeconds(5)), Is.True, "the handler did not finish"); + Assert.That(sawCancelled, Is.True); + } + + [Test] + public void AnAbandonedHandlerDoesNotHoldTheCaller() + { + // The handler ignores cancellation for three seconds; the call ends at + // its one-second deadline, and no late response follows. + using var finished = new ManualResetEventSlim(); + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(1)), (context, args) => + { + Thread.Sleep(3000); + finished.Set(); + return new TextValue("late"); + }); + TimeSpan elapsed = watch.Elapsed; + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(elapsed, Is.LessThan(TimeSpan.FromSeconds(2.5)), "the caller waited for the handler"); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a late response was sent"); + Assert.That(finished.Wait(TimeSpan.FromSeconds(5)), Is.True); + } + + [Test] + public void ARegistryOverTheOperationCeilingIsRefusedLocally() + { + var registry = new OperationRegistry(); + for (int i = 0; i <= Hard.MaxOperations; i++) + { + registry.Bind(Pinned.Identity("kit-world"), new Hash(i.ToString("x64", System.Globalization.CultureInfo.InvariantCulture)), (context, args) => Value.Unit); + } + + InvocationResult result = Faulty("ok", TimeSpan.FromSeconds(10)).Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, registry); + Assert.That(result.Kind == ResultKind.HostError && result.Code == "E1602" && Reason(result) == "invoke_refused_locally", Is.True, result.ToString()); + } + + [Test] + public void CancellationWhileAHugeResultIsEncodedEndsTheCallPromptly() + { + using var cancel = new CancellationTokenSource(); + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeEcho(Replaying("one-once-effect-success"), (context, args) => + { + // Cancel from elsewhere once the result is being encoded. A + // bounded encoder has finished by then; an unbounded one is still + // encoding, and only then sees the cancellation. + After(TimeSpan.FromMilliseconds(100), cancel.Cancel); + return CodecTests.HugeResult(); + }, cancel.Token); + Assert.That(result.Ok || Reason(result) == "aborted", Is.True, result.ToString()); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + } + + [Test] + public void DisposalFromAnotherThreadEndsAWaitingRead() + { + Worker worker = Worker.Start("unused", Standin("silent", string.Empty, TimeSpan.FromSeconds(30)), CancellationToken.None); + After(TimeSpan.FromMilliseconds(300), worker.Dispose); + var watch = Stopwatch.StartNew(); + CarrierException error = Within(TimeSpan.FromSeconds(10), () => Throws(() => worker.ReadFrame(Hard))!); + Assert.That(error.IsAborted, Is.True, error.Message); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + Assert.That(Eventually(() => worker.Signal == "SIGKILL"), Is.True, "the worker was not killed"); + } +} diff --git a/dotnet/test/JacquardHost.Tests/JacquardHost.Tests.csproj b/dotnet/test/JacquardHost.Tests/JacquardHost.Tests.csproj new file mode 100644 index 0000000..15e13ae --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/JacquardHost.Tests.csproj @@ -0,0 +1,34 @@ + + + + + net8.0 + net8.0;net10.0 + JacquardLang.Host.Tests + JacquardLang.Host.Tests + false + true + 8.0 + + $(NoWarn);CA1707 + + + + + + + + + + + <_Parameter1>DotnetRoot + <_Parameter2>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)../..')) + + + + + + + + + diff --git a/dotnet/test/JacquardHost.Tests/LiveTests.cs b/dotnet/test/JacquardHost.Tests/LiveTests.cs new file mode 100644 index 0000000..c07f5cd --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/LiveTests.cs @@ -0,0 +1,217 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.IO; +using System.Linq; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Conformance; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// +/// Live tests against the real Core named by JACQUARD_BIN and +/// JACQUARD_PRELUDE. Each test is ignored without them, and fails +/// instead when JACQUARD_REQUIRE_LIVE is set, as CI does. +/// +[TestFixture] +[Platform("Linux,MacOsX")] +public sealed class LiveTests +{ + private string root = string.Empty; + private string store = string.Empty; + private string binary = string.Empty; + private Dictionary env = new(); + + [SetUp] + public void Prepare() + { + string? bin = Environment.GetEnvironmentVariable("JACQUARD_BIN"); + string? prelude = Environment.GetEnvironmentVariable("JACQUARD_PRELUDE"); + if (string.IsNullOrEmpty(bin) || string.IsNullOrEmpty(prelude)) + { + if (Environment.GetEnvironmentVariable("JACQUARD_REQUIRE_LIVE") is not null) + { + Assert.Fail("live tests need JACQUARD_BIN and JACQUARD_PRELUDE"); + } + + Assert.Ignore("live tests need JACQUARD_BIN and JACQUARD_PRELUDE"); + } + + binary = bin!; + root = Directory.CreateTempSubdirectory("jacquard-host-dotnet-live-").FullName; + string source = Path.Combine(root, "fixtures.jac"); + File.WriteAllBytes(source, Kit.File("kit/fixtures.jac")!); + store = Path.Combine(root, "store"); + env = Environment0(); + env["JACQUARD_PRELUDE"] = prelude!; + + // Output goes to a file, so a hung install cannot hold a pipe read. + string log = Path.Combine(root, "install.log"); + var info = new ProcessStartInfo("sh") { UseShellExecute = false }; + foreach (string argument in new[] { "-c", "exec \"$0\" run \"$1\" --store \"$2\" >\"$3\" 2>&1", binary, source, store, log }) + { + info.ArgumentList.Add(argument); + } + + info.Environment.Clear(); + foreach (KeyValuePair variable in env) + { + info.Environment[variable.Key] = variable.Value; + } + + using Process install = Process.Start(info)!; + if (!install.WaitForExit(60_000)) + { + install.Kill(entireProcessTree: true); + Assert.Fail("installing fixtures timed out"); + } + + Assert.That(install.ExitCode, Is.EqualTo(0), "installing fixtures: " + File.ReadAllText(log)); + } + + [TearDown] + public void Remove() + { + if (root.Length > 0 && Directory.Exists(root)) + { + Directory.Delete(root, recursive: true); + } + + Host.HandlerCheckHook = null; + } + + private WorkerOptions Options(TimeSpan? deadline = null) => new() { Command = new[] { binary }, Environment = env, Deadline = deadline }; + + private Host NewHost(TimeSpan? deadline = null) => new(store, new HostOptions { Carrier = Options(deadline) }); + + private static OperationRegistry Registry(Handler handler) => new OperationRegistry().Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), handler); + + private static InvocationResult InvokeEcho(Host host, OperationRegistry registry) => + host.Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, registry); + + [Test] + public void LiveCoreMatchesAllRecordings() + { + IReadOnlyList replays = Replayer.ReplayKit(store, Options(), Pinned); + foreach (Replay replay in replays) + { + Assert.That( + replay.Differences.Count == 0 && replay.ProseDrift.Count == 0, + Is.True, + replay.Name + ": " + string.Join("; ", replay.Differences) + ", prose drift " + string.Join("; ", replay.ProseDrift)); + } + + Assert.That(replays, Has.Count.EqualTo(Kit.TranscriptCount)); + } + + [Test] + public void LivePureAndEffectfulInvocationsRetainEvidence() + { + Host host = NewHost(); + InvocationResult pure = host.Invoke(Pinned.Identity("kit-ready"), Unit(), Array.Empty(), new OperationRegistry()); + Assert.That(pure.ExitStatus, Is.EqualTo(0), pure.ToString()); + Assert.That(pure.ValueOrThrow(), Is.EqualTo(Value.Unit)); + int calls = 0; + OperationRegistry registry = Registry((context, args) => + { + Interlocked.Increment(ref calls); + return args[0] is TextValue text ? new TextValue(text.Value + "!") : throw HandlerException.OutsideFailed("expected text"); + }); + InvocationResult result = InvokeEcho(host, registry); + Assert.That(result.Requests, Has.Count.EqualTo(1)); + Assert.That(result.HostFrames[2].Get("kind"), Is.EqualTo(Json.Of("effect_ok"))); + Assert.That(((JsonObject)result.Evidence!["core"])["terminal"], Is.EqualTo(Json.Of("ok"))); + Assert.That(result.ValueOrThrow(), Is.EqualTo(new TextValue("ping!"))); + Assert.That(calls, Is.EqualTo(1)); + InvocationResult probe = host.ShutdownProbe(); + Assert.That(probe.Kind, Is.EqualTo(ResultKind.ShutdownAck), probe.ToString()); + } + + [Test] + public void LiveReadmeExampleRuns() + { + // The README example takes its command from JACQUARD_BIN and inherits + // this environment, which carries JACQUARD_PRELUDE. + Assert.That( + Readme.Echo.Run(store, Pinned.Identity("kit-echo"), Pinned.Identity("text"), Pinned.Identity("kit-world"), Pinned.Identity("send")), + Is.EqualTo(new TextValue("ping!"))); + } + + [Test] + public void LiveFailurePairsAreNeverRetried() + { + foreach ((Func make, string expected) in new (Func, string)[] + { + (HandlerException.Unsupported, "E1606"), + (HandlerException.Refused, "E1607"), + (HandlerException.OutsideFailed, "E1613"), + (HandlerException.CompletionUnknown, "E1612"), + }) + { + int calls = 0; + InvocationResult result = InvokeEcho(NewHost(), Registry((context, args) => + { + Interlocked.Increment(ref calls); + throw make("synthetic failure"); + })); + Assert.That(result.Code, Is.EqualTo(expected), result.ToString()); + Assert.That(calls, Is.EqualTo(1), expected); + Assert.That(result.Diagnostics[0].Cause, Does.EndWith("synthetic failure"), expected); + } + + foreach ((string name, Handler handler) in new (string, Handler)[] + { + ("I/O exception", (context, args) => throw new IOException("private detail")), + ("invalid operation", (context, args) => throw new InvalidOperationException("private detail")), + ("argument", (context, args) => throw new ArgumentException("private detail")), + }) + { + InvocationResult result = InvokeEcho(NewHost(), Registry(handler)); + Assert.That(result.Code == "E1612" && !result.Message.Contains("private detail", StringComparison.Ordinal), Is.True, name + ": " + result); + Assert.That(Throws(() => result.ValueOrThrow())!.Failure, Is.EqualTo(Failure.OperationFailed), name); + } + } + + [Test] + public void LiveHandlerDeadlineKillsTheWorkerWithoutALateResponse() + { + foreach (bool cooperative in new[] { true, false }) + { + int calls = 0; + using var finished = new ManualResetEventSlim(); + bool cancelledSeen = false; + OperationRegistry registry = Registry((context, args) => + { + Interlocked.Increment(ref calls); + if (cooperative) + { + // Waits for cancellation. + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + } + else + { + // Ignores cancellation; the host abandons it. + Thread.Sleep(1000); + } + + cancelledSeen = context.IsCancelled; + finished.Set(); + return new TextValue("late"); + }); + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeEcho(NewHost(TimeSpan.FromMilliseconds(500)), registry); + TimeSpan elapsed = watch.Elapsed; + Assert.That( + Reason(result) == "deadline" && result.Code == "E1611" && calls == 1 && result.HostFrames.Count == 2 && elapsed < TimeSpan.FromSeconds(2), + Is.True, + "cooperative=" + cooperative + ": " + result.HostEvidence + " frames=" + result.HostFrames.Count + " after " + elapsed); + Assert.That(finished.Wait(TimeSpan.FromSeconds(3)), Is.True, "handler did not finish"); + Assert.That(cancelledSeen, Is.True, "cooperative=" + cooperative + ": handler context was not cancelled"); + Assert.That(result.Ok, Is.False); + } + } +} diff --git a/dotnet/test/JacquardHost.Tests/ProcessTreeTests.cs b/dotnet/test/JacquardHost.Tests/ProcessTreeTests.cs new file mode 100644 index 0000000..82bbd25 --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/ProcessTreeTests.cs @@ -0,0 +1,191 @@ +using System; +using System.Diagnostics; +using System.Threading; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// +/// The worker's process tree at expiry and after exit. The stand-in's +/// descendants use sh and sleep, so these run on Unix only. +/// +[TestFixture] +[Platform("Linux,MacOsX")] +public sealed class ProcessTreeTests +{ + [Test] + public void TheDeadlineStopsTheWorkersProcessTree() + { + // A child, and a grandchild under a waiting shell, hold the pipes + // while the worker stays silent. + foreach (string mode in new[] { "child-silent", "grandchild-silent" }) + { + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeReady(Faulty(mode, TimeSpan.FromSeconds(2))); + Assert.That(watch.Elapsed < TimeSpan.FromSeconds(5) && Reason(result) == "deadline", Is.True, mode + ": " + watch.Elapsed + " " + result.HostEvidence); + Dies(mode, Descendant(mode, result.Stderr)); + } + } + + [Test] + public void CancellationStopsAWorkerAndTheChildHoldingABlockedWrite() + { + // The worker stays alive and its child holds stdin without reading, + // so the invoke frame cannot be written. Expiry, or cancellation, + // returns promptly and kills both. + foreach (bool cancelled in new[] { false, true }) + { + using var cancel = new CancellationTokenSource(); + if (cancelled) + { + After(TimeSpan.FromMilliseconds(500), cancel.Cancel); + } + + var watch = Stopwatch.StartNew(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => Faulty("stdin-child-alive", TimeSpan.FromSeconds(cancelled ? 20 : 1)) + .Invoke(Pinned.Identity("kit-ready"), Echo(withWorld: false), new Value[] { new TextValue(new string('x', 220_000)) }, new OperationRegistry(), cancel.Token)); + int pid = Descendant("stdin-child-alive", result.Stderr); + try + { + Assert.That(Reason(result), Is.EqualTo(cancelled ? "aborted" : "deadline"), result.ToString()); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(4))); + Dies("stdin-child-alive", pid); + } + finally + { + Kill(pid); + } + } + } + + [Test] + public void DescendantsReparentedBeforeTheKillAreNotFound() + { + // A descendant that daemonized while the worker ran is outside the + // tree: the host still returns at the deadline, but it survives. + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeReady(Faulty("daemon-silent", TimeSpan.FromSeconds(2))); + int pid = Descendant("daemon-silent", result.Stderr); + try + { + Assert.That(watch.Elapsed < TimeSpan.FromSeconds(5) && Reason(result) == "deadline", Is.True, result.ToString()); + Assert.That(Alive(pid), Is.True, "the daemonized descendant was found after all; update LIMITS.md"); + } + finally + { + Kill(pid); + } + + // A launcher that exits at once, leaving a descendant holding stdout: + // the host sees the pipe still held after the exit and reports a loss + // after one second; the descendant survives. + watch.Restart(); + result = InvokeReady(Faulty("descendant", TimeSpan.FromSeconds(10))); + pid = Descendant("descendant", result.Stderr); + try + { + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + Assert.That(Reason(result), Is.EqualTo("carrier_loss"), result.ToString()); + Assert.That(result.Message, Does.Contain("worker descendants kept its pipes open after exit")); + Assert.That(Alive(pid), Is.True, "the reparented descendant was found after all; update LIMITS.md"); + } + finally + { + Kill(pid); + } + } + + [Test] + public void AWriteBlockedByAnEscapedDescendantEndsAtTheDeadline() + { + // The worker exits after the selection while its child, now + // reparented, holds stdin and never reads, so the invoke frame + // cannot be written. No thread is left blocked on the pipe. + var watch = Stopwatch.StartNew(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => Faulty("stdin-descendant", TimeSpan.FromSeconds(2)) + .Invoke(Pinned.Identity("kit-ready"), Echo(withWorld: false), new Value[] { new TextValue(new string('x', 220_000)) }, new OperationRegistry())); + int pid = Descendant("stdin-descendant", result.Stderr); + try + { + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + Assert.That(result.ExitStatus, Is.EqualTo(0)); + Assert.That(Alive(pid), Is.True, "the reparented descendant was found after all; update LIMITS.md"); + } + finally + { + Kill(pid); + } + } + + [Test] + public void TheExitWaitObeysTheDeadline() + { + InvocationResult ok = InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))); + Assert.That(ok.Ok && ok.ExitStatus == 0 && ok.Signal is null, Is.True, ok.ToString()); + + // The worker sends its terminal, closes stdout, and never exits. + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeReady(Faulty("hang-after-close", TimeSpan.FromSeconds(2))); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + Assert.That(result.Signal, Is.EqualTo("SIGKILL")); + Assert.That(result.ExitStatus, Is.Null); + } + + [Test] + public void ASlowCleanExitSucceeds() + { + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeReady(Faulty("slow-exit", TimeSpan.FromSeconds(10))); + Assert.That(result.Ok, Is.True, result.ToString()); + Assert.That(watch.Elapsed, Is.GreaterThanOrEqualTo(TimeSpan.FromMilliseconds(1500))); + } + + [Test] + public void CleanupAfterAnEarlyErrorIsNotHeldByAnEscapedDescendant() + { + // The worker sends a malformed hello and exits; its daemonized + // descendant keeps stderr open, which disposal does not wait for. + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeReady(Faulty("escaped-descendant-unknown-field", TimeSpan.FromSeconds(10))); + TimeSpan elapsed = watch.Elapsed; + Kill(Descendant("escaped-descendant-unknown-field", result.Stderr)); + Assert.That(result.Kind == ResultKind.ProtocolError && result.Code == "E1601", Is.True, result.ToString()); + Assert.That(elapsed, Is.LessThan(TimeSpan.FromSeconds(3))); + } + + [Test] + public void ADescendantHoldingStderrAfterTheExitIsALoss() + { + // As in the Go and Rust hosts, and unlike the Java host: once the + // worker has exited, stderr must close within one second. The + // descendant keeps running; the host cannot find it. + foreach (string mode in new[] { "stderr-descendant", "escaped-descendant" }) + { + var watch = Stopwatch.StartNew(); + InvocationResult result = InvokeReady(Faulty(mode, TimeSpan.FromSeconds(10))); + int pid = Descendant(mode, result.Stderr); + try + { + Assert.That(result.Kind == ResultKind.CarrierLoss && result.Code == "E1611" && Reason(result) == "carrier_loss", Is.True, mode + ": " + result); + Assert.That(result.Message, Does.Contain("worker descendants kept its pipes open after exit")); + Assert.That(result.ExitStatus, Is.EqualTo(0), mode); + Assert.That(result.CoreFrames, Has.Count.EqualTo(2), mode + ": the outcome was read"); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5)), mode); + Assert.That(Alive(pid), Is.True, mode + ": the descendant was stopped after all; update LIMITS.md"); + } + finally + { + Kill(pid); + } + } + + InvocationResult failed = InvokeReady(Faulty("stderr-descendant-nonzero", TimeSpan.FromSeconds(10))); + Kill(Descendant("stderr-descendant-nonzero", failed.Stderr)); + Assert.That(failed.Kind == ResultKind.CarrierLoss && failed.Code == "E1611", Is.True, failed.ToString()); + Assert.That(failed.ExitStatus, Is.EqualTo(74)); + } +} diff --git a/dotnet/test/JacquardHost.Tests/Readme/Echo.cs b/dotnet/test/JacquardHost.Tests/Readme/Echo.cs new file mode 100644 index 0000000..4ed3f9b --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/Readme/Echo.cs @@ -0,0 +1,33 @@ +namespace JacquardLang.Host.Tests.Readme; + +using System; +using JacquardLang.Host; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Protocol; + +public static class Echo +{ + public static Value Run(string store, Hash target, Hash text, Hash world, Hash send) + { + // The command defaults to $JACQUARD_BIN, then `jacquard`. + var options = new HostOptions { Carrier = new WorkerOptions { Deadline = TimeSpan.FromSeconds(5) } }; + var host = new Host(store, options); + OperationRegistry registry = new OperationRegistry().Bind(world, send, (context, args) => + { + if (context.IsCancelled) + { + throw HandlerException.CompletionUnknown("cancelled"); + } + + if (args is [TextValue input]) + { + return new TextValue(input.Value + "!"); + } + + throw HandlerException.OutsideFailed("expected one text argument"); + }); + var textType = new NominalType(text); + var spec = new CallInterface(new[] { textType }, textType, new[] { world }); + return host.Invoke(target, spec, new[] { new TextValue("ping") }, registry).ValueOrThrow(); + } +} diff --git a/dotnet/test/JacquardHost.Tests/Support.cs b/dotnet/test/JacquardHost.Tests/Support.cs new file mode 100644 index 0000000..65a012a --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/Support.cs @@ -0,0 +1,290 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Globalization; +using System.IO; +using System.Linq; +using System.Text; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Conformance; +using JacquardLang.Host.Protocol; +using NUnit.Framework; + +[assembly: LevelOfParallelism(1)] + +namespace JacquardLang.Host.Tests; + +/// Shared helpers for the tests. +internal static class Support +{ + private static readonly Lazy LoadedKit = new(Kit.Load); + + internal static Kit Pinned => LoadedKit.Value; + + /// Runs code expected to fail with a protocol refusal carrying . + internal static ProtocolException Code(string expected, string context, Action body) + { + try + { + body(); + } + catch (ProtocolException error) + { + Assert.That(error.Code, Is.EqualTo(expected), context + ": " + error.Message); + return error; + } + + Assert.Fail(context + ": expected " + expected + ", got success"); + throw new InvalidOperationException("unreachable"); + } + + /// Asserts that throws , and returns it. + internal static T Throws(Action body, string message = "") + where T : Exception => Assert.Throws(body, message)!; + + internal static Json Parse(string text) => JsonCodec.Decode(Encoding.UTF8.GetBytes(text), 64); + + internal static Hash HashOf(char c) => new(new string(c, 64)); + + internal static CallInterface Unit() => new(Array.Empty(), new TupleType(), Array.Empty()); + + internal static CallInterface Echo(bool withWorld = true) + { + var text = new NominalType(Pinned.Identity("text")); + return new CallInterface(new[] { text }, text, withWorld ? new[] { Pinned.Identity("kit-world") } : Array.Empty()); + } + + /// The dotnet executable that runs the stand-in. + private static string Muxer() + { + string? host = Environment.GetEnvironmentVariable("DOTNET_HOST_PATH"); + if (!string.IsNullOrEmpty(host) && File.Exists(host)) + { + return host; + } + + string? process = Environment.ProcessPath; + if (process is not null && Path.GetFileNameWithoutExtension(process) == "dotnet") + { + return process; + } + + string? root = Environment.GetEnvironmentVariable("DOTNET_ROOT"); + return root is not null && File.Exists(Path.Combine(root, "dotnet")) ? Path.Combine(root, "dotnet") : "dotnet"; + } + + /// The stand-in worker command: this runtime running the stand-in assembly. + internal static IReadOnlyList StandinCommand() => new[] + { + Muxer(), + "exec", + "--runtimeconfig", + Path.Combine(AppContext.BaseDirectory, "JacquardLang.Host.Tests.runtimeconfig.json"), + Path.Combine(AppContext.BaseDirectory, "JacquardLang.Host.StandIn.dll"), + }; + + internal static Dictionary Environment0() + { + var env = new Dictionary(StringComparer.Ordinal); + foreach (System.Collections.DictionaryEntry entry in Environment.GetEnvironmentVariables()) + { + env[(string)entry.Key] = (string?)entry.Value ?? string.Empty; + } + + return env; + } + + internal static WorkerOptions Standin(string mode, string transcript, TimeSpan deadline) + { + Dictionary env = Environment0(); + env[StandIn.Program.ModeVariable] = mode; + env[StandIn.Program.CaseVariable] = transcript; + + // Keep each stand-in small. + env["DOTNET_GCHeapHardLimit"] = "0x8000000"; + env["DOTNET_TieredPGO"] = "0"; + return new WorkerOptions { Command = StandinCommand(), Environment = env, Deadline = deadline }; + } + + internal static Host Faulty(string mode, TimeSpan deadline, Limits? limits = null) => + new("unused", new HostOptions { Carrier = Standin(mode, string.Empty, deadline), Limits = limits }); + + internal static Host Replaying(string transcript, TimeSpan? deadline = null) => + new("unused", new HostOptions { Carrier = Standin("replay", transcript, deadline ?? TimeSpan.FromSeconds(20)) }); + + internal static InvocationResult InvokeReady(Host host, CancellationToken cancellationToken = default) => + host.Invoke(Pinned.Identity("kit-ready"), Unit(), Array.Empty(), new OperationRegistry(), cancellationToken); + + internal static string? Reason(InvocationResult result) => result.HostEvidence.Get("reason")?.AsString(); + + /// The pid a stand-in printed on stderr. + internal static int Descendant(string mode, IReadOnlyList stderr) + { + string text = Encoding.UTF8.GetString(stderr.ToArray()).Trim(); + if (!int.TryParse(text, NumberStyles.None, CultureInfo.InvariantCulture, out int pid)) + { + Assert.Fail(mode + ": stand-in stderr " + text); + } + + return pid; + } + + /// Whether is a live process (not a zombie). + internal static bool Alive(int pid) + { + string stat = "/proc/" + pid.ToString(CultureInfo.InvariantCulture) + "/stat"; + if (Directory.Exists("/proc/self")) + { + try + { + string text = File.ReadAllText(stat); + return text[(text.LastIndexOf(')') + 2)..][0] != 'Z'; + } + catch (IOException) + { + return false; + } + catch (UnauthorizedAccessException) + { + return false; + } + } + + try + { + using Process process = Process.GetProcessById(pid); + return !process.HasExited; + } + catch (ArgumentException) + { + return false; + } + } + + internal static void Kill(int pid) + { + try + { + using Process process = Process.GetProcessById(pid); + process.Kill(); + } + catch (Exception error) when (error is ArgumentException or InvalidOperationException or System.ComponentModel.Win32Exception) + { + // Already gone. + } + } + + /// Waits up to two seconds for to exit, killing it and failing otherwise. + internal static void Dies(string mode, int pid) + { + if (!Eventually(() => !Alive(pid))) + { + Kill(pid); + Assert.Fail(mode + ": descendant survived the worker"); + } + } + + /// Waits up to (two seconds by default) for a condition. + internal static bool Eventually(Func condition, TimeSpan? limit = null) + { + var watch = Stopwatch.StartNew(); + while (!condition()) + { + if (watch.Elapsed > (limit ?? TimeSpan.FromSeconds(2))) + { + return false; + } + + Thread.Sleep(10); + } + + return true; + } + + /// + /// Runs on another thread and fails if it has not + /// returned within , so a call that hangs fails + /// the test instead of holding the run. + /// + internal static T Within(TimeSpan limit, Func body) + { + T? value = default; + Exception? failure = null; + var thread = new Thread(() => + { + try + { + value = body(); + } + catch (Exception error) + { + failure = error; + } + }) + { + IsBackground = true, + Name = "test-bounded", + }; + thread.Start(); + if (!thread.Join(limit)) + { + Assert.Fail("the call did not return within " + limit); + } + + if (failure is not null) + { + throw new InvalidOperationException("the call failed", failure); + } + + return value!; + } + + /// Open descriptors of this process (Linux). + internal static int Descriptors() => Directory.GetFiles("/proc/self/fd").Length; + + /// Live threads of this process named by the adapter (Linux, names truncated to 15 bytes). + internal static int AdapterThreads() + { + int count = 0; + foreach (string task in Directory.GetDirectories("/proc/self/task")) + { + try + { + if (File.ReadAllText(Path.Combine(task, "comm")).StartsWith("jacquard-", StringComparison.Ordinal)) + { + count++; + } + } + catch (IOException) + { + // The thread ended while it was listed. + } + } + + return count; + } + + /// Calls after on a background thread. + internal static Thread After(TimeSpan delay, Action action) + { + var thread = new Thread(() => + { + Thread.Sleep(delay); + try + { + action(); + } + catch (ObjectDisposedException) + { + // The test already finished and disposed what it would stop. + } + }) + { + IsBackground = true, + Name = "test-trigger", + }; + thread.Start(); + return thread; + } +} From 3d0c60d0475e7103a901d6c4b221bdb42f9a8ee1 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 01:22:32 -0400 Subject: [PATCH 03/33] Run the .NET adapter's tests in CI A dotnet job runs the offline tests on .NET 8 and 10, and the shared pinned-Core job runs the live tests on both with JACQUARD_REQUIRE_LIVE set, so a misconfigured job fails rather than skips. --- .github/workflows/ci.yml | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8a628ec..d8d9657 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -91,6 +91,23 @@ jobs: - name: Offline adapter and SDK tests run: mvn -B -ntp verify + dotnet: + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + dotnet: ["8.0", "10.0"] + defaults: + run: + working-directory: dotnet + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-dotnet@v4 + with: + dotnet-version: ${{ matrix.dotnet }}.x + - name: Offline tests + run: dotnet test test/JacquardHost.Tests -f net${{ matrix.dotnet }} + bootstrap: runs-on: ubuntu-latest strategy: @@ -183,3 +200,15 @@ jobs: test "$status" = 1 run --workdir "$RUNNER_TEMP/java-recovery" --deliver-pending fulfillment | grep -q '{"delivered":1}' run --workdir "$RUNNER_TEMP/java-recovery" --deliver-pending fulfillment | grep -q '{"delivered":0}' + - uses: actions/setup-dotnet@v4 + with: + dotnet-version: | + 8.0.x + 10.0.x + - name: Run independent .NET adapter against Core + run: | + dotnet test test/JacquardHost.Tests -f net8.0 + dotnet test test/JacquardHost.Tests -f net10.0 + working-directory: dotnet + env: + JACQUARD_REQUIRE_LIVE: "1" From de2cc299455851cc851badd0c06e2b17d1283e19 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 01:28:44 -0400 Subject: [PATCH 04/33] Send no handler answer once the handler's token is cancelled The handlers' token is cancelled by a timer at the deadline, and the timer and the worker's clock differ: a live run saw a cooperative handler return on its token a moment before the worker reached its deadline, and its answer was sent. After a handler returns, a cancelled token now means the call is stopping, and the host waits for the worker to report the stop instead of answering. A test seam (null outside tests) makes the timer fire 300 ms early; without the guard the test sees the answer sent. --- dotnet/src/JacquardHost/Host.cs | 22 ++++++++++++++++++--- dotnet/test/JacquardHost.Tests/HostTests.cs | 18 +++++++++++++++++ 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 417ed37..cc172a3 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -85,6 +85,12 @@ public Host(string store, HostOptions? options = null) /// internal static volatile Action? HandlerCheckHook; + /// + /// Test seam: maps the lifetime left to the delay of the handlers' token + /// timer, so a test can make the timer fire early. Null outside tests. + /// + internal static volatile Func? HandlerTimerHook; + /// /// Runs with . The invoke /// frame is validated before a worker starts. Cancelling @@ -400,6 +406,16 @@ private JsonObject Respond(EffectRequest request) } answer = RunHandler(handler, request.Arguments); + + // A cancelled handler token means the call is stopping. Its + // deadline timer may fire a moment before the worker's clock + // reaches the deadline, and a handler that returned on it must + // not have its answer sent: wait for the worker to report the + // stop (at most that moment), which throws. + if (handlerStop is { IsCancellationRequested: true }) + { + Worker.Await(static () => false); + } } CarrierException? fault = Worker.Fault(); @@ -472,10 +488,10 @@ private CancellationToken HandlerToken() handlerStop = new CancellationTokenSource(); // A timer cannot wait longer than about 24 days; IsCancelled // sees a later deadline exactly in any case. - TimeSpan remaining = Worker.Remaining; - if (remaining <= TimeSpan.FromMilliseconds(int.MaxValue)) + TimeSpan remaining = HandlerTimerHook is { } hook ? hook(Worker.Remaining) : Worker.Remaining; + if (remaining <= TimeSpan.FromMilliseconds(int.MaxValue - 1)) { - handlerStop.CancelAfter(remaining); + handlerStop.CancelAfter(TimeSpan.FromMilliseconds(Math.Ceiling(remaining.TotalMilliseconds))); } callerRegistration = cancellationToken.UnsafeRegister(static state => ((CancellationTokenSource)state!).Cancel(), handlerStop); } diff --git a/dotnet/test/JacquardHost.Tests/HostTests.cs b/dotnet/test/JacquardHost.Tests/HostTests.cs index 435f914..bd232b7 100644 --- a/dotnet/test/JacquardHost.Tests/HostTests.cs +++ b/dotnet/test/JacquardHost.Tests/HostTests.cs @@ -25,6 +25,7 @@ public void ClearSeams() Host.RequestDecodedHook = null; Host.HandlerThreadStartedHook = null; Host.HandlerCheckHook = null; + Host.HandlerTimerHook = null; } [Test] @@ -385,6 +386,23 @@ public void AnAbandonedHandlerDoesNotHoldTheCaller() Assert.That(finished.Wait(TimeSpan.FromSeconds(5)), Is.True); } + [Test] + public void ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent() + { + // The handler returns as soon as its token is cancelled at the + // deadline. The token's timer and the worker's clock differ, and the + // seam makes the timer fire 300 ms early: the answer must still not + // reach Core, and the call ends at the deadline. + Host.HandlerTimerHook = remaining => remaining - TimeSpan.FromMilliseconds(300); + InvocationResult result = InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(2)), (context, args) => + { + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + return new TextValue("late"); + }); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "an answer was sent after the handler's token was cancelled"); + } + [Test] public void ARegistryOverTheOperationCeilingIsRefusedLocally() { From 1551f25153704f6ac1e3cc275446b06d5979b1d8 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 01:35:33 -0400 Subject: [PATCH 05/33] Document the .NET adapter, its limits, and its evidence LIMITS.md records the process-tree kill, the poll-based pipe I/O and its Linux and macOS scope, cooperative handler cancellation, inherited descriptors, and the protocol-level scope. EVIDENCE.md records the offline and live results on .NET 8 and 10 against jacquard-core-0.3.0, and which test fails when each guard is removed. The README documents bounded work on the caller's thread. CONTRIBUTING, COMPATIBILITY, the root README, and the changelog name the new adapter, and the source archive carries it. --- AGENTS.md | 4 ++-- CHANGELOG.md | 16 ++++++++++++++ CONTRIBUTING.md | 41 ++++++++++++++++++++++++++++++----- MANIFEST.in | 8 +++++++ README.md | 9 +++++--- docs/COMPATIBILITY.md | 17 +++++++++------ docs/EVIDENCE.md | 19 ++++++++++++++++ docs/LIMITS.md | 6 ++++- dotnet/README.md | 23 +++++++++++++++++++- scripts/check_distribution.py | 2 +- 10 files changed, 125 insertions(+), 20 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index decc05c..6380fed 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,8 +47,8 @@ invent messages that Core has not specified. - Keep the first carrier serial, bounded, versioned, and fail-closed. - Preserve transport independence: process framing is one carrier, not the semantic definition of the boundary. -- The Python, TypeScript, Go, Rust, and Java adapters each implement the - pinned fixtures independently. Keep pin constants and kit copies generated +- The Python, TypeScript, Go, Rust, Java, and .NET adapters each implement + the pinned fixtures independently. Keep pin constants and kit copies generated by `scripts/core_pin.py`. - Add tests with behavior. Use the standard library until a reviewed need justifies a dependency. diff --git a/CHANGELOG.md b/CHANGELOG.md index cf7186e..3391481 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,22 @@ worker's output FIFOs, and `HandlerContext.of` for calling a handler outside an invocation. CI runs the SDK's offline and live tests and the four projects. +- .NET adapter: an independent project in `dotnet/` (package + `JacquardLang.Host`, targeting .NET 8 and tested on .NET 8 and 10) with no + runtime dependencies beyond the .NET base class library. It provides strict + codecs that parse integers exactly from their text, the session state + machine, a bounded process carrier for `jacquard host worker`, and a `Host` + with a closed operation registry. It carries the pinned kit as embedded + resources and replays all 21 recordings. It is protocol-level only, with no + SDK layer. .NET cannot start the worker in its own process group or + interrupt a pipe read or write, so the carrier never calls one that can + block: it waits on the C library's `poll(2)` and runs workers on Linux and + macOS only. It kills the worker's process tree with `Process.Kill(true)`, + reports a descendant that keeps the pipes open after the worker exits as a + carrier loss, and documents what it misses in `docs/LIMITS.md`. + `scripts/core_pin.py` now generates `dotnet/src/JacquardHost/Compat.cs` and + keeps the .NET kit copy in sync, and CI runs the offline tests on .NET 8 and + 10 and the live tests against the pinned Core. ## 0.2.0 — 2026-10-25 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ed91a33..ce606f1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -18,8 +18,10 @@ Contributions are licensed under [Apache 2.0](LICENSE), matching this repository `core-pin.json` names the one Core release all adapters target. Do not edit `src/jacquard_host/compat.py`, `typescript/src/compat.ts`, `go/protocol/compat.go`, `rust/src/compat.rs`, `java/src/main/java/org/jacquardlang/host/Compat.java`, -the Go, Rust, and Java kit copies under `go/kit/data/`, `rust/kit/`, and -`java/src/main/resources/org/jacquardlang/host/kit/`, or the generated block +`dotnet/src/JacquardHost/Compat.cs`, the Go, Rust, Java, and .NET kit copies +under `go/kit/data/`, `rust/kit/`, +`java/src/main/resources/org/jacquardlang/host/kit/`, and +`dotnet/src/JacquardHost/kit/`, or the generated block in `docs/COMPATIBILITY.md` by hand: change the pin with `python3 scripts/core_pin.py update --tag ...`, which also vendors the kit, and `python3 scripts/core_pin.py check` (run in CI) reports any drift. @@ -142,6 +144,33 @@ sync. The README's Java examples are compiled test sources under differ). Copy the repository `NOTICE` to `java/NOTICE` when it changes (a test fails if they differ). +## .NET + +The independent .NET adapter lives in `dotnet/` as package `JacquardLang.Host` +(`dotnet/src/JacquardHost`). It targets .NET 8, has no runtime dependencies +beyond the .NET base class library, and does not call Python, Node.js, Go, +Rust, or Java; it calls the C library's `poll(2)` and runs workers on Linux +and macOS only. NUnit, its adapter, and the test SDK are its only (test) +dependencies. The .NET 8 or 10 SDK is required. + +```sh +cd dotnet +dotnet test test/JacquardHost.Tests -f net8.0 +dotnet test test/JacquardHost.Tests -f net10.0 # with a .NET 10 SDK +# With the pinned Core configured (eval "$(python3 ../scripts/core_pin.py install)"): +JACQUARD_REQUIRE_LIVE=1 dotnet test test/JacquardHost.Tests -f net8.0 +``` + +CI runs the offline tests on .NET 8 and 10, and the live tests on both in the +shared pinned-Core job. The build treats warnings as errors, with nullable +reference types and the `8.0-recommended` analyzers. +`dotnet/src/JacquardHost/Compat.cs` is generated and +`dotnet/src/JacquardHost/kit/` is a byte copy of the pinned kit. +`scripts/core_pin.py` maintains both; do not edit them by hand. The +`JacquardHost.StandIn` program under `dotnet/test/` is a fixture for the +offline tests. Copy the repository `NOTICE` to `dotnet/NOTICE` when it changes +(a test fails if they differ). + ## Releasing A release is one commit on `main` with green CI. To cut one: @@ -149,7 +178,8 @@ A release is one commit on `main` with green CI. To cut one: 1. Open a pull request that dates the CHANGELOG entry and updates any version references in the docs. Keep the Python (`src/jacquard_host/__init__.py`), npm (`typescript/package.json`, `typescript/src/version.ts`), Rust - (`rust/Cargo.toml`), Java (`java/pom.xml`), and release versions equal. + (`rust/Cargo.toml`), Java (`java/pom.xml`), .NET + (`dotnet/Directory.Build.props`), and release versions equal. 2. After it merges, tag the merge commit twice, with annotated tags (`git tag -a`): `jacquard-host-X.Y.Z` for the repository and its assets, and `go/vX.Y.Z` @@ -162,8 +192,9 @@ A release is one commit on `main` with green CI. To cut one: 4. Publish a GitHub release for `jacquard-host-X.Y.Z` with the CHANGELOG entry as its notes and the assets attached. -Nothing is published to PyPI, npm, crates.io, or Maven Central. The Rust -crate is used from the repository tag, and the Java artifact is built from it. +Nothing is published to PyPI, npm, crates.io, Maven Central, or NuGet. The +Rust crate is used from the repository tag, and the Java artifact and the .NET +package are built from it. ## TypeScript diff --git a/MANIFEST.in b/MANIFEST.in index 581e7b4..3808d03 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -17,3 +17,11 @@ prune rust/target include java/pom.xml java/LICENSE java/NOTICE recursive-include java *.java *.md *.json *.jac prune java/target +include dotnet/Directory.Build.props dotnet/.editorconfig dotnet/LICENSE dotnet/NOTICE dotnet/JacquardHost.sln +recursive-include dotnet *.cs *.csproj *.md *.json *.jac +prune dotnet/src/JacquardHost/bin +prune dotnet/src/JacquardHost/obj +prune dotnet/test/JacquardHost.Tests/bin +prune dotnet/test/JacquardHost.Tests/obj +prune dotnet/test/JacquardHost.StandIn/bin +prune dotnet/test/JacquardHost.StandIn/obj diff --git a/README.md b/README.md index 0302666..c72341d 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,9 @@ the [Go SDK](go/README.md), the [Rust SDK](rust/README.md), and the TypeScript SDK supports async service methods, the Go SDK maps native Go types and context cancellation, the Rust SDK maps native Rust types through `record!` and `variants!` declarations, and the Java SDK maps records, enums, and sealed -interfaces through explicit codecs. +interfaces through explicit codecs. A protocol-level [.NET](dotnet/README.md) +(.NET 8+) adapter also implements the pinned protocol independently. It has a +closed operation registry and kit replay, but no SDK layer yet. **Experimental:** the SDK targets Core's pinned `jacquard-host-v0` protocol. Core is installed separately. Each invocation runs one fresh, serial worker; @@ -116,8 +118,9 @@ participant collection require separate application work. See [CONTRIBUTING.md](CONTRIBUTING.md) for offline tests, installed-distribution checks, and live runs. CI tests Python 3.11–3.13, Node.js 22/24, Go 1.24 and -stable, Rust 1.88 and stable, and Java 21 and 25. It separately installs the -pinned Core release for every adapter's live suite and the public examples. +stable, Rust 1.88 and stable, Java 21 and 25, and .NET 8 and 10. It +separately installs the pinned Core release for every adapter's live suite and +the public examples. [Evidence](docs/EVIDENCE.md) records the checks and platform claims; [development experiments](docs/development/README.md) preserve the usability diff --git a/docs/COMPATIBILITY.md b/docs/COMPATIBILITY.md index 6de1279..cf2b4aa 100644 --- a/docs/COMPATIBILITY.md +++ b/docs/COMPATIBILITY.md @@ -7,12 +7,14 @@ release, recorded once in [`core-pin.json`](../core-pin.json). The Python constants (`src/jacquard_host/compat.py`), the TypeScript constants (`typescript/src/compat.ts`), the Go constants (`go/protocol/compat.go`), the Rust constants (`rust/src/compat.rs`), the Java constants -(`java/src/main/java/org/jacquardlang/host/Compat.java`), and the block below -are generated from that file by `scripts/core_pin.py sync`. +(`java/src/main/java/org/jacquardlang/host/Compat.java`), the .NET constants +(`dotnet/src/JacquardHost/Compat.cs`), and the block below are generated from +that file by `scripts/core_pin.py sync`. `scripts/core_pin.py check` fails CI when any of them drifts from the pin. It also fails when any vendored kit copy drifts: the Python package data, Go's -embedded `go/kit/data/`, the Rust crate's embedded `rust/kit/`, or the Java -class-path copy under `java/src/main/resources/`. Adapters verify the kit's four +embedded `go/kit/data/`, the Rust crate's embedded `rust/kit/`, the Java +class-path copy under `java/src/main/resources/`, or the .NET embedded copy +under `dotnet/src/JacquardHost/kit/`. Adapters verify the kit's four digests independently at load time, and the SDKs refuse a Core whose `--version` differs from the pinned version. @@ -43,7 +45,7 @@ Release assets verified by `scripts/core_pin.py install`: | macos-x86_64 | `jacquard-macos-x86_64.tar.gz` | `8efc54bcf7573cb7cfd35fa3e77a584680ce68de872bdb212ed8369a974ff967` | -Python, TypeScript, Go, Rust, and Java replay the same 21 recordings against the pinned +Python, TypeScript, Go, Rust, Java, and .NET replay the same 21 recordings against the pinned worker, including Core's recorded `noncanonical-target-hash` divergence: the HB.1 vector expects E1603 while the shipped preflight classifies an uppercase target hash as a malformed scalar (E1601). The transcript is recorded as @@ -54,8 +56,8 @@ independent v0 evidence; it does not change the protocol version or declare v1. The adapters decode boundary descriptors in Core's order: the kind, then the exact fields, then one node of `max_value_nodes`, then any identity, then the list count. Malformed input therefore gets the same error code from every -adapter and from Core. For recording comparisons, Python, Go, Rust, and Java sort object -keys by code point, while TypeScript sorts them by UTF-16 code unit. The two orders +adapter and from Core. For recording comparisons, Python, Go, Rust, Java, and .NET sort +object keys by code point, while TypeScript sorts them by UTF-16 code unit. The two orders differ only for keys outside the Basic Multilingual Plane. Every protocol key is ASCII, so the comparisons agree. @@ -117,6 +119,7 @@ python3 -m pip install . && python3 scripts/check_examples.py (cd go && go test -race ./... && go test -count=1 ./...) (cd rust && cargo test) (cd java && JACQUARD_REQUIRE_LIVE=1 mvn -B verify) +(cd dotnet && JACQUARD_REQUIRE_LIVE=1 dotnet test test/JacquardHost.Tests -f net8.0) ``` `update` resolves the tag to its commit, copies the kit files from diff --git a/docs/EVIDENCE.md b/docs/EVIDENCE.md index 66f5540..fe4acbb 100644 --- a/docs/EVIDENCE.md +++ b/docs/EVIDENCE.md @@ -125,6 +125,24 @@ only test dependency. | Compiler | `-Xlint:all -Werror` is clean on both JDKs | | Unchanged lanes | The adapter's 60 offline and 5 live tests, the 21-recording kit replay, `scripts/core_pin.py check`, the offline Python suite (222 tests, 55 live skipped), and `scripts/check_distribution.py` pass | +## Independent .NET adapter (2026-10-29) + +Recorded against the same `jacquard-core-0.3.0` release asset (linux-x86_64), +on Linux 7.0 with the .NET 8.0.427 and 10.0.403 SDKs (runtimes 8.0.33 and +10.0.14). The library targets .NET 8 and has no runtime dependencies beyond +the base class library; it calls the C library's `poll(2)`. NUnit 4.6.1, +NUnit3TestAdapter 5.2.0, and Microsoft.NET.Test.Sdk 17.14.1 are its only test +dependencies. + +| Lane | Result | +|---|---| +| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 61 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 26 kit-replay and host-fault tests, 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | +| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 66 tests pass on .NET 8 and on .NET 10, none ignored: the 61 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | +| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); no guard on a cancelled handler token (`ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent`: the answer is sent); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 fail | +| Build | `TreatWarningsAsErrors`, nullable reference types, and the `8.0-recommended` analyzers are clean with both SDKs; the .NET 8 SDK builds and passes the net8.0 suite on its own | +| Pin consistency | `dotnet/src/JacquardHost/Compat.cs` is generated, and the embedded kit copy matches the pinned digests and the Python copy (`scripts/core_pin.py check`); `tests/test_core_pin.py` covers drift in both, and the offline Python suite passes | +| Distribution | `scripts/check_distribution.py` passes; the source archive carries the .NET sources, `dotnet/NOTICE`, and the kit copy | + ## Outcome frame node budget (2026-10-25) Core's `outcome` encodes the evidence's interface types and the result value @@ -214,6 +232,7 @@ reaches persistence has request ordinal 1 and one accepted `ok` observation. | Linux, Go 1.24 and stable | CI lanes | Offline Go tests with the race detector on both; live Go tests in the shared pinned-Core job | | Linux, Rust 1.88 and stable | CI lanes | fmt, clippy, and offline Rust tests on both; live Rust tests in the shared pinned-Core job | | Linux, Java 21 and 25 | CI lanes | Offline Java adapter and SDK tests on both; live Java tests and the four Java projects on Java 21 in the shared pinned-Core job; offline and live suites also run locally on Temurin 21.0.13 and 25.0.5 | +| Linux, .NET 8 and 10 | CI lanes | Offline .NET tests on both; live .NET tests on both in the shared pinned-Core job; offline and live suites also run locally on .NET 8.0.33 and 10.0.14 | | Linux, Node.js 26.7 | exercised | TypeScript offline/live suites and tarball installation exercised locally; no broader runtime or OS claim | | macOS | development only | no CI; the descriptor-count assertions are skipped without `/proc` (the socket host tests skip, the kit and adapter tests skip only the count); SQLite WAL mode is recorded, not assumed | | Windows and others | no claim | see LIMITS.md | diff --git a/docs/LIMITS.md b/docs/LIMITS.md index ba90f97..9b92686 100644 --- a/docs/LIMITS.md +++ b/docs/LIMITS.md @@ -9,7 +9,7 @@ also updates this document. | First-party Jacquard programs only | Capability checks do not make hostile uploaded code safe. | A separate-process or equivalent isolation design, adversarial review, resource ceilings, and a narrower public threat model. | | Trusted host and operating system | A malicious or buggy adapter can misreport or perform actions outside Jacquard. | An authenticated, independently enforced host boundary and external security review; canonical hashes alone are insufficient. | | Application capability boundary, not OS sandbox | Declared and checked effects are review and application enforcement, not kernel isolation. | An explicit OS/WebAssembly/container isolation layer with its own evidence and deployment contract. | -| Experimental process carrier (Core's `v0`, evidence pack published) | Message framing may change and is not a permanent ABI promise; Python, TypeScript, Go, Rust, and Java replay the pinned kit independently, while a stable protocol decision remains Core's. | Two independent adapters and the serial application pass the same frozen fixtures before Core declares protocol v1 stable. | +| Experimental process carrier (Core's `v0`, evidence pack published) | Message framing may change and is not a permanent ABI promise; Python, TypeScript, Go, Rust, Java, and .NET replay the pinned kit independently, while a stable protocol decision remains Core's. | Two independent adapters and the serial application pass the same frozen fixtures before Core declares protocol v1 stable. | | Handlers run inside the adapter's read loop | A Python operation handler that blocks holds the worker until it returns. The host deadline is one lifetime budget from spawn, enforced on pipe reads, writes, and exit waits; a slow handler spends the budget and the next carrier operation expires. It cannot interrupt Python handler code. | A handler timeout policy with its own cancel/unknown-completion evidence, once an application needs one. | | Go worker descendants that leave the process group | The Go host stops the worker's process group at expiry, and on Linux also after exit while a descendant still holds stderr; elsewhere, and for a descendant that called `setsid` (or otherwise left the group), it only stops reading the pipe. | An OS-level containment mechanism (for example a cgroup or job object) with its own evidence. | | Go handler cancellation is cooperative | At the deadline or on context cancellation the Go host kills the worker's process group (while the worker is unreaped, which Linux guarantees), closes its pipes, and reaps it, cancels the handler's context, and sends no late response; a handler that ignores its context keeps running in its goroutine until it returns. No outside action is stopped or rolled back. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | @@ -26,6 +26,10 @@ also updates this document. | Java SDK record order is declared, not exported | A record's components, in declaration order, are a constructor's fields; the pinned CLI exports no field types, so a wrong order is not detected locally, and Core refuses it only where the swapped fields' types differ. | Core's structured host interface exporter, so bindings can be generated or checked. | | Java fulfillment is a single-process snapshot example | Atomic JSON replacement demonstrates booking/outbox recovery and deduplication, without cross-process locking or power-loss durability evidence. | A concrete production storage contract and crash/concurrent-writer evidence. | | Java platforms | Linux runs in CI. macOS has no CI; the offline descendant and fake-Core tests need `sh` and `sleep`, and the descriptor tests need `/proc`. Windows is not claimed. | CI, lifecycle, and cleanup evidence on each added platform. | +| .NET worker process tree, not group | .NET cannot start the worker in its own process group, so the .NET host kills a process tree. At expiry, cancellation, or disposal of a running worker, it calls `Process.Kill(true)`, which stops the worker, lists its children from the process table, kills it, and does the same for each child. A descendant that was already reparented keeps running: one whose parent exited, or one that daemonized. So does one started after its parent was listed. Listing the process table takes time proportional to the number of processes on the machine. The runtime reaps the worker as soon as it exits. A descendant still holding stdout or stderr after the exit is detected and, after a one-second grace, reported as a carrier loss, as the Go and Rust hosts report it; it is not stopped. .NET reports a signal as exit status 128 + its number, so only the host's own SIGKILL is reported as a signal. | An OS-level containment mechanism (for example a cgroup or job object) with its own evidence, or a .NET process-group API. | +| .NET pipe I/O goes through `poll(2)` | .NET's pipe streams have no read or write that a deadline or a cancellation can interrupt, so the .NET host never calls one that can block: the caller's thread asks the C library's `poll(2)` whether a pipe is ready, in waits of at most 5 ms that recheck the deadline, the `CancellationToken`, and disposal, and then reads only bytes already in the pipe or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a writable pipe accepts at once. No thread is left blocked on a worker pipe, even while an escaped descendant holds stdin without reading or floods stdout or stderr. The `pollfd` layout and event bits are the same on Linux and macOS; elsewhere, including Windows, the host does not start a worker and reports a spawn failure. .NET opens its own descriptors close-on-exec, but passes the worker any descriptor the host process itself inherited without that flag. | A .NET API for interruptible or non-blocking process pipes, and CI on each added platform. | +| .NET handler cancellation is cooperative | Each .NET handler runs on its own background thread. At the deadline or on cancellation the host kills the worker's process tree, closes its input, and sends no late response. `HandlerContext.IsCancelled` turns true, and stays true once the invocation ends, and `HandlerContext.CancellationToken` is cancelled. A handler that ignores both keeps running on its thread until it returns. No outside action is stopped or rolled back. A stack overflow in a handler ends the process, as it does any .NET code. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | +| .NET adapter is protocol-level only | .NET callers pass exact identities, boundary types, and protocol values to a prepared store; there is no .NET SDK for program preparation, native type mapping, or caching, and `Invoke` is synchronous. Linux runs in CI on .NET 8 and 10. macOS has no CI; the offline descendant tests need `sh`, `sleep`, and `yes`, and the descriptor tests need `/proc`. Windows is not supported. | A .NET SDK with typed-binding and preparation evidence comparable to the other SDKs, and CI on each added platform. | | Node handler cancellation is cooperative | The Node adapter can expire while awaiting a Promise, signal the handler, and kill the worker; it cannot stop an outside action or preempt JavaScript that blocks the event loop. No late response or automatic retry follows expiration. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | | Socket host closes without a response on its own failures | A ceiling violation, early disconnect, or failed invocation ends the connection silently; clients see a reset, not an HTTP error. | An owner decision that the host may emit fixed minimal status lines, recorded here with the exact bytes and the rule that they carry no application meaning. | | Request bytes cross as Latin-1 `Text` | v0 has no bytes value; each byte is one codepoint 0..255 and a response codepoint above 255 is refused by the host. | A frozen bytes value in a later protocol version. | diff --git a/dotnet/README.md b/dotnet/README.md index 418f97a..be6c776 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -95,6 +95,25 @@ structurally. `max_text_bytes` or the frame, which is refused with E1602 before it is scanned. +## Bounded work + +Work on the caller's thread is bounded by the selected limits, not by the +size of what a handler or caller hands over: + +- Frame encoding counts UTF-8 bytes as it writes them and stops as soon as + the count passes `max_frame_bytes`. A handler result of 256 references to + one 256 KiB text (within every value limit, over 384 MiB once escaped) is + refused as an outside failure after about one frame of work. +- Text validation refuses a text whose length in chars already exceeds + `max_text_bytes` before scanning it, and charges every text in a frame + against `max_frame_bytes`, so validating a value scans at most about one + frame of text however often a long text is referenced. +- A frame given as a caller's own dictionary (rather than a `JsonObject`) is + refused by its field count and key lengths before it is copied or sorted. + A registry with more than `max_operations` bindings is refused before a + worker starts. +- Reads are bounded by `max_frame_bytes`, and stderr by `max_stderr_bytes`. + ## Handlers and cancellation A handler returns a value or throws a `HandlerException`. The four structured @@ -162,7 +181,9 @@ thread. The carrier is built around both: What this guarantees: an invocation returns at its deadline plus bounded cleanup. Each cleanup wait is capped at one second: for a killed worker's exit to be recorded, and for a still-running worker to exit at disposal. So in the -worst case `Invoke` returns about two seconds after the deadline. This holds +worst case `Invoke` returns about two seconds after the deadline, plus the +time .NET takes to list the process table for the kill (the tests allow three +seconds). This holds while a descendant that escaped the kill floods stdout or stderr, or holds stdin without reading, and the flood does not reach the heap. The carrier leaves no thread or pipe descriptor behind; a handler that ignores diff --git a/scripts/check_distribution.py b/scripts/check_distribution.py index 977c91f..15d963c 100644 --- a/scripts/check_distribution.py +++ b/scripts/check_distribution.py @@ -36,7 +36,7 @@ def main() -> None: # Only our freshly built archive is extracted, never an external input. archive.extractall(root / "source", filter="data") source = next((root / "source").iterdir()) - for relative in ("LICENSE", "NOTICE", "go/NOTICE", "rust/NOTICE", "java/NOTICE", "java/pom.xml", "CONTRIBUTING.md", "CHANGELOG.md", "docs/SETUP.md", "scripts/check_examples.py", + for relative in ("LICENSE", "NOTICE", "go/NOTICE", "rust/NOTICE", "java/NOTICE", "java/pom.xml", "dotnet/NOTICE", "dotnet/src/JacquardHost/JacquardHost.csproj", "dotnet/src/JacquardHost/Compat.cs", "dotnet/src/JacquardHost/kit/host-protocol-v0/kit/transcripts.json", "CONTRIBUTING.md", "CHANGELOG.md", "docs/SETUP.md", "scripts/check_examples.py", "examples/python_sdk/app.py", "examples/python_sdk/exercise.py"): assert (source / relative).is_file(), f"sdist missing {relative}" for project in ("quickstart", "triage", "planner", "fulfillment"): From 37dd85813147c8a41128c1f8883e4b70050f763f Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:00:16 -0400 Subject: [PATCH 06/33] Bound the abandoned-handler test so a regression fails instead of hanging --- docs/EVIDENCE.md | 2 +- dotnet/test/JacquardHost.Tests/HostTests.cs | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/EVIDENCE.md b/docs/EVIDENCE.md index fe4acbb..7d67b0c 100644 --- a/docs/EVIDENCE.md +++ b/docs/EVIDENCE.md @@ -138,7 +138,7 @@ dependencies. |---|---| | .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 61 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 26 kit-replay and host-fault tests, 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | | .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 66 tests pass on .NET 8 and on .NET 10, none ignored: the 61 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | -| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); no guard on a cancelled handler token (`ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent`: the answer is sent); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 fail | +| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); no guard on a cancelled handler token (`ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent`: the answer is sent); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests | | Build | `TreatWarningsAsErrors`, nullable reference types, and the `8.0-recommended` analyzers are clean with both SDKs; the .NET 8 SDK builds and passes the net8.0 suite on its own | | Pin consistency | `dotnet/src/JacquardHost/Compat.cs` is generated, and the embedded kit copy matches the pinned digests and the Python copy (`scripts/core_pin.py check`); `tests/test_core_pin.py` covers drift in both, and the offline Python suite passes | | Distribution | `scripts/check_distribution.py` passes; the source archive carries the .NET sources, `dotnet/NOTICE`, and the kit copy | diff --git a/dotnet/test/JacquardHost.Tests/HostTests.cs b/dotnet/test/JacquardHost.Tests/HostTests.cs index bd232b7..fbfff09 100644 --- a/dotnet/test/JacquardHost.Tests/HostTests.cs +++ b/dotnet/test/JacquardHost.Tests/HostTests.cs @@ -373,12 +373,12 @@ public void AnAbandonedHandlerDoesNotHoldTheCaller() // its one-second deadline, and no late response follows. using var finished = new ManualResetEventSlim(); var watch = Stopwatch.StartNew(); - InvocationResult result = InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(1)), (context, args) => + InvocationResult result = Within(TimeSpan.FromSeconds(10), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(1)), (context, args) => { Thread.Sleep(3000); finished.Set(); return new TextValue("late"); - }); + })); TimeSpan elapsed = watch.Elapsed; Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); Assert.That(elapsed, Is.LessThan(TimeSpan.FromSeconds(2.5)), "the caller waited for the handler"); From 8e49a6b89eb3ce6cc72669decbcc55a7695b6054 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:26:39 -0400 Subject: [PATCH 07/33] Run handler token callbacks on a pool thread, never on the caller's Cancelling the handlers' token ran every callback a handler had registered synchronously: on the invoking thread when the call ended, on the thread that cancelled the caller's token, and on the CancelAfter timer. A blocking callback held Invoke past its deadline, a throwing one escaped Invoke as AggregateException or crashed the process from the timer, and disposing the caller-token registration waited for a running callback. The token is now cancelled on a thread-pool work item that discards callback exceptions. The deadline uses a Timer that queues that work item (re-armed in waits of at most a day, so no millisecond conversion overflows), the caller-token link only queues it, and End unregisters without waiting. --- dotnet/src/JacquardHost/Handlers.cs | 7 +- dotnet/src/JacquardHost/Host.cs | 102 +++++++-- .../JacquardHost.Tests/HandlerTokenTests.cs | 208 ++++++++++++++++++ 3 files changed, 302 insertions(+), 15 deletions(-) create mode 100644 dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs diff --git a/dotnet/src/JacquardHost/Handlers.cs b/dotnet/src/JacquardHost/Handlers.cs index ecf85fe..44a2e8c 100644 --- a/dotnet/src/JacquardHost/Handlers.cs +++ b/dotnet/src/JacquardHost/Handlers.cs @@ -36,8 +36,11 @@ internal HandlerContext(Func stopping, Func ended, CancellationToken public bool IsCancelled => stopping() || ended() || CancellationToken.IsCancellationRequested; /// - /// A token cancelled when the caller cancels, the lifetime deadline - /// passes, the host stops the worker, or the invocation ends. + /// A token cancelled when the caller cancels, shortly after the lifetime + /// deadline passes (timer granularity; is + /// exact), or when the invocation ends. Callbacks registered on it run on + /// a thread-pool thread, never on the caller's thread; the host does not + /// wait for them, and any exception they throw is discarded. /// public CancellationToken CancellationToken { get; } } diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index cc172a3..827998b 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -251,10 +251,17 @@ private sealed record Answer(Value? Value, FailureCategory Category, string? Mes [SuppressMessage("Design", "CA1001", Justification = "The handlers' token source is cancelled, never disposed: an abandoned handler may still hold its token.")] private sealed class Call { + /// + /// The longest single timer wait. A longer lifetime is covered by + /// re-arming, so no conversion to milliseconds can overflow. + /// + private static readonly TimeSpan MaxTimerDelay = TimeSpan.FromDays(1); + private readonly OperationRegistry registry; private readonly CancellationToken cancellationToken; private volatile bool ended; private CancellationTokenSource? handlerStop; + private Timer? handlerTimer; private CancellationTokenRegistration callerRegistration; internal Call(Worker worker, Session session, InvocationResult result, OperationRegistry registry, CancellationToken cancellationToken) @@ -275,12 +282,18 @@ internal Call(Worker worker, Session session, InvocationResult result, Operation /// Whether a host frame was being encoded or sent when a failure occurred. internal bool Sending { get; private set; } - /// Marks the invocation ended and cancels the handlers' token. + /// + /// Marks the invocation ended and cancels the handlers' token on a + /// pool thread. Nothing here runs or waits for a handler's callback: + /// the registration is removed without waiting, and the timer is + /// disposed without waiting. + /// internal void End() { ended = true; - callerRegistration.Dispose(); - handlerStop?.Cancel(); + callerRegistration.Unregister(); + handlerTimer?.Dispose(); + CancelHandlers(handlerStop); } private void Send(JsonObject frame, Limits limits) @@ -476,27 +489,90 @@ private Answer RunHandler(Handler handler, IReadOnlyList arguments) } /// - /// The handlers' token: cancelled by the caller's token, at the - /// lifetime deadline, and when the invocation ends. It is never - /// disposed, because an abandoned handler may still hold it; once - /// cancelled it holds no timer. + /// The handlers' token: cancelled when the caller's token is + /// cancelled, shortly after the lifetime deadline, and when the + /// invocation ends. Callbacks a handler registers on it run on a pool + /// thread, never on the caller's thread, the canceller's thread, or the + /// deadline timer, and nothing waits for them; their exceptions are + /// discarded. The source is never disposed, because an abandoned + /// handler may still hold its token. /// private CancellationToken HandlerToken() { if (handlerStop is null) { handlerStop = new CancellationTokenSource(); - // A timer cannot wait longer than about 24 days; IsCancelled - // sees a later deadline exactly in any case. TimeSpan remaining = HandlerTimerHook is { } hook ? hook(Worker.Remaining) : Worker.Remaining; - if (remaining <= TimeSpan.FromMilliseconds(int.MaxValue - 1)) + handlerTimer = new Timer(static state => ((Call)state!).TimerFired(), this, TimerDelay(remaining), Timeout.InfiniteTimeSpan); + callerRegistration = cancellationToken.UnsafeRegister(static state => CancelHandlers((CancellationTokenSource)state!), handlerStop); + } + + return handlerStop.Token; + } + + /// The timer delay for : rounded up to whole milliseconds, at least zero, at most . + private static TimeSpan TimerDelay(TimeSpan remaining) + { + if (remaining <= TimeSpan.Zero) + { + return TimeSpan.Zero; + } + + if (remaining >= MaxTimerDelay) + { + return MaxTimerDelay; + } + + return TimeSpan.FromMilliseconds(Math.Ceiling(remaining.TotalMilliseconds)); + } + + /// + /// Runs on the timer's pool thread: re-arms while a long lifetime has + /// more than one timer wait left, otherwise queues the cancellation. + /// + private void TimerFired() + { + TimeSpan remaining = Worker.Remaining; + if (remaining > MaxTimerDelay && !ended) + { + try + { + handlerTimer?.Change(TimerDelay(remaining), Timeout.InfiniteTimeSpan); + return; + } + catch (ObjectDisposedException) { - handlerStop.CancelAfter(TimeSpan.FromMilliseconds(Math.Ceiling(remaining.TotalMilliseconds))); + // The invocation ended; End cancels the token. } - callerRegistration = cancellationToken.UnsafeRegister(static state => ((CancellationTokenSource)state!).Cancel(), handlerStop); } - return handlerStop.Token; + CancelHandlers(handlerStop); + } + + /// + /// Cancels on a pool work item, so handler + /// callbacks never run on, or hold, the thread that asks. + /// + private static void CancelHandlers(CancellationTokenSource? source) + { + if (source is not null && !source.IsCancellationRequested) + { + ThreadPool.UnsafeQueueUserWorkItem(static s => CancelQuietly(s), source, preferLocal: false); + } + } + + /// Cancels a source, discarding whatever its callbacks throw. + [SuppressMessage("Design", "CA1031", Justification = "A handler's token callback may throw anything; it must not crash the process.")] + private static void CancelQuietly(CancellationTokenSource source) + { + try + { + source.Cancel(); + } + catch (Exception) + { + // Handler code; the host has no use for its failure. + } } /// Calls a handler, turning every exception into a failure answer. diff --git a/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs new file mode 100644 index 0000000..907071d --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs @@ -0,0 +1,208 @@ +using System; +using System.Diagnostics; +using System.Threading; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// +/// Callbacks a handler registers on +/// are handler code. They must never run on, or hold, the caller's thread, +/// the thread that cancels the caller's token, or the deadline timer, and +/// whatever they throw must neither escape nor +/// crash the process. +/// +[TestFixture] +public sealed class HandlerTokenTests +{ + private static InvocationResult InvokeEcho(Host host, Handler handler, CancellationToken cancellationToken = default) => + host.Invoke( + Pinned.Identity("kit-echo"), + Echo(), + new Value[] { new TextValue("ping") }, + new OperationRegistry().Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), handler), + cancellationToken); + + /// Cancels on its own thread after a delay, recording when Cancel returned and what it threw. + private sealed class Canceller + { + private readonly ManualResetEventSlim returned = new(); + private Exception? thrown; + + internal Canceller(CancellationTokenSource source, TimeSpan delay) + { + var thread = new Thread(() => + { + Thread.Sleep(delay); + try + { + source.Cancel(); + } + catch (Exception error) + { + thrown = error; + } + + returned.Set(); + }) + { + IsBackground = true, + Name = "test-canceller", + }; + thread.Start(); + } + + internal bool Returned(TimeSpan limit) => returned.Wait(limit); + + internal Exception? Thrown => Volatile.Read(ref thrown); + } + + [Test] + public void AThrowingCallbackDoesNotEscapeASuccessfulInvoke() + { + using var ran = new ManualResetEventSlim(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success"), (context, args) => + { + context.CancellationToken.Register(() => + { + ran.Set(); + throw new InvalidOperationException("callback"); + }); + return new TextValue("pong"); + })); + Assert.That(result.Ok, Is.True, result.ToString()); + Assert.That(ran.Wait(TimeSpan.FromSeconds(5)), Is.True, "the token was not cancelled when the invocation ended"); + } + + [Test] + public void ABlockingCallbackDoesNotHoldASuccessfulInvoke() + { + // The callback blocks until the test releases it; the caller must not + // wait for it, whatever the deadline. + using var release = new ManualResetEventSlim(); + using var entered = new ManualResetEventSlim(); + try + { + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(2)), (context, args) => + { + context.CancellationToken.Register(() => + { + entered.Set(); + release.Wait(TimeSpan.FromSeconds(60)); + }); + return new TextValue("pong"); + })); + Assert.That(result.Ok, Is.True, result.ToString()); + Assert.That(entered.Wait(TimeSpan.FromSeconds(5)), Is.True, "the callback never ran"); + } + finally + { + release.Set(); + } + } + + [Test] + public void ABlockingCallbackHoldsNeitherTheCancellerNorInvoke() + { + using var release = new ManualResetEventSlim(); + using var cancel = new CancellationTokenSource(); + Canceller? canceller = null; + var watch = Stopwatch.StartNew(); + try + { + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(30)), (context, args) => + { + context.CancellationToken.Register(() => release.Wait(TimeSpan.FromSeconds(60))); + canceller = new Canceller(cancel, TimeSpan.FromMilliseconds(300)); + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(20)); + return new TextValue("late"); + }, cancel.Token)); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a late response was sent"); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5)), "Invoke waited for the callback"); + Assert.That(canceller!.Returned(TimeSpan.FromSeconds(5)), Is.True, "the canceller's thread ran the callback"); + } + finally + { + release.Set(); + } + } + + [Test] + public void AThrowingCallbackDoesNotEscapeToTheCanceller() + { + using var cancel = new CancellationTokenSource(); + using var ran = new ManualResetEventSlim(); + Canceller? canceller = null; + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(30)), (context, args) => + { + context.CancellationToken.Register(() => + { + ran.Set(); + throw new InvalidOperationException("callback"); + }); + canceller = new Canceller(cancel, TimeSpan.FromMilliseconds(300)); + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(20)); + return new TextValue("late"); + }, cancel.Token)); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(ran.Wait(TimeSpan.FromSeconds(5)), Is.True, "the callback never ran"); + Assert.That(canceller!.Returned(TimeSpan.FromSeconds(5)), Is.True); + Assert.That(canceller.Thrown, Is.Null, "the handler's callback threw into the caller's Cancel"); + } + + [Test] + public void ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke() + { + using var release = new ManualResetEventSlim(); + using var entered = new ManualResetEventSlim(); + var watch = Stopwatch.StartNew(); + try + { + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(1)), (context, args) => + { + context.CancellationToken.Register(() => + { + entered.Set(); + release.Wait(TimeSpan.FromSeconds(60)); + }); + + // Ignores cancellation until its callback has run. + entered.Wait(TimeSpan.FromSeconds(10)); + return new TextValue("late"); + })); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a late response was sent"); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(3.5)), "Invoke waited for the callback"); + Assert.That(entered.Wait(TimeSpan.FromSeconds(5)), Is.True, "the token was not cancelled at the deadline"); + } + finally + { + release.Set(); + } + } + + [Test] + public void AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess() + { + // Callbacks run last-registered first, and Cancel throws only after + // all have run, so "after" signals just before the exception leaves + // Cancel. Unhandled on a timer thread, it would end this process. + using var after = new ManualResetEventSlim(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(1)), (context, args) => + { + context.CancellationToken.Register(after.Set); + context.CancellationToken.Register(() => throw new InvalidOperationException("timer callback")); + after.Wait(TimeSpan.FromSeconds(10)); + return new TextValue("late"); + })); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(after.Wait(TimeSpan.FromSeconds(5)), Is.True, "the token was not cancelled at the deadline"); + + // An unhandled exception ends the process within this pause. + Thread.Sleep(1000); + Assert.Pass("the process survived the callback's exception"); + } +} From d8fa91261165693c2d64f24819e54f11a0959f42 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:29:47 -0400 Subject: [PATCH 08/33] Cap poll timeouts in ticks so a long deadline cannot poll forever The poll timeout multiplied the ticks left by 1000 before applying the 5 ms cap. A deadline beyond about 107 days overflowed to a negative timeout, and poll with a negative timeout waits forever, so cancellation and disposal no longer ended reads, writes or exit waits. The +1 also let a poll end up to a millisecond after the deadline. The timeout is now capped in ticks before any multiplication and rounded down. The lifetime clamp is named (Worker.MaxDeadline, about 73 years, for any positive TimeSpan up to MaxValue) and guarded against overflow when added to the current timestamp. --- dotnet/src/JacquardHost/Carrier/Worker.cs | 44 ++++++++++--- .../src/JacquardHost/Carrier/WorkerOptions.cs | 4 +- .../JacquardHost.Tests/HandlerTokenTests.cs | 19 ++++++ dotnet/test/JacquardHost.Tests/HostTests.cs | 63 +++++++++++++++++++ 4 files changed, 120 insertions(+), 10 deletions(-) diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index 15b8544..cffee1d 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -108,6 +108,12 @@ private Worker(Process process, long expires, CancellationToken cancel) stderrFd = Descriptor(stderr); } + /// + /// The longest lifetime: a longer , + /// up to , is clamped to it (about 73 years). + /// + public static TimeSpan MaxDeadline { get; } = TimeSpan.FromDays(73 * 365); + /// The worker lifetime when is null. public static TimeSpan DefaultDeadline { get; } = TimeSpan.FromSeconds(30); @@ -158,9 +164,7 @@ public static Worker Start(string store, WorkerOptions options, CancellationToke options.Validate(); TimeSpan deadline = options.Deadline ?? DefaultDeadline; - // Clamp to about 73 years so the deadline in timestamp ticks cannot overflow. - double ticks = Math.Min(deadline.TotalSeconds, 73 * 365.25 * 86400) * Stopwatch.Frequency; - long expires = Stopwatch.GetTimestamp() + (long)ticks; + long expires = Expires(Stopwatch.GetTimestamp(), deadline); if (!Posix.Supported) { return Failed(expires, "unsupported platform", "the .NET host runs workers on Linux and macOS only", cancellationToken); @@ -226,6 +230,17 @@ public static Worker Start(string store, WorkerOptions options, CancellationToke } } + /// + /// The timestamp after . + /// Any positive is accepted; the lifetime is + /// clamped to , so the sum cannot overflow. + /// + internal static long Expires(long now, TimeSpan deadline) + { + double ticks = Math.Min(deadline.TotalSeconds, MaxDeadline.TotalSeconds) * Stopwatch.Frequency; + return ticks >= long.MaxValue - now ? long.MaxValue : now + (long)ticks; + } + private static int Descriptor(Stream stream) => stream is PipeStream pipe ? (int)pipe.SafePipeHandle.DangerousGetHandle() @@ -331,16 +346,26 @@ private static void KillQuietly(Process process) } } - private static int RemainingMs(long expires) + /// + /// The poll timeout: whole milliseconds left before the deadline, rounded + /// down so a poll never ends after it, and at most one slice. It is + /// capped in timestamp ticks before any multiplication, so a long + /// deadline cannot overflow into a negative (infinite) poll timeout. + /// + internal static int RemainingMs(long expires, long now) { - long left = expires - Stopwatch.GetTimestamp(); + long left = expires - now; if (left <= 0) { return 0; } - long ms = (left * 1000 / Stopwatch.Frequency) + 1; - return (int)Math.Min(ms, PollSliceMs); + if (left >= PollSliceMs * Stopwatch.Frequency / 1000) + { + return PollSliceMs; + } + + return (int)(left * 1000 / Stopwatch.Frequency); } /// @@ -363,7 +388,7 @@ private void Pause(int fd, short events) fds[count++] = new Posix.PollFd { Fd = stderrFd, Events = Posix.PollIn }; } - int timeout = RemainingMs(expires); + int timeout = RemainingMs(expires, Stopwatch.GetTimestamp()); if (count == 0) { Thread.Sleep(timeout); @@ -649,7 +674,8 @@ public void Await(Func isDone) } /// The time left in the lifetime, or zero. - internal TimeSpan Remaining => TimeSpan.FromSeconds(Math.Max(0, expires - Stopwatch.GetTimestamp()) / (double)Stopwatch.Frequency); + internal TimeSpan Remaining => + TimeSpan.FromSeconds(Math.Min(Math.Max(0, expires - Stopwatch.GetTimestamp()) / (double)Stopwatch.Frequency, MaxDeadline.TotalSeconds)); /// Closes the worker's stdin. Nothing is writing, so this never blocks. public void CloseInput() diff --git a/dotnet/src/JacquardHost/Carrier/WorkerOptions.cs b/dotnet/src/JacquardHost/Carrier/WorkerOptions.cs index aee13f5..f9fce51 100644 --- a/dotnet/src/JacquardHost/Carrier/WorkerOptions.cs +++ b/dotnet/src/JacquardHost/Carrier/WorkerOptions.cs @@ -32,7 +32,9 @@ public IReadOnlyDictionary? Environment /// /// Bounds the whole lifetime: spawn, pipes, handlers, and exit; null - /// means . + /// means . Any positive value up to + /// is accepted; a value above + /// is clamped to it. /// public TimeSpan? Deadline { get; init; } diff --git a/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs index 907071d..eb3ce5a 100644 --- a/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs +++ b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs @@ -205,4 +205,23 @@ public void AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess() Thread.Sleep(1000); Assert.Pass("the process survived the callback's exception"); } + + [Test] + public void AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller() + { + // A lifetime longer than any single timer wait: the token still + // works, and the caller's cancellation ends the call promptly. + using var cancel = new CancellationTokenSource(); + bool sawCancelled = false; + var watch = Stopwatch.StartNew(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromDays(365)), (context, args) => + { + cancel.CancelAfter(300); + sawCancelled = context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + return new TextValue("late"); + }, cancel.Token)); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + Assert.That(Eventually(() => Volatile.Read(ref sawCancelled), TimeSpan.FromSeconds(5)), Is.True, "the handler's token was not cancelled"); + } } diff --git a/dotnet/test/JacquardHost.Tests/HostTests.cs b/dotnet/test/JacquardHost.Tests/HostTests.cs index fbfff09..c6ffe6b 100644 --- a/dotnet/test/JacquardHost.Tests/HostTests.cs +++ b/dotnet/test/JacquardHost.Tests/HostTests.cs @@ -444,4 +444,67 @@ public void DisposalFromAnotherThreadEndsAWaitingRead() Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); Assert.That(Eventually(() => worker.Signal == "SIGKILL"), Is.True, "the worker was not killed"); } + + [Test] + public void PollTimeoutsAreNeverNegativeNorPastTheDeadline() + { + // Each poll waits whole milliseconds, at most one slice (5 ms), and + // never past the deadline, however far away the deadline is. + long now = 1_000; + long ms = Stopwatch.Frequency / 1000; + foreach (long left in new[] { long.MaxValue - now, Stopwatch.Frequency * 86_400 * 365, Stopwatch.Frequency * 86_400 * 200, 5 * ms, 5 * ms - 1, ms, ms - 1, 1, 0, -1, -Stopwatch.Frequency }) + { + int timeout = Worker.RemainingMs(now + left, now); + Assert.That(timeout, Is.InRange(0, 5), "left " + left); + Assert.That(timeout * ms, Is.LessThanOrEqualTo(Math.Max(0, left)), "a poll could end after the deadline; left " + left); + } + + Assert.That(Worker.RemainingMs(now + Stopwatch.Frequency * 86_400 * 365, now), Is.EqualTo(5)); + Assert.That(Worker.RemainingMs(now + ms - 1, now), Is.EqualTo(0)); + foreach (TimeSpan deadline in new[] { TimeSpan.FromDays(365), Worker.MaxDeadline, TimeSpan.MaxValue }) + { + Assert.That(Worker.Expires(now, deadline), Is.GreaterThan(now + (Stopwatch.Frequency * 86_400 * 364)), deadline.ToString()); + } + + Assert.That(Worker.Expires(long.MaxValue - 10, TimeSpan.MaxValue), Is.EqualTo(long.MaxValue)); + } + + [Test] + public void CancellationEndsAReadUnderTheLongestDeadlines() + { + // A deadline past about 107 days once overflowed the poll timeout to + // a negative value, which waits forever. + foreach (TimeSpan deadline in new[] { TimeSpan.FromDays(365), TimeSpan.MaxValue }) + { + using var cancel = new CancellationTokenSource(); + Worker worker = Worker.Start("unused", Standin("silent", string.Empty, deadline), cancel.Token); + try + { + After(TimeSpan.FromMilliseconds(300), cancel.Cancel); + var watch = Stopwatch.StartNew(); + CarrierException error = Within(TimeSpan.FromSeconds(10), () => Throws(() => worker.ReadFrame(Hard))!); + Assert.That(error.IsAborted, Is.True, deadline + ": " + error.Message); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5)), deadline.ToString()); + } + finally + { + Within(TimeSpan.FromSeconds(10), () => + { + worker.Dispose(); + return true; + }); + } + } + } + + [Test] + public void ALongDeadlineStillEndsAnInvocationAtCancellation() + { + using var cancel = new CancellationTokenSource(); + After(TimeSpan.FromMilliseconds(300), cancel.Cancel); + var watch = Stopwatch.StartNew(); + InvocationResult result = Within(TimeSpan.FromSeconds(10), () => InvokeReady(Faulty("silent", TimeSpan.MaxValue), cancel.Token)); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); + } } From c2c7b3d2b2e989f828710918907ff4e41ebd034a Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:30:36 -0400 Subject: [PATCH 09/33] Refuse an undefined failure category instead of letting it escape Invoke HandlerException accepted any cast FailureCategory value, and dispatch trusted it, so throw new HandlerException((FailureCategory)999, ...) reached the wire-name mapping and threw ArgumentOutOfRangeException out of Invoke. The constructor now refuses an undefined category with ArgumentOutOfRangeException, and dispatch treats any undefined category that still arrives as an unexpected exception: completion unknown. --- dotnet/src/JacquardHost/Handlers.cs | 6 +++++ dotnet/src/JacquardHost/Host.cs | 2 +- dotnet/test/JacquardHost.Tests/HostTests.cs | 29 +++++++++++++++++++++ 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/dotnet/src/JacquardHost/Handlers.cs b/dotnet/src/JacquardHost/Handlers.cs index 44a2e8c..aa7e9db 100644 --- a/dotnet/src/JacquardHost/Handlers.cs +++ b/dotnet/src/JacquardHost/Handlers.cs @@ -53,9 +53,15 @@ internal HandlerContext(Func stopping, Func ended, CancellationToken public sealed class HandlerException : Exception { /// A failure in one of the four permitted categories, with a message for Core. + /// is not a defined category. public HandlerException(FailureCategory category, string message) : base(message ?? throw new ArgumentNullException(nameof(message))) { + if (!Enum.IsDefined(category)) + { + throw new ArgumentOutOfRangeException(nameof(category), category, "not a failure category"); + } + Category = category; } diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 827998b..26a9cae 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -584,7 +584,7 @@ private static Answer CallHandler(Handler handler, HandlerContext context, IRead Value value = handler(context, arguments); return new Answer(value, FailureCategory.OutsideFailure, null); } - catch (HandlerException failure) + catch (HandlerException failure) when (Enum.IsDefined(failure.Category)) { return new Answer(null, failure.Category, failure.Message); } diff --git a/dotnet/test/JacquardHost.Tests/HostTests.cs b/dotnet/test/JacquardHost.Tests/HostTests.cs index c6ffe6b..1f7a5e4 100644 --- a/dotnet/test/JacquardHost.Tests/HostTests.cs +++ b/dotnet/test/JacquardHost.Tests/HostTests.cs @@ -3,6 +3,8 @@ using System.Diagnostics; using System.IO; using System.Linq; +using System.Reflection; +using System.Runtime.CompilerServices; using System.Text; using System.Threading; using JacquardLang.Host.Carrier; @@ -261,6 +263,33 @@ public void HandlerAnswersMapToTheirFailureCategories() "�" + string.Concat(Enumerable.Repeat("é", 2046))); } + [Test] + public void AnUndefinedFailureCategoryIsRefusedAtConstruction() + { + foreach (FailureCategory category in new[] { (FailureCategory)999, (FailureCategory)(-1), (FailureCategory)4 }) + { + ArgumentOutOfRangeException error = Throws(() => _ = new HandlerException(category, "bad category"))!; + Assert.That(error.ParamName, Is.EqualTo("category")); + } + + foreach (FailureCategory category in Enum.GetValues()) + { + Assert.That(new HandlerException(category, "fine").Category, Is.EqualTo(category)); + } + } + + [Test] + public void AnUndefinedFailureCategoryReachingDispatchIsCompletionUnknown() + { + // The constructor refuses it; this forges one past the constructor, + // as reflection or a future constructor could, and checks dispatch + // still returns a frozen failure instead of throwing out of Invoke. + var forged = (HandlerException)RuntimeHelpers.GetUninitializedObject(typeof(HandlerException)); + typeof(HandlerException).GetField("k__BackingField", BindingFlags.Instance | BindingFlags.NonPublic)!.SetValue(forged, (FailureCategory)999); + Assert.That((int)forged.Category, Is.EqualTo(999)); + AssertFailure(Response((context, args) => throw forged), FailureCategory.CompletionUnknown, "handler threw an exception"); + } + [Test] public void AResultFarOverTheFrameCeilingIsAnOutsideFailure() { From d4a53d6f628da0ad8480df2bc217e6f6e0162943 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:32:03 -0400 Subject: [PATCH 10/33] Dispatch from the same registry snapshot the invocation advertised Capabilities were copied from the registry when the invoke frame was built, but effect requests were looked up in the live registry, so a concurrent Bind could make the host run an operation it never advertised. Invoke now takes one immutable snapshot (bounded by max_operations before it is copied) and uses it for both. --- dotnet/src/JacquardHost/Handlers.cs | 34 +++++++++++++++++++-- dotnet/src/JacquardHost/Host.cs | 28 ++++++++--------- dotnet/test/JacquardHost.Tests/HostTests.cs | 21 +++++++++++++ 3 files changed, 66 insertions(+), 17 deletions(-) diff --git a/dotnet/src/JacquardHost/Handlers.cs b/dotnet/src/JacquardHost/Handlers.cs index aa7e9db..2e62ba6 100644 --- a/dotnet/src/JacquardHost/Handlers.cs +++ b/dotnet/src/JacquardHost/Handlers.cs @@ -125,11 +125,41 @@ public IReadOnlyList Entries() } } - internal Handler? Find(EffectRequest request) + /// + /// An immutable copy for one invocation, or null when more than + /// are bound; the count is checked + /// before anything is copied. + /// + internal RegistrySnapshot? Snapshot(int maxOperations) { lock (sync) { - return handlers.TryGetValue(new OperationEntry(request.Effect, request.Operation), out Handler? handler) ? handler : null; + return handlers.Count > maxOperations ? null : new RegistrySnapshot(new SortedDictionary(handlers)); } } } + +/// +/// The operations one invocation advertises and dispatches: a copy taken +/// once, so a concurrent cannot make +/// the host run an operation it did not advertise. +/// +internal sealed class RegistrySnapshot +{ + private readonly SortedDictionary handlers; + + internal RegistrySnapshot(SortedDictionary handlers) + { + this.handlers = handlers; + Entries = new List(handlers.Keys); + } + + /// No operations. + internal static RegistrySnapshot Empty { get; } = new(new SortedDictionary()); + + /// The advertised operations in protocol order. + internal IReadOnlyList Entries { get; } + + internal Handler? Find(EffectRequest request) => + handlers.TryGetValue(new OperationEntry(request.Effect, request.Operation), out Handler? handler) ? handler : null; +} diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 26a9cae..449c7cf 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -68,8 +68,8 @@ public Host(string store, HostOptions? options = null) /// /// Test seam: runs on the caller's thread once an effect request is - /// decoded, just before the caller's check. Null outside tests. It is - /// global, so tests that set it must not run in parallel. + /// decoded, before its handler is looked up and before the caller's + /// check. /// internal static volatile Action? RequestDecodedHook; @@ -106,16 +106,14 @@ public InvocationResult Invoke(Hash target, CallInterface spec, IReadOnlyList entries; + RegistrySnapshot operations; Value[] arguments; try { - // The caller's registry and arguments are bounded before they are copied. - if (registry.Count > Limits.MaxOperations) - { - throw ProtocolException.Exceeded("registry exceeds max_operations"); - } - - entries = registry.Entries(); + // The caller's registry and arguments are bounded before they are + // copied. The invocation advertises and dispatches one snapshot. + operations = registry.Snapshot(Limits.MaxOperations) ?? throw ProtocolException.Exceeded("registry exceeds max_operations"); + entries = operations.Entries; JsonObject frame = Codec.InvokeFrame(target, spec, args, entries, Limits); JsonCodec.FrameBytes(frame, Limits.MaxFrameBytes, Limits.MaxJsonDepth); arguments = args.ToArray(); @@ -132,12 +130,12 @@ public InvocationResult Invoke(Hash target, CallInterface spec, IReadOnlyList session.Invoke(target, spec, arguments, entries), registry, cancellationToken); + return Run(session => session.Invoke(target, spec, arguments, entries), operations, cancellationToken); } /// Negotiates a session and shuts it down without invoking. public InvocationResult ShutdownProbe(CancellationToken cancellationToken = default) => - Run(session => session.Shutdown(), new OperationRegistry(), cancellationToken); + Run(session => session.Shutdown(), RegistrySnapshot.Empty, cancellationToken); /// /// At most UTF-8 bytes of , @@ -175,7 +173,7 @@ internal static string Bounded(string text, int limit) return output.ToString(); } - private InvocationResult Run(Func opening, OperationRegistry registry, CancellationToken cancellationToken) + private InvocationResult Run(Func opening, RegistrySnapshot registry, CancellationToken cancellationToken) { var session = new Session(); var result = new InvocationResult(ResultKind.CarrierLoss); @@ -257,14 +255,14 @@ private sealed class Call /// private static readonly TimeSpan MaxTimerDelay = TimeSpan.FromDays(1); - private readonly OperationRegistry registry; + private readonly RegistrySnapshot registry; private readonly CancellationToken cancellationToken; private volatile bool ended; private CancellationTokenSource? handlerStop; private Timer? handlerTimer; private CancellationTokenRegistration callerRegistration; - internal Call(Worker worker, Session session, InvocationResult result, OperationRegistry registry, CancellationToken cancellationToken) + internal Call(Worker worker, Session session, InvocationResult result, RegistrySnapshot registry, CancellationToken cancellationToken) { Worker = worker; Session = session; @@ -399,6 +397,7 @@ internal void Drive(Func opening, Limits limits) /// private JsonObject Respond(EffectRequest request) { + RequestDecodedHook?.Invoke(); Handler? handler = registry.Find(request); Answer answer; if (handler is null) @@ -411,7 +410,6 @@ private JsonObject Respond(EffectRequest request) // begin once the call is cancelled or the lifetime passed. The // caller checks before creating the handler thread, and the // thread checks again before calling the handler. - RequestDecodedHook?.Invoke(); CarrierException? early = Worker.Fault(); if (early is not null) { diff --git a/dotnet/test/JacquardHost.Tests/HostTests.cs b/dotnet/test/JacquardHost.Tests/HostTests.cs index 1f7a5e4..9d9bc3e 100644 --- a/dotnet/test/JacquardHost.Tests/HostTests.cs +++ b/dotnet/test/JacquardHost.Tests/HostTests.cs @@ -445,6 +445,27 @@ public void ARegistryOverTheOperationCeilingIsRefusedLocally() Assert.That(result.Kind == ResultKind.HostError && result.Code == "E1602" && Reason(result) == "invoke_refused_locally", Is.True, result.ToString()); } + [Test] + public void AnOperationBoundAfterTheInvokeFrameIsNotDispatched() + { + // The seam binds `send` after the invoke frame advertised no + // operations and before the request is dispatched. The host must + // answer from what it advertised. + var registry = new OperationRegistry(); + bool ran = false; + Host.RequestDecodedHook = () => registry.Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), (context, args) => + { + ran = true; + return new TextValue("pong"); + }); + InvocationResult result = Replaying("one-once-effect-success").Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, registry); + Assert.That(result.HostFrames, Has.Count.EqualTo(3), result.ToString()); + Assert.That(((JsonObject)result.HostFrames[1]["capabilities"])["operations"].AsArray(), Is.Empty, "the invoke frame advertised an operation"); + Assert.That(registry.Count, Is.EqualTo(1), "the seam did not bind"); + Assert.That(Volatile.Read(ref ran), Is.False, "the host ran an operation it never advertised"); + AssertFailure(result.HostFrames[2], FailureCategory.UnsupportedOperation, "operation is not configured"); + } + [Test] public void CancellationWhileAHugeResultIsEncodedEndsTheCallPromptly() { From 4e885058dd1515b8f534fce74c471736f494fb3c Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:32:20 -0400 Subject: [PATCH 11/33] Explain the process-global test seams in Host They are internal, null outside tests, and global to the process, which is why the tests run serially. --- dotnet/src/JacquardHost/Host.cs | 9 ++++++++- dotnet/test/JacquardHost.Tests/Support.cs | 3 +++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 449c7cf..0133afe 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -66,10 +66,17 @@ public Host(string store, HostOptions? options = null) /// The limits each session selects. public Limits Limits { get; } + // Test seams. They are internal (reachable only from the test assembly + // through InternalsVisibleTo) and null outside tests, so production pays + // one volatile read of null at each. They are process-global: every + // invocation in the process sees them, so the tests run serially + // (LevelOfParallelism(1)) and each test fixture that sets one clears it + // in its TearDown. + /// /// Test seam: runs on the caller's thread once an effect request is /// decoded, before its handler is looked up and before the caller's - /// check. + /// check. Null outside tests. /// internal static volatile Action? RequestDecodedHook; diff --git a/dotnet/test/JacquardHost.Tests/Support.cs b/dotnet/test/JacquardHost.Tests/Support.cs index 65a012a..f5acecd 100644 --- a/dotnet/test/JacquardHost.Tests/Support.cs +++ b/dotnet/test/JacquardHost.Tests/Support.cs @@ -11,6 +11,9 @@ using JacquardLang.Host.Protocol; using NUnit.Framework; +// Host's test seams (RequestDecodedHook and the others) are process-global +// statics, so no two tests may run at once. No fixture is Parallelizable, and +// this keeps the worker count at one if one becomes so. [assembly: LevelOfParallelism(1)] namespace JacquardLang.Host.Tests; From 81be343ab315c6becf32c8f3d2ee0c4cda1c46ec Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:32:50 -0400 Subject: [PATCH 12/33] Build the .NET 8 CI job with the .NET 8 SDK Without a global.json, dotnet uses the newest SDK on the runner, so the 8.0 job could compile with SDK 10. The job now writes a global.json pinning 8.0.x (rollForward latestFeature) and checks the selected SDK before testing. The 10.0 job and local builds are unchanged. --- .github/workflows/ci.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d8d9657..732b828 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -105,6 +105,14 @@ jobs: - uses: actions/setup-dotnet@v4 with: dotnet-version: ${{ matrix.dotnet }}.x + # The runner may also have a newer SDK, which dotnet would pick by + # default. Pin this job to the .NET 8 SDK so it builds as an 8.0-only + # toolchain would (the test project targets net8.0 alone there). + - name: Pin the .NET 8 SDK + if: matrix.dotnet == '8.0' + run: | + printf '{"sdk":{"version":"8.0.100","rollForward":"latestFeature"}}\n' > global.json + dotnet --version | grep '^8\.0\.' - name: Offline tests run: dotnet test test/JacquardHost.Tests -f net${{ matrix.dotnet }} From 147eb6cf01cf0393a1693cd376c45f654e50b9a7 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 02:40:14 -0400 Subject: [PATCH 13/33] Document the .NET handler token, deadline range, and pid-based tree kill The README and LIMITS now say that handler token callbacks run on a pool thread the host never waits for, that the token is cancelled shortly after the deadline while IsCancelled is exact, which deadlines are accepted, that dispatch uses the advertised registry snapshot, and that Process.Kill(true) signals descendants by pid, so a reused pid can be signalled. EVIDENCE records the new tests and which fail without their fixes. --- docs/EVIDENCE.md | 6 +++--- docs/LIMITS.md | 4 ++-- dotnet/README.md | 35 +++++++++++++++++++++++++++-------- 3 files changed, 32 insertions(+), 13 deletions(-) diff --git a/docs/EVIDENCE.md b/docs/EVIDENCE.md index 7d67b0c..9b52808 100644 --- a/docs/EVIDENCE.md +++ b/docs/EVIDENCE.md @@ -136,9 +136,9 @@ dependencies. | Lane | Result | |---|---| -| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 61 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 26 kit-replay and host-fault tests, 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | -| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 66 tests pass on .NET 8 and on .NET 10, none ignored: the 61 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | -| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); no guard on a cancelled handler token (`ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent`: the answer is sent); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests | +| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 74 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 7 handler-token tests, 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline fires it; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | +| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 79 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 74 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | +| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); no guard on a cancelled handler token (`ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent`: the answer is sent); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched` | | Build | `TreatWarningsAsErrors`, nullable reference types, and the `8.0-recommended` analyzers are clean with both SDKs; the .NET 8 SDK builds and passes the net8.0 suite on its own | | Pin consistency | `dotnet/src/JacquardHost/Compat.cs` is generated, and the embedded kit copy matches the pinned digests and the Python copy (`scripts/core_pin.py check`); `tests/test_core_pin.py` covers drift in both, and the offline Python suite passes | | Distribution | `scripts/check_distribution.py` passes; the source archive carries the .NET sources, `dotnet/NOTICE`, and the kit copy | diff --git a/docs/LIMITS.md b/docs/LIMITS.md index 9b92686..6fcb95f 100644 --- a/docs/LIMITS.md +++ b/docs/LIMITS.md @@ -26,9 +26,9 @@ also updates this document. | Java SDK record order is declared, not exported | A record's components, in declaration order, are a constructor's fields; the pinned CLI exports no field types, so a wrong order is not detected locally, and Core refuses it only where the swapped fields' types differ. | Core's structured host interface exporter, so bindings can be generated or checked. | | Java fulfillment is a single-process snapshot example | Atomic JSON replacement demonstrates booking/outbox recovery and deduplication, without cross-process locking or power-loss durability evidence. | A concrete production storage contract and crash/concurrent-writer evidence. | | Java platforms | Linux runs in CI. macOS has no CI; the offline descendant and fake-Core tests need `sh` and `sleep`, and the descriptor tests need `/proc`. Windows is not claimed. | CI, lifecycle, and cleanup evidence on each added platform. | -| .NET worker process tree, not group | .NET cannot start the worker in its own process group, so the .NET host kills a process tree. At expiry, cancellation, or disposal of a running worker, it calls `Process.Kill(true)`, which stops the worker, lists its children from the process table, kills it, and does the same for each child. A descendant that was already reparented keeps running: one whose parent exited, or one that daemonized. So does one started after its parent was listed. Listing the process table takes time proportional to the number of processes on the machine. The runtime reaps the worker as soon as it exits. A descendant still holding stdout or stderr after the exit is detected and, after a one-second grace, reported as a carrier loss, as the Go and Rust hosts report it; it is not stopped. .NET reports a signal as exit status 128 + its number, so only the host's own SIGKILL is reported as a signal. | An OS-level containment mechanism (for example a cgroup or job object) with its own evidence, or a .NET process-group API. | +| .NET worker process tree, not group | .NET cannot start the worker in its own process group, so the .NET host kills a process tree. At expiry, cancellation, or disposal of a running worker, it calls `Process.Kill(true)`, which stops the worker, lists its children from the process table, kills it, and does the same for each child. A descendant that was already reparented keeps running: one whose parent exited, or one that daemonized. So does one started after its parent was listed. Descendants are found and signalled by pid, with nothing pinning their identity: one that exits, and is reaped once the worker is killed, between the listing and the signal can have its pid reused by an unrelated process, which is then signalled (the Java host signals process handles, and the Rust and Go hosts a process group). Listing the process table takes time proportional to the number of processes on the machine. The runtime reaps the worker as soon as it exits. A descendant still holding stdout or stderr after the exit is detected and, after a one-second grace, reported as a carrier loss, as the Go and Rust hosts report it; it is not stopped. .NET reports a signal as exit status 128 + its number, so only the host's own SIGKILL is reported as a signal. | An OS-level containment mechanism (for example a cgroup or job object) with its own evidence, or a .NET process-group API. | | .NET pipe I/O goes through `poll(2)` | .NET's pipe streams have no read or write that a deadline or a cancellation can interrupt, so the .NET host never calls one that can block: the caller's thread asks the C library's `poll(2)` whether a pipe is ready, in waits of at most 5 ms that recheck the deadline, the `CancellationToken`, and disposal, and then reads only bytes already in the pipe or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a writable pipe accepts at once. No thread is left blocked on a worker pipe, even while an escaped descendant holds stdin without reading or floods stdout or stderr. The `pollfd` layout and event bits are the same on Linux and macOS; elsewhere, including Windows, the host does not start a worker and reports a spawn failure. .NET opens its own descriptors close-on-exec, but passes the worker any descriptor the host process itself inherited without that flag. | A .NET API for interruptible or non-blocking process pipes, and CI on each added platform. | -| .NET handler cancellation is cooperative | Each .NET handler runs on its own background thread. At the deadline or on cancellation the host kills the worker's process tree, closes its input, and sends no late response. `HandlerContext.IsCancelled` turns true, and stays true once the invocation ends, and `HandlerContext.CancellationToken` is cancelled. A handler that ignores both keeps running on its thread until it returns. No outside action is stopped or rolled back. A stack overflow in a handler ends the process, as it does any .NET code. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | +| .NET handler cancellation is cooperative | Each .NET handler runs on its own background thread. At the deadline or on cancellation the host kills the worker's process tree, closes its input, and sends no late response. `HandlerContext.IsCancelled` turns true (it reads the deadline exactly), and stays true once the invocation ends, and `HandlerContext.CancellationToken` is cancelled, shortly after the deadline (timer granularity). Callbacks a handler registers on that token run on a thread-pool thread, which the host never waits for; a callback that blocks holds that pool thread, and anything a callback throws is discarded. A handler that ignores both keeps running on its thread until it returns. No outside action is stopped or rolled back. A stack overflow in a handler ends the process, as it does any .NET code. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | | .NET adapter is protocol-level only | .NET callers pass exact identities, boundary types, and protocol values to a prepared store; there is no .NET SDK for program preparation, native type mapping, or caching, and `Invoke` is synchronous. Linux runs in CI on .NET 8 and 10. macOS has no CI; the offline descendant tests need `sh`, `sleep`, and `yes`, and the descriptor tests need `/proc`. Windows is not supported. | A .NET SDK with typed-binding and preparation evidence comparable to the other SDKs, and CI on each added platform. | | Node handler cancellation is cooperative | The Node adapter can expire while awaiting a Promise, signal the handler, and kill the worker; it cannot stop an outside action or preempt JavaScript that blocks the event loop. No late response or automatic retry follows expiration. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | | Socket host closes without a response on its own failures | A ceiling violation, early disconnect, or failed invocation ends the connection silently; clients see a reset, not an HTTP error. | An owner decision that the host may emit fixed minimal status lines, recorded here with the exact bytes and the rule that they carry no application meaning. | diff --git a/dotnet/README.md b/dotnet/README.md index be6c776..fd50db1 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -132,11 +132,22 @@ A stop that lands after that check is seen by the handler through its context. At the deadline, or when the caller's `CancellationToken` is cancelled, the host kills the worker's process tree and sends no late response. Cancellation is cooperative: `HandlerContext.IsCancelled` turns true -as soon as the call is cancelled or its lifetime passes, and stays true once -the invocation ends, and `HandlerContext.CancellationToken` is cancelled when -the caller's token is, at the deadline, and when the invocation ends. A -handler that ignores both keeps running on its thread until it returns. -Nothing is retried. +as soon as the call is cancelled or its lifetime passes (it reads the +deadline exactly), and stays true once the invocation ends. +`HandlerContext.CancellationToken` is cancelled when the caller's token is, +shortly after the deadline (a timer, so up to a few milliseconds late), and +when the invocation ends. Callbacks registered on that token run on a +thread-pool thread, never on the caller's thread, the thread that cancelled +the caller's token, or the timer; the host does not wait for them, and +anything they throw is discarded. A callback that blocks holds only that +pool thread. A handler that ignores both keeps running on its thread until it +returns. Nothing is retried. + +`Invoke` advertises and dispatches one snapshot of the registry, taken when +it is called: an operation bound during the invocation is not served by it. +`HandlerException` refuses a category outside the four +(`ArgumentOutOfRangeException`); one that reaches dispatch anyway is reported +as completion unknown. ## Process carrier @@ -147,7 +158,7 @@ thread. The carrier is built around both: - **No read or write ever blocks.** The caller's thread asks the C library's `poll(2)` (the one native call the adapter makes) whether a pipe is ready, - in waits of at most 5 ms, and only then reads the bytes already in the pipe, + in waits of at most 5 ms that never end after the deadline, and only then reads the bytes already in the pipe, or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a pipe that poll reports writable accepts at once. Every wait, whether for a frame, a write, a handler, or the exit, rechecks the one lifetime deadline, @@ -165,7 +176,12 @@ thread. The carrier is built around both: stops the worker, lists its children from the process table, kills it, and does the same for each child. A descendant that was already reparented is not found and keeps running: one whose parent exited, or one that - daemonized. So is one started after its parent was listed. + daemonized. So is one started after its parent was listed. Descendants are + found and signalled by pid, so a descendant that exits, and is reaped, + between the listing and the signal can have its pid reused by an unrelated + process, which would then be signalled. The window is short, but the Java + host (process handles) and the Rust and Go hosts (process groups) do not + have it. - **A descendant holding the pipes after the exit is detected.** The runtime reaps the worker when it exits. If stdout or stderr still has a writer after that, a descendant holds it: the host waits up to one second for it to @@ -179,7 +195,10 @@ thread. The carrier is built around both: exit status. What this guarantees: an invocation returns at its deadline plus bounded -cleanup. Each cleanup wait is capped at one second: for a killed worker's exit +cleanup. `WorkerOptions.Deadline` accepts any positive `TimeSpan`, up to +`TimeSpan.MaxValue`; one longer than `Worker.MaxDeadline` (about 73 years) is +clamped to it, and cancellation and disposal end the call promptly whatever +the deadline. Each cleanup wait is capped at one second: for a killed worker's exit to be recorded, and for a still-running worker to exit at disposal. So in the worst case `Invoke` returns about two seconds after the deadline, plus the time .NET takes to list the process table for the kill (the tests allow three From f906ebb708fae1a1afcec72421d7064048c7ddfc Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 03:02:06 -0400 Subject: [PATCH 14/33] Cancel the .NET handler token from a dedicated thread when the invocation ends Remove the handler token's timer and its registration on the caller's token. The caller's thread already sees the deadline and cancellation in its 5 ms waits and ends the invocation; End now starts one short-lived background thread that cancels the token, before the worker is disposed. A lifetime over a day no longer cancels the token a day early, and a saturated thread pool no longer delays the cancellation. Add a lifetime clock seam to Worker so a test can move a 36- or 48-hour lifetime to its deadline, and tests that starve the pool and check the token is cancelled at the deadline, at the caller's cancellation, and at the end of a successful invocation before any later pool work item runs. --- dotnet/src/JacquardHost/Carrier/Worker.cs | 25 ++- dotnet/src/JacquardHost/Host.cs | 128 ++++--------- .../JacquardHost.Tests/HandlerTokenTests.cs | 177 +++++++++++++++++- dotnet/test/JacquardHost.Tests/HostTests.cs | 7 +- 4 files changed, 229 insertions(+), 108 deletions(-) diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index cffee1d..5fcb6d5 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -117,6 +117,17 @@ private Worker(Process process, long expires, CancellationToken cancel) /// The worker lifetime when is null. public static TimeSpan DefaultDeadline { get; } = TimeSpan.FromSeconds(30); + /// + /// Test seam: the lifetime clock, in timestamp + /// ticks, so a test can move a long lifetime to its deadline without + /// waiting. It sets the deadline and decides expiry; the one-second exit + /// graces keep the real clock. Process-global; null outside tests. + /// + internal static volatile Func? ClockHook; + + /// The lifetime clock: outside tests. + private static long Now() => ClockHook is { } clock ? clock() : Stopwatch.GetTimestamp(); + /// The spawn error, when the worker never started. public string? SpawnError { get; } @@ -138,7 +149,7 @@ private Worker(Process process, long expires, CancellationToken cancel) /// or its lifetime passed. Safe from any thread, and exact: it does not /// wait for the caller's next poll to notice the deadline. /// - public bool Stopping => expired || disposeRequested || cancel.IsCancellationRequested || Stopwatch.GetTimestamp() >= expires; + public bool Stopping => expired || disposeRequested || cancel.IsCancellationRequested || Now() >= expires; /// The retained stderr prefix. public byte[] Stderr @@ -164,7 +175,7 @@ public static Worker Start(string store, WorkerOptions options, CancellationToke options.Validate(); TimeSpan deadline = options.Deadline ?? DefaultDeadline; - long expires = Expires(Stopwatch.GetTimestamp(), deadline); + long expires = Expires(Now(), deadline); if (!Posix.Supported) { return Failed(expires, "unsupported platform", "the .NET host runs workers on Linux and macOS only", cancellationToken); @@ -246,7 +257,7 @@ stream is PipeStream pipe ? (int)pipe.SafePipeHandle.DangerousGetHandle() : throw new NotSupportedException("worker pipes are " + stream.GetType().Name + ", not pipe streams"); - private static bool Stopped(long expires, CancellationToken cancel) => cancel.IsCancellationRequested || Stopwatch.GetTimestamp() >= expires; + private static bool Stopped(long expires, CancellationToken cancel) => cancel.IsCancellationRequested || Now() >= expires; private static Worker Failed(long expires, string spawnError, string message, CancellationToken cancel) => new(expires, spawnError, cancel) { fault = CarrierException.Loss(message), exitRecorded = true }; @@ -281,7 +292,7 @@ private static Worker StoppedBeforeSpawn(long expires, CancellationToken cancel) Expire(); fault = CarrierException.Aborted("invocation aborted; outside completion is unknown"); } - else if (Stopwatch.GetTimestamp() >= expires) + else if (Now() >= expires) { Expire(); fault = CarrierException.Timeout("worker lifetime deadline exceeded"); @@ -388,7 +399,7 @@ private void Pause(int fd, short events) fds[count++] = new Posix.PollFd { Fd = stderrFd, Events = Posix.PollIn }; } - int timeout = RemainingMs(expires, Stopwatch.GetTimestamp()); + int timeout = RemainingMs(expires, Now()); if (count == 0) { Thread.Sleep(timeout); @@ -673,10 +684,6 @@ public void Await(Func isDone) } } - /// The time left in the lifetime, or zero. - internal TimeSpan Remaining => - TimeSpan.FromSeconds(Math.Min(Math.Max(0, expires - Stopwatch.GetTimestamp()) / (double)Stopwatch.Frequency, MaxDeadline.TotalSeconds)); - /// Closes the worker's stdin. Nothing is writing, so this never blocks. public void CloseInput() { diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 0133afe..076639d 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -92,12 +92,6 @@ public Host(string store, HostOptions? options = null) /// internal static volatile Action? HandlerCheckHook; - /// - /// Test seam: maps the lifetime left to the delay of the handlers' token - /// timer, so a test can make the timer fire early. Null outside tests. - /// - internal static volatile Func? HandlerTimerHook; - /// /// Runs with . The invoke /// frame is validated before a worker starts. Cancelling @@ -195,7 +189,9 @@ private InvocationResult Run(Func opening, RegistrySnapshot return result; } - var call = new Call(worker, session, result, registry, cancellationToken); + // The worker observes the caller's token in its waits on this thread; + // nothing is registered on it. + var call = new Call(worker, session, result, registry); try { call.Drive(opening, Limits); @@ -206,8 +202,10 @@ private InvocationResult Run(Func opening, RegistrySnapshot } finally { - worker.Dispose(); + // The handlers' token is cancelled before the worker is disposed, + // which may wait up to a second for a killed worker to exit. call.End(); + worker.Dispose(); } result.Requests = session.Requests.ToArray(); @@ -256,26 +254,16 @@ private sealed record Answer(Value? Value, FailureCategory Category, string? Mes [SuppressMessage("Design", "CA1001", Justification = "The handlers' token source is cancelled, never disposed: an abandoned handler may still hold its token.")] private sealed class Call { - /// - /// The longest single timer wait. A longer lifetime is covered by - /// re-arming, so no conversion to milliseconds can overflow. - /// - private static readonly TimeSpan MaxTimerDelay = TimeSpan.FromDays(1); - private readonly RegistrySnapshot registry; - private readonly CancellationToken cancellationToken; private volatile bool ended; private CancellationTokenSource? handlerStop; - private Timer? handlerTimer; - private CancellationTokenRegistration callerRegistration; - internal Call(Worker worker, Session session, InvocationResult result, RegistrySnapshot registry, CancellationToken cancellationToken) + internal Call(Worker worker, Session session, InvocationResult result, RegistrySnapshot registry) { Worker = worker; Session = session; Result = result; this.registry = registry; - this.cancellationToken = cancellationToken; } internal Worker Worker { get; } @@ -288,17 +276,16 @@ internal Call(Worker worker, Session session, InvocationResult result, RegistryS internal bool Sending { get; private set; } /// - /// Marks the invocation ended and cancels the handlers' token on a - /// pool thread. Nothing here runs or waits for a handler's callback: - /// the registration is removed without waiting, and the timer is - /// disposed without waiting. + /// Marks the invocation ended and starts the cancellation of the + /// handlers' token, if any handler was given one. Called once, on the + /// caller's thread, as soon as the invocation stops for any reason: + /// a deadline, cancellation, or disposal seen by the worker's waits + /// ends the invocation at once. It does not wait. /// internal void End() { ended = true; - callerRegistration.Unregister(); - handlerTimer?.Dispose(); - CancelHandlers(handlerStop); + StopHandlers(); } private void Send(JsonObject frame, Limits limits) @@ -424,16 +411,6 @@ private JsonObject Respond(EffectRequest request) } answer = RunHandler(handler, request.Arguments); - - // A cancelled handler token means the call is stopping. Its - // deadline timer may fire a moment before the worker's clock - // reaches the deadline, and a handler that returned on it must - // not have its answer sent: wait for the worker to report the - // stop (at most that moment), which throws. - if (handlerStop is { IsCancellationRequested: true }) - { - Worker.Await(static () => false); - } } CarrierException? fault = Worker.Fault(); @@ -494,74 +471,45 @@ private Answer RunHandler(Handler handler, IReadOnlyList arguments) } /// - /// The handlers' token: cancelled when the caller's token is - /// cancelled, shortly after the lifetime deadline, and when the - /// invocation ends. Callbacks a handler registers on it run on a pool - /// thread, never on the caller's thread, the canceller's thread, or the - /// deadline timer, and nothing waits for them; their exceptions are - /// discarded. The source is never disposed, because an abandoned - /// handler may still hold its token. + /// The handlers' token, created on the caller's thread for the first + /// handler. Only cancels it, once the + /// invocation has ended: there is no timer, no registration on the + /// caller's token, and no pool work item. The source is never + /// disposed, because an abandoned handler may still hold its token. /// private CancellationToken HandlerToken() { - if (handlerStop is null) - { - handlerStop = new CancellationTokenSource(); - TimeSpan remaining = HandlerTimerHook is { } hook ? hook(Worker.Remaining) : Worker.Remaining; - handlerTimer = new Timer(static state => ((Call)state!).TimerFired(), this, TimerDelay(remaining), Timeout.InfiniteTimeSpan); - callerRegistration = cancellationToken.UnsafeRegister(static state => CancelHandlers((CancellationTokenSource)state!), handlerStop); - } - + handlerStop ??= new CancellationTokenSource(); return handlerStop.Token; } - /// The timer delay for : rounded up to whole milliseconds, at least zero, at most . - private static TimeSpan TimerDelay(TimeSpan remaining) + /// + /// Called only from : starts a dedicated background + /// thread that cancels the handlers' token and discards whatever its + /// callbacks throw. Nothing waits for that thread, so a callback that + /// blocks holds only it, and the shared thread pool is not involved. + /// + private void StopHandlers() { - if (remaining <= TimeSpan.Zero) + CancellationTokenSource? source = handlerStop; + if (source is null) { - return TimeSpan.Zero; + return; } - if (remaining >= MaxTimerDelay) + var thread = new Thread(static state => CancelQuietly((CancellationTokenSource)state!)) { - return MaxTimerDelay; - } - - return TimeSpan.FromMilliseconds(Math.Ceiling(remaining.TotalMilliseconds)); - } - - /// - /// Runs on the timer's pool thread: re-arms while a long lifetime has - /// more than one timer wait left, otherwise queues the cancellation. - /// - private void TimerFired() - { - TimeSpan remaining = Worker.Remaining; - if (remaining > MaxTimerDelay && !ended) + IsBackground = true, + Name = "jacquard-handler-cancel", + }; + try { - try - { - handlerTimer?.Change(TimerDelay(remaining), Timeout.InfiniteTimeSpan); - return; - } - catch (ObjectDisposedException) - { - // The invocation ended; End cancels the token. - } + thread.Start(source); } - - CancelHandlers(handlerStop); - } - - /// - /// Cancels on a pool work item, so handler - /// callbacks never run on, or hold, the thread that asks. - /// - private static void CancelHandlers(CancellationTokenSource? source) - { - if (source is not null && !source.IsCancellationRequested) + catch (OutOfMemoryException) { + // No thread could be created. Fall back to the pool rather + // than run handler callbacks on the caller's thread. ThreadPool.UnsafeQueueUserWorkItem(static s => CancelQuietly(s), source, preferLocal: false); } } diff --git a/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs index eb3ce5a..cd7a468 100644 --- a/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs +++ b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs @@ -1,6 +1,7 @@ using System; using System.Diagnostics; using System.Threading; +using JacquardLang.Host.Carrier; using JacquardLang.Host.Protocol; using NUnit.Framework; using static JacquardLang.Host.Tests.Support; @@ -9,14 +10,18 @@ namespace JacquardLang.Host.Tests; /// /// Callbacks a handler registers on -/// are handler code. They must never run on, or hold, the caller's thread, -/// the thread that cancels the caller's token, or the deadline timer, and -/// whatever they throw must neither escape nor -/// crash the process. +/// are handler code. They must never run on, or hold, the caller's thread or +/// the thread that cancels the caller's token, and whatever they throw must +/// neither escape nor crash the process. The token +/// is cancelled at the deadline and not before, however long the lifetime, +/// and its cancellation does not depend on the shared thread pool. /// [TestFixture] public sealed class HandlerTokenTests { + [TearDown] + public void ClearSeams() => Worker.ClockHook = null; + private static InvocationResult InvokeEcho(Host host, Handler handler, CancellationToken cancellationToken = default) => host.Invoke( Pinned.Identity("kit-echo"), @@ -224,4 +229,168 @@ public void AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller() Assert.That(watch.Elapsed, Is.LessThan(TimeSpan.FromSeconds(5))); Assert.That(Eventually(() => Volatile.Read(ref sawCancelled), TimeSpan.FromSeconds(5)), Is.True, "the handler's token was not cancelled"); } + + [TestCase(36)] + [TestCase(48)] + public void ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore(int hours) + { + // The seam moves the lifetime clock: first a day and a minute in + // (where a one-day timer would have fired with hours left), then to a + // second before the deadline, then to the deadline itself. + long offset = 0; + Worker.ClockHook = () => Stopwatch.GetTimestamp() + Interlocked.Read(ref offset); + TimeSpan lifetime = TimeSpan.FromHours(hours); + bool cancelledAfterADay = true; + bool cancelledJustBefore = true; + bool flaggedJustBefore = true; + bool cancelledAtTheDeadline = false; + InvocationResult result = Within(TimeSpan.FromSeconds(20), () => InvokeEcho(Replaying("one-once-effect-success", lifetime), (context, args) => + { + Interlocked.Exchange(ref offset, Ticks(TimeSpan.FromDays(1) + TimeSpan.FromMinutes(1))); + cancelledAfterADay = context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromMilliseconds(300)); + Interlocked.Exchange(ref offset, Ticks(lifetime - TimeSpan.FromSeconds(1))); + cancelledJustBefore = context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromMilliseconds(300)); + flaggedJustBefore = context.IsCancelled; + Interlocked.Exchange(ref offset, Ticks(lifetime)); + cancelledAtTheDeadline = context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + return new TextValue("late"); + })); + Assert.That(Volatile.Read(ref cancelledAfterADay), Is.False, "the token was cancelled a day into the lifetime"); + Assert.That(Volatile.Read(ref cancelledJustBefore), Is.False, "the token was cancelled before the deadline"); + Assert.That(Volatile.Read(ref flaggedJustBefore), Is.False, "IsCancelled was true before the deadline"); + Assert.That(Volatile.Read(ref cancelledAtTheDeadline), Is.True, "the token was not cancelled at the deadline"); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a late response was sent"); + } + + private static long Ticks(TimeSpan span) => (long)(span.TotalSeconds * Stopwatch.Frequency); + + /// + /// Fills every thread-pool thread, and 40 more that the pool would have + /// to inject, with work items blocked on a gate, then queues a probe + /// behind them. While the probe has not run, any work item queued later + /// has not run either. Disposal opens the gate and waits for every item. + /// + private sealed class Starvation : IDisposable + { + private readonly ManualResetEventSlim gate = new(); + private readonly ManualResetEventSlim probe = new(); + private readonly CountdownEvent done; + + internal Starvation() + { + ThreadPool.GetMinThreads(out int workers, out _); + int blockers = Math.Max(workers, ThreadPool.ThreadCount) + 40; + done = new CountdownEvent(blockers + 1); + for (int i = 0; i < blockers; i++) + { + ThreadPool.UnsafeQueueUserWorkItem( + static s => + { + s.gate.Wait(TimeSpan.FromSeconds(60)); + s.done.Signal(); + }, + this, + preferLocal: false); + } + + ThreadPool.UnsafeQueueUserWorkItem( + static s => + { + s.probe.Set(); + s.done.Signal(); + }, + this, + preferLocal: false); + Thread.Sleep(200); + Assert.That(probe.IsSet, Is.False, "the thread pool was not saturated"); + } + + /// Whether the probe, and so any work item queued after it, has run. + internal bool PoolRan => probe.IsSet; + + public void Dispose() + { + gate.Set(); + Assert.That(done.Wait(TimeSpan.FromSeconds(30)), Is.True, "the blocked work items did not finish"); + gate.Dispose(); + probe.Dispose(); + done.Dispose(); + } + } + + /// Records, from the token's callback, whether the starved pool had run by then. + private sealed class CancelWitness + { + private readonly ManualResetEventSlim cancelled = new(); + private bool poolRanFirst; + + internal void Watch(CancellationToken token, Starvation starvation) => + token.Register(() => + { + Volatile.Write(ref poolRanFirst, starvation.PoolRan); + cancelled.Set(); + }); + + internal bool Cancelled(TimeSpan limit) => cancelled.Wait(limit); + + internal bool PoolRanFirst => Volatile.Read(ref poolRanFirst); + } + + [Test] + public void UnderAStarvedThreadPoolTheTokenIsCancelledAtTheDeadline() + { + var witness = new CancelWitness(); + using (var starvation = new Starvation()) + { + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(1)), (context, args) => + { + witness.Watch(context.CancellationToken, starvation); + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + return new TextValue("late"); + })); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a late response was sent"); + Assert.That(witness.Cancelled(TimeSpan.FromSeconds(10)), Is.True, "the token was not cancelled"); + Assert.That(witness.PoolRanFirst, Is.False, "the token was cancelled only after the starved pool ran"); + } + } + + [Test] + public void UnderAStarvedThreadPoolTheTokenIsCancelledWhenTheCallerCancels() + { + var witness = new CancelWitness(); + using var cancel = new CancellationTokenSource(); + using (var starvation = new Starvation()) + { + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(30)), (context, args) => + { + witness.Watch(context.CancellationToken, starvation); + _ = new Canceller(cancel, TimeSpan.FromMilliseconds(300)); + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + return new TextValue("late"); + }, cancel.Token)); + Assert.That(Reason(result), Is.EqualTo("aborted"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a late response was sent"); + Assert.That(witness.Cancelled(TimeSpan.FromSeconds(10)), Is.True, "the token was not cancelled"); + Assert.That(witness.PoolRanFirst, Is.False, "the token was cancelled only after the starved pool ran"); + } + } + + [Test] + public void UnderAStarvedThreadPoolTheTokenIsCancelledWhenTheInvocationEnds() + { + var witness = new CancelWitness(); + using (var starvation = new Starvation()) + { + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success"), (context, args) => + { + witness.Watch(context.CancellationToken, starvation); + return new TextValue("pong"); + })); + Assert.That(result.Ok, Is.True, result.ToString()); + Assert.That(witness.Cancelled(TimeSpan.FromSeconds(10)), Is.True, "the token was not cancelled"); + Assert.That(witness.PoolRanFirst, Is.False, "the token was cancelled only after the starved pool ran"); + } + } } diff --git a/dotnet/test/JacquardHost.Tests/HostTests.cs b/dotnet/test/JacquardHost.Tests/HostTests.cs index 9d9bc3e..0de96eb 100644 --- a/dotnet/test/JacquardHost.Tests/HostTests.cs +++ b/dotnet/test/JacquardHost.Tests/HostTests.cs @@ -27,7 +27,6 @@ public void ClearSeams() Host.RequestDecodedHook = null; Host.HandlerThreadStartedHook = null; Host.HandlerCheckHook = null; - Host.HandlerTimerHook = null; } [Test] @@ -419,10 +418,8 @@ public void AnAbandonedHandlerDoesNotHoldTheCaller() public void ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent() { // The handler returns as soon as its token is cancelled at the - // deadline. The token's timer and the worker's clock differ, and the - // seam makes the timer fire 300 ms early: the answer must still not - // reach Core, and the call ends at the deadline. - Host.HandlerTimerHook = remaining => remaining - TimeSpan.FromMilliseconds(300); + // deadline: the answer must not reach Core, and the call ends at the + // deadline. InvocationResult result = InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(2)), (context, args) => { context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); From 6e3849ae3bb371bb50cd32ef48d855a57505967b Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 03:05:47 -0400 Subject: [PATCH 15/33] Cancel the .NET handler token before the worker is killed Killing the worker's process tree scans the process table and took about 240 ms here, and the token was cancelled only after it. The handler wait now ends as soon as the call is stopping, the caller's thread starts the token's cancellation (at most once per invocation), and only then does the fault check kill the worker. The deadline-to-cancellation time measured in the clock-seam test fell from about 244 ms to 2-4 ms. A test seam runs just before the kill and waits for the token's callback; with the old order it times out. Another test pins where callbacks run: on the host's cancelling thread, or, when registered after the token is cancelled, synchronously on the handler's thread. --- dotnet/src/JacquardHost/Carrier/Worker.cs | 16 ++++- dotnet/src/JacquardHost/Host.cs | 30 ++++++--- .../JacquardHost.Tests/HandlerTokenTests.cs | 67 ++++++++++++++++++- 3 files changed, 100 insertions(+), 13 deletions(-) diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index 5fcb6d5..f9625ce 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -125,6 +125,13 @@ private Worker(Process process, long expires, CancellationToken cancel) /// internal static volatile Func? ClockHook; + /// + /// Test seam: runs on the caller's thread when the worker's lifetime + /// ends by expiry or cancellation, before its process tree is killed. + /// Process-global; null outside tests. + /// + internal static volatile Action? ExpireHook; + /// The lifetime clock: outside tests. private static long Now() => ClockHook is { } clock ? clock() : Stopwatch.GetTimestamp(); @@ -316,6 +323,7 @@ private void ThrowIfFault() /// Kills the worker's process tree and closes its input. private void Expire() { + ExpireHook?.Invoke(); expired = true; KillTree(); CloseInputCore(); @@ -663,7 +671,10 @@ public void WriteFrame(IReadOnlyDictionary frame, Limits limits) /// /// Waits until returns true, the deadline /// passes, the call is cancelled, or the worker is disposed, reading - /// stderr meanwhile. + /// stderr meanwhile. is asked before each + /// check, so a condition that includes returns + /// before the worker is killed; the caller must then check + /// . /// /// The lifetime ended first. public void Await(Func isDone) @@ -673,12 +684,13 @@ public void Await(Func isDone) { while (true) { - ThrowIfFault(); if (isDone()) { return; } + ThrowIfFault(); + Pause(-1, 0); } } diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 076639d..991f0a6 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -257,6 +257,7 @@ private sealed class Call private readonly RegistrySnapshot registry; private volatile bool ended; private CancellationTokenSource? handlerStop; + private bool handlersStopped; internal Call(Worker worker, Session session, InvocationResult result, RegistrySnapshot registry) { @@ -277,10 +278,9 @@ internal Call(Worker worker, Session session, InvocationResult result, RegistryS /// /// Marks the invocation ended and starts the cancellation of the - /// handlers' token, if any handler was given one. Called once, on the - /// caller's thread, as soon as the invocation stops for any reason: - /// a deadline, cancellation, or disposal seen by the worker's waits - /// ends the invocation at once. It does not wait. + /// handlers' token, if any handler was given one and it was not + /// already started. Called once, on the caller's thread. It does not + /// wait. /// internal void End() { @@ -466,14 +466,22 @@ private Answer RunHandler(Handler handler, IReadOnlyList arguments) } HandlerThreadStartedHook?.Invoke(); - Worker.Await(() => completion.Answer is not null); + // The wait ends as soon as the call is stopping, before the + // worker is killed (which can take a while), so the handlers' + // token is cancelled first; the fault check after the wait then + // kills the worker and reports why. + Worker.Await(() => completion.Answer is not null || Worker.Stopping); + if (Worker.Stopping) + { + StopHandlers(); + } return completion.Answer ?? new Answer(null, FailureCategory.CompletionUnknown, "handler threw an exception"); } /// /// The handlers' token, created on the caller's thread for the first /// handler. Only cancels it, once the - /// invocation has ended: there is no timer, no registration on the + /// caller's thread has seen the call stop or end: there is no timer, no registration on the /// caller's token, and no pool work item. The source is never /// disposed, because an abandoned handler may still hold its token. /// @@ -484,19 +492,21 @@ private CancellationToken HandlerToken() } /// - /// Called only from : starts a dedicated background - /// thread that cancels the handlers' token and discards whatever its - /// callbacks throw. Nothing waits for that thread, so a callback that + /// Called only on the caller's thread, when it sees the call stopping + /// while a handler runs and at : starts, at most + /// once, a dedicated background thread that cancels the handlers' + /// token and discards whatever its callbacks throw. Nothing waits for that thread, so a callback that /// blocks holds only it, and the shared thread pool is not involved. /// private void StopHandlers() { CancellationTokenSource? source = handlerStop; - if (source is null) + if (source is null || handlersStopped) { return; } + handlersStopped = true; var thread = new Thread(static state => CancelQuietly((CancellationTokenSource)state!)) { IsBackground = true, diff --git a/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs index cd7a468..97c456d 100644 --- a/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs +++ b/dotnet/test/JacquardHost.Tests/HandlerTokenTests.cs @@ -20,7 +20,11 @@ namespace JacquardLang.Host.Tests; public sealed class HandlerTokenTests { [TearDown] - public void ClearSeams() => Worker.ClockHook = null; + public void ClearSeams() + { + Worker.ClockHook = null; + Worker.ExpireHook = null; + } private static InvocationResult InvokeEcho(Host host, Handler handler, CancellationToken cancellationToken = default) => host.Invoke( @@ -251,8 +255,10 @@ public void ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore(int Interlocked.Exchange(ref offset, Ticks(lifetime - TimeSpan.FromSeconds(1))); cancelledJustBefore = context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromMilliseconds(300)); flaggedJustBefore = context.IsCancelled; + var latency = Stopwatch.StartNew(); Interlocked.Exchange(ref offset, Ticks(lifetime)); cancelledAtTheDeadline = context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + TestContext.Progress.WriteLine("token cancelled " + latency.Elapsed.TotalMilliseconds.ToString("F1", System.Globalization.CultureInfo.InvariantCulture) + " ms after the clock reached the deadline"); return new TextValue("late"); })); Assert.That(Volatile.Read(ref cancelledAfterADay), Is.False, "the token was cancelled a day into the lifetime"); @@ -393,4 +399,63 @@ public void UnderAStarvedThreadPoolTheTokenIsCancelledWhenTheInvocationEnds() Assert.That(witness.PoolRanFirst, Is.False, "the token was cancelled only after the starved pool ran"); } } + + [Test] + public void CallbacksRunOnTheHostsCancellingThreadUnlessRegisteredAfterTheCancellation() + { + // A callback registered before the cancellation runs on the host's + // dedicated cancelling thread, not a pool thread. One registered + // after it runs at once on the registering (handler) thread, as + // CancellationToken.Register does for any cancelled token. + string? before = null; + bool beforeOnPool = true; + int handlerThread = -1; + int lateThread = -2; + using var done = new ManualResetEventSlim(); + InvocationResult result = Within(TimeSpan.FromSeconds(15), () => InvokeEcho(Replaying("one-once-effect-success", TimeSpan.FromSeconds(1)), (context, args) => + { + handlerThread = Environment.CurrentManagedThreadId; + context.CancellationToken.Register(() => + { + before = Thread.CurrentThread.Name; + beforeOnPool = Thread.CurrentThread.IsThreadPoolThread; + }); + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(10)); + context.CancellationToken.Register(() => lateThread = Environment.CurrentManagedThreadId); + done.Set(); + return new TextValue("late"); + })); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(done.Wait(TimeSpan.FromSeconds(10)), Is.True, "the handler did not see its token cancelled"); + Assert.That(Volatile.Read(ref before), Is.EqualTo("jacquard-handler-cancel")); + Assert.That(Volatile.Read(ref beforeOnPool), Is.False); + Assert.That(Volatile.Read(ref lateThread), Is.EqualTo(Volatile.Read(ref handlerThread)), "a late registration did not run on the handler's thread"); + } + + [TestCase(false)] + [TestCase(true)] + public void TheTokenIsCancelledBeforeTheWorkerIsKilled(bool callerCancels) + { + // The seam holds the kill until the token's callback has run, for up + // to five seconds. If the kill came first, the callback would not run + // until after it, and the wait would time out. + using var cancelled = new ManualResetEventSlim(); + using var cancel = new CancellationTokenSource(); + bool cancelledBeforeKill = false; + Worker.ExpireHook = () => Volatile.Write(ref cancelledBeforeKill, cancelled.Wait(TimeSpan.FromSeconds(5))); + TimeSpan deadline = callerCancels ? TimeSpan.FromSeconds(30) : TimeSpan.FromSeconds(1); + InvocationResult result = Within(TimeSpan.FromSeconds(20), () => InvokeEcho(Replaying("one-once-effect-success", deadline), (context, args) => + { + context.CancellationToken.Register(cancelled.Set); + if (callerCancels) + { + _ = new Canceller(cancel, TimeSpan.FromMilliseconds(300)); + } + + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(15)); + return new TextValue("late"); + }, cancel.Token)); + Assert.That(Reason(result), Is.EqualTo(callerCancels ? "aborted" : "deadline"), result.ToString()); + Assert.That(Volatile.Read(ref cancelledBeforeKill), Is.True, "the worker was killed before the token was cancelled"); + } } From ada0c27954a83821d380e0e4710f6cfda6fd2e4c Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 03:06:52 -0400 Subject: [PATCH 16/33] Document when the .NET handler token is cancelled and where callbacks run The token is cancelled when the call stops or ends and never earlier; while a handler runs, the caller's thread checks the deadline and the caller's token at least every 5 ms and starts the cancellation before it kills the worker. Callbacks run on the host's own cancelling thread, not a pool thread, and one registered after the cancellation runs at once on the handler's thread. --- docs/LIMITS.md | 2 +- dotnet/README.md | 23 +++++++++++++++-------- dotnet/src/JacquardHost/Handlers.cs | 20 +++++++++++++++----- dotnet/src/JacquardHost/Host.cs | 7 ++++--- 4 files changed, 35 insertions(+), 17 deletions(-) diff --git a/docs/LIMITS.md b/docs/LIMITS.md index 6fcb95f..88e679a 100644 --- a/docs/LIMITS.md +++ b/docs/LIMITS.md @@ -28,7 +28,7 @@ also updates this document. | Java platforms | Linux runs in CI. macOS has no CI; the offline descendant and fake-Core tests need `sh` and `sleep`, and the descriptor tests need `/proc`. Windows is not claimed. | CI, lifecycle, and cleanup evidence on each added platform. | | .NET worker process tree, not group | .NET cannot start the worker in its own process group, so the .NET host kills a process tree. At expiry, cancellation, or disposal of a running worker, it calls `Process.Kill(true)`, which stops the worker, lists its children from the process table, kills it, and does the same for each child. A descendant that was already reparented keeps running: one whose parent exited, or one that daemonized. So does one started after its parent was listed. Descendants are found and signalled by pid, with nothing pinning their identity: one that exits, and is reaped once the worker is killed, between the listing and the signal can have its pid reused by an unrelated process, which is then signalled (the Java host signals process handles, and the Rust and Go hosts a process group). Listing the process table takes time proportional to the number of processes on the machine. The runtime reaps the worker as soon as it exits. A descendant still holding stdout or stderr after the exit is detected and, after a one-second grace, reported as a carrier loss, as the Go and Rust hosts report it; it is not stopped. .NET reports a signal as exit status 128 + its number, so only the host's own SIGKILL is reported as a signal. | An OS-level containment mechanism (for example a cgroup or job object) with its own evidence, or a .NET process-group API. | | .NET pipe I/O goes through `poll(2)` | .NET's pipe streams have no read or write that a deadline or a cancellation can interrupt, so the .NET host never calls one that can block: the caller's thread asks the C library's `poll(2)` whether a pipe is ready, in waits of at most 5 ms that recheck the deadline, the `CancellationToken`, and disposal, and then reads only bytes already in the pipe or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a writable pipe accepts at once. No thread is left blocked on a worker pipe, even while an escaped descendant holds stdin without reading or floods stdout or stderr. The `pollfd` layout and event bits are the same on Linux and macOS; elsewhere, including Windows, the host does not start a worker and reports a spawn failure. .NET opens its own descriptors close-on-exec, but passes the worker any descriptor the host process itself inherited without that flag. | A .NET API for interruptible or non-blocking process pipes, and CI on each added platform. | -| .NET handler cancellation is cooperative | Each .NET handler runs on its own background thread. At the deadline or on cancellation the host kills the worker's process tree, closes its input, and sends no late response. `HandlerContext.IsCancelled` turns true (it reads the deadline exactly), and stays true once the invocation ends, and `HandlerContext.CancellationToken` is cancelled, shortly after the deadline (timer granularity). Callbacks a handler registers on that token run on a thread-pool thread, which the host never waits for; a callback that blocks holds that pool thread, and anything a callback throws is discarded. A handler that ignores both keeps running on its thread until it returns. No outside action is stopped or rolled back. A stack overflow in a handler ends the process, as it does any .NET code. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | +| .NET handler cancellation is cooperative | Each .NET handler runs on its own background thread. At the deadline or on cancellation the host kills the worker's process tree, closes its input, and sends no late response. `HandlerContext.IsCancelled` turns true (it reads the deadline exactly), and stays true once the invocation ends. `HandlerContext.CancellationToken` is cancelled when the call stops or ends, never earlier (a lifetime of any length does not cancel it before its deadline); the caller's thread, which checks the deadline and the caller's token at least every 5 ms while a handler runs, starts its cancellation when it sees either, before it kills the worker, and otherwise shortly after the invocation ends. The cancellation runs on a short-lived thread the host starts for it, one per invocation that gave a handler a token, so it does not wait for the shared thread pool (if no thread can be created, it falls back to a pool work item). Callbacks a handler registers on that token run on that thread, which the host never waits for; a callback that blocks holds that thread, and anything a callback throws is discarded. A callback registered after the token is cancelled runs at once on the handler's own thread and throws there, as with any `CancellationToken`. A handler that ignores both keeps running on its thread until it returns. No outside action is stopped or rolled back. A stack overflow in a handler ends the process, as it does any .NET code. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | | .NET adapter is protocol-level only | .NET callers pass exact identities, boundary types, and protocol values to a prepared store; there is no .NET SDK for program preparation, native type mapping, or caching, and `Invoke` is synchronous. Linux runs in CI on .NET 8 and 10. macOS has no CI; the offline descendant tests need `sh`, `sleep`, and `yes`, and the descriptor tests need `/proc`. Windows is not supported. | A .NET SDK with typed-binding and preparation evidence comparable to the other SDKs, and CI on each added platform. | | Node handler cancellation is cooperative | The Node adapter can expire while awaiting a Promise, signal the handler, and kill the worker; it cannot stop an outside action or preempt JavaScript that blocks the event loop. No late response or automatic retry follows expiration. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | | Socket host closes without a response on its own failures | A ceiling violation, early disconnect, or failed invocation ends the connection silently; clients see a reset, not an HTTP error. | An owner decision that the host may emit fixed minimal status lines, recorded here with the exact bytes and the rule that they carry no application meaning. | diff --git a/dotnet/README.md b/dotnet/README.md index fd50db1..9e85494 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -134,14 +134,21 @@ cancelled, the host kills the worker's process tree and sends no late response. Cancellation is cooperative: `HandlerContext.IsCancelled` turns true as soon as the call is cancelled or its lifetime passes (it reads the deadline exactly), and stays true once the invocation ends. -`HandlerContext.CancellationToken` is cancelled when the caller's token is, -shortly after the deadline (a timer, so up to a few milliseconds late), and -when the invocation ends. Callbacks registered on that token run on a -thread-pool thread, never on the caller's thread, the thread that cancelled -the caller's token, or the timer; the host does not wait for them, and -anything they throw is discarded. A callback that blocks holds only that -pool thread. A handler that ignores both keeps running on its thread until it -returns. Nothing is retried. +`HandlerContext.CancellationToken` is cancelled when the call stops or ends, +and never earlier: however long the lifetime, the deadline alone does not +cancel it before it passes. While a handler runs, the caller's thread checks +the deadline and the caller's token at least every 5 ms; when it sees +either, it starts the token's cancellation before it kills the worker. +Otherwise the token is cancelled shortly after the invocation ends. The host +cancels it on a short-lived thread of its own, one per invocation, so a busy +thread pool does not delay it. Callbacks registered on the token run on that +thread, never on the caller's thread or the thread that cancelled the +caller's token; the host does not wait for them, and anything they throw is +discarded. A callback that blocks holds only that thread. A callback +registered after the token is already cancelled runs at once on the +registering (handler) thread, as with any `CancellationToken`, and its +exception is thrown there. A handler that ignores both keeps running on its +thread until it returns. Nothing is retried. `Invoke` advertises and dispatches one snapshot of the registry, taken when it is called: an operation bound during the invocation is not served by it. diff --git a/dotnet/src/JacquardHost/Handlers.cs b/dotnet/src/JacquardHost/Handlers.cs index 2e62ba6..93723e7 100644 --- a/dotnet/src/JacquardHost/Handlers.cs +++ b/dotnet/src/JacquardHost/Handlers.cs @@ -36,11 +36,21 @@ internal HandlerContext(Func stopping, Func ended, CancellationToken public bool IsCancelled => stopping() || ended() || CancellationToken.IsCancellationRequested; /// - /// A token cancelled when the caller cancels, shortly after the lifetime - /// deadline passes (timer granularity; is - /// exact), or when the invocation ends. Callbacks registered on it run on - /// a thread-pool thread, never on the caller's thread; the host does not - /// wait for them, and any exception they throw is discarded. + /// A token cancelled when the invocation stops or ends, and never + /// earlier: the lifetime alone does not cancel it before its deadline, + /// however long it is. While the handler runs, the caller's thread + /// checks the deadline and the caller's token at least every 5 ms; when + /// it sees either, it starts the cancellation before it kills the worker. + /// Otherwise the token is cancelled shortly after the invocation ends. + /// is exact; the token follows it. + /// The host cancels the token on a dedicated thread of its own, not a + /// thread-pool thread, so callbacks registered on it run there, never on + /// the caller's thread or the thread that cancelled the caller's token; + /// the host does not wait for them, and any exception they throw is + /// discarded. A callback registered after the token is cancelled runs at + /// once on the registering (handler) thread, as for any + /// , and its exception is + /// thrown there. /// public CancellationToken CancellationToken { get; } } diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 991f0a6..29beb75 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -481,9 +481,10 @@ private Answer RunHandler(Handler handler, IReadOnlyList arguments) /// /// The handlers' token, created on the caller's thread for the first /// handler. Only cancels it, once the - /// caller's thread has seen the call stop or end: there is no timer, no registration on the - /// caller's token, and no pool work item. The source is never - /// disposed, because an abandoned handler may still hold its token. + /// caller's thread has seen the call stop or end: there is no timer, + /// no registration on the caller's token, and no pool work item + /// unless no thread can be created. The source is never disposed, + /// because an abandoned handler may still hold its token. /// private CancellationToken HandlerToken() { From 3d2f20c808d8908ec9467cf0f90e69ac3e2111f6 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 03:14:13 -0400 Subject: [PATCH 17/33] Record the .NET handler-token evidence: 82 offline and 87 live tests --- docs/EVIDENCE.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/EVIDENCE.md b/docs/EVIDENCE.md index 9b52808..6735c65 100644 --- a/docs/EVIDENCE.md +++ b/docs/EVIDENCE.md @@ -136,9 +136,9 @@ dependencies. | Lane | Result | |---|---| -| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 74 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 7 handler-token tests, 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline fires it; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | -| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 79 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 74 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | -| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); no guard on a cancelled handler token (`ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent`: the answer is sent); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched` | +| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 82 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 15 handler-token tests, 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline cancels the token; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. With a test clock, 36- and 48-hour lifetimes leave the token uncancelled a day and a minute in and one second before the deadline (`IsCancelled` false too), and cancel it once the clock reaches the deadline, with no late response; the cancellation came 2 to 4 ms after the clock reached it in these runs (an observation, not asserted). With every thread-pool thread and 40 more blocked and a probe work item queued behind them, the token is cancelled at a 1 s deadline, at the caller's cancellation, and at the end of a successful invocation before the probe runs. A seam that holds the worker's kill until the token's callback has run sees the callback first, at the deadline and at caller cancellation. Callbacks run on the host's `jacquard-handler-cancel` thread, not a pool thread; one registered after the cancellation runs on the handler's own thread. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | +| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 87 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 82 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | +| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched`. Against the source with the handler token's timer and pool work items (with the test clock seam copied in, and with the timer's first wait shortened to 100 ms to stand for the day it waits), both `ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore` cases fail (the token is cancelled with 12 and 24 hours left), and the three `UnderAStarvedThreadPool...` tests fail (the token is not cancelled within 10 s). Cancelling through a pool work item instead of the host's thread fails the three starvation tests; killing the worker before starting the cancellation fails both `TheTokenIsCancelledBeforeTheWorkerIsKilled` cases (and the measured delay rose to about 244 ms, the time the process-tree kill took). `ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent` now holds by order: the token is cancelled only after the caller's thread has stopped waiting for the answer | | Build | `TreatWarningsAsErrors`, nullable reference types, and the `8.0-recommended` analyzers are clean with both SDKs; the .NET 8 SDK builds and passes the net8.0 suite on its own | | Pin consistency | `dotnet/src/JacquardHost/Compat.cs` is generated, and the embedded kit copy matches the pinned digests and the Python copy (`scripts/core_pin.py check`); `tests/test_core_pin.py` covers drift in both, and the offline Python suite passes | | Distribution | `scripts/check_distribution.py` passes; the source archive carries the .NET sources, `dotnet/NOTICE`, and the kit copy | From b681efe01e41af5fa6320ed145ea6b29a4731d3d Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 03:44:51 -0400 Subject: [PATCH 18/33] Start the .NET handler token's cancellation before every worker kill The wait for a handler read the stop condition in its predicate and again in its fault check, so a deadline, caller cancellation, or disposal landing between the two reads (or between the post-handler stopping and fault checks) killed the worker before the token's cancellation started. The worker now gives a stop notice before every kill, from whichever thread kills it, and the host starts the cancellation from that notice. --- dotnet/src/JacquardHost/Carrier/Worker.cs | 40 +++-- dotnet/src/JacquardHost/Host.cs | 65 ++++++-- .../test/JacquardHost.Tests/KillOrderTests.cs | 145 ++++++++++++++++++ .../JacquardHost.Tests/StopNoticeTests.cs | 108 +++++++++++++ 4 files changed, 332 insertions(+), 26 deletions(-) create mode 100644 dotnet/test/JacquardHost.Tests/KillOrderTests.cs create mode 100644 dotnet/test/JacquardHost.Tests/StopNoticeTests.cs diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index f9625ce..55b5ec6 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -126,12 +126,28 @@ private Worker(Process process, long expires, CancellationToken cancel) internal static volatile Func? ClockHook; /// - /// Test seam: runs on the caller's thread when the worker's lifetime - /// ends by expiry or cancellation, before its process tree is killed. - /// Process-global; null outside tests. + /// Test seam: runs on the thread that is about to kill the worker's live + /// process tree (at expiry, cancellation, or disposal), after + /// and before the kill. Process-global; null + /// outside tests. /// internal static volatile Action? ExpireHook; + private volatile Action? stopNotice; + + /// + /// Runs whenever the worker's lifetime ends by expiry, cancellation, or + /// disposal, before its process tree is killed, on the thread that ends + /// it (the caller's thread, or the disposing thread). The host uses it to + /// start the cancellation of its handlers' token before any kill. It must + /// not throw or block, and may run more than once. + /// + internal Action? StopNotice + { + get => stopNotice; + set => stopNotice = value; + } + /// The lifetime clock: outside tests. private static long Now() => ClockHook is { } clock ? clock() : Stopwatch.GetTimestamp(); @@ -320,10 +336,9 @@ private void ThrowIfFault() private CarrierException FaultOr(CarrierException fallback) => FaultCore() ?? fallback; - /// Kills the worker's process tree and closes its input. + /// Ends the lifetime: kills the worker's process tree (after the stop notice) and closes its input. private void Expire() { - ExpireHook?.Invoke(); expired = true; KillTree(); CloseInputCore(); @@ -341,7 +356,10 @@ private bool HasExited() } } - /// Kills the worker and the descendants .NET finds, while the worker is alive. + /// + /// Kills the worker and the descendants .NET finds, while the worker is + /// alive. Every kill comes through here, after the stop notice. + /// private void KillTree() { if (process is null || HasExited()) @@ -349,6 +367,8 @@ private void KillTree() return; } + stopNotice?.Invoke(); + ExpireHook?.Invoke(); killed = true; KillQuietly(process); } @@ -672,9 +692,11 @@ public void WriteFrame(IReadOnlyDictionary frame, Limits limits) /// Waits until returns true, the deadline /// passes, the call is cancelled, or the worker is disposed, reading /// stderr meanwhile. is asked before each - /// check, so a condition that includes returns - /// before the worker is killed; the caller must then check - /// . + /// check, so a condition that includes usually + /// returns before the worker is killed; the caller must then check + /// . A stop that lands between the two (the deadline, + /// a cancellation, or a disposal) is seen by the check, which gives the + /// before it kills the worker. /// /// The lifetime ended first. public void Await(Func isDone) diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 29beb75..a820219 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -255,8 +255,10 @@ private sealed record Answer(Value? Value, FailureCategory Category, string? Mes private sealed class Call { private readonly RegistrySnapshot registry; + private readonly object stopSync = new(); private volatile bool ended; private CancellationTokenSource? handlerStop; + private bool stopRequested; private bool handlersStopped; internal Call(Worker worker, Session session, InvocationResult result, RegistrySnapshot registry) @@ -265,6 +267,10 @@ internal Call(Worker worker, Session session, InvocationResult result, RegistryS Session = session; Result = result; this.registry = registry; + + // Every kill of the worker, whichever check or thread sees the + // stop first, starts the token's cancellation before it. + worker.StopNotice = StopHandlers; } internal Worker Worker { get; } @@ -466,10 +472,12 @@ private Answer RunHandler(Handler handler, IReadOnlyList arguments) } HandlerThreadStartedHook?.Invoke(); - // The wait ends as soon as the call is stopping, before the - // worker is killed (which can take a while), so the handlers' + // The wait usually ends as soon as the call is stopping, before + // the worker is killed (which can take a while), so the handlers' // token is cancelled first; the fault check after the wait then - // kills the worker and reports why. + // kills the worker and reports why. A stop that lands between the + // wait's checks, or after it, is seen by a fault check, whose + // kill gives the worker's stop notice (StopHandlers) first. Worker.Await(() => completion.Answer is not null || Worker.Stopping); if (Worker.Stopping) { @@ -480,34 +488,57 @@ private Answer RunHandler(Handler handler, IReadOnlyList arguments) /// /// The handlers' token, created on the caller's thread for the first - /// handler. Only cancels it, once the - /// caller's thread has seen the call stop or end: there is no timer, - /// no registration on the caller's token, and no pool work item - /// unless no thread can be created. The source is never disposed, + /// handler. Only cancels it, once the call + /// has been seen to stop or end: there is no timer, no registration + /// on the caller's token, and no pool work item unless no thread can + /// be created. A token created after the stop was seen starts + /// cancelled (no callback can be registered on it yet, so none runs + /// here); its handler does not start. The source is never disposed, /// because an abandoned handler may still hold its token. /// private CancellationToken HandlerToken() { - handlerStop ??= new CancellationTokenSource(); - return handlerStop.Token; + lock (stopSync) + { + if (handlerStop is null) + { + handlerStop = new CancellationTokenSource(); + if (stopRequested) + { + handlersStopped = true; + handlerStop.Cancel(); + } + } + + return handlerStop.Token; + } } /// - /// Called only on the caller's thread, when it sees the call stopping - /// while a handler runs and at : starts, at most + /// Called when the caller's thread sees the call stopping while a + /// handler runs, at , and through the worker's + /// before any kill (on the caller's + /// thread, or on a thread disposing the worker): starts, at most /// once, a dedicated background thread that cancels the handlers' - /// token and discards whatever its callbacks throw. Nothing waits for that thread, so a callback that - /// blocks holds only it, and the shared thread pool is not involved. + /// token and discards whatever its callbacks throw. Nothing waits for + /// that thread, so a callback that blocks holds only it, and the + /// shared thread pool is not involved. /// private void StopHandlers() { - CancellationTokenSource? source = handlerStop; - if (source is null || handlersStopped) + CancellationTokenSource? source; + lock (stopSync) { - return; + stopRequested = true; + source = handlerStop; + if (source is null || handlersStopped) + { + return; + } + + handlersStopped = true; } - handlersStopped = true; var thread = new Thread(static state => CancelQuietly((CancellationTokenSource)state!)) { IsBackground = true, diff --git a/dotnet/test/JacquardHost.Tests/KillOrderTests.cs b/dotnet/test/JacquardHost.Tests/KillOrderTests.cs new file mode 100644 index 0000000..96b3475 --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/KillOrderTests.cs @@ -0,0 +1,145 @@ +using System; +using System.Diagnostics; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// +/// Every kill of a worker while a handler may be running comes after the +/// cancellation of the handler's token has started, whichever check sees the +/// stop first. The wait for a handler reads the stop condition twice per +/// slice (its own predicate, then the fault check), and the call checks again +/// after the handler returns; these tests use the clock seam to land the +/// stop exactly between two such reads, so the outcome does not depend on +/// timing. +/// +[TestFixture] +public sealed class KillOrderTests +{ + /// How long the kill seam waits for the token's callback; the old order never runs it before the kill. + private static readonly TimeSpan CallbackWait = TimeSpan.FromSeconds(3); + + [TearDown] + public void ClearSeams() + { + Worker.ClockHook = null; + Worker.ExpireHook = null; + Host.HandlerThreadStartedHook = null; + } + + /// What the stop is, and when it lands. + public enum Stop + { + /// The deadline passes between the wait's predicate and its fault check. + DeadlineInTheWait, + + /// The caller's token is cancelled between the wait's predicate and its fault check. + CallerCancelInTheWait, + + /// The handler has returned; the deadline passes between the call's stopping check and its fault check. + DeadlineAfterTheHandler, + } + + /// + /// A lifetime clock that, once armed on the caller's thread, answers that + /// caller's first read with the real time and every later read (on any + /// thread) with a time two hours on, past the one-hour deadline. With the + /// cancel action set, the first read cancels the caller's token instead, + /// after the stopping check has read it and before the fault check does. + /// + private sealed class SplitClock + { + private static readonly long Jump = (long)(2 * 3600.0 * Stopwatch.Frequency); + private readonly Action? cancelAtFirstRead; + private int caller = -1; + private int reads; + private readonly bool jumpAfterFirstRead; + + internal SplitClock(Action? cancelAtFirstRead, bool jumpAfterFirstRead) + { + this.cancelAtFirstRead = cancelAtFirstRead; + this.jumpAfterFirstRead = jumpAfterFirstRead; + } + + /// Arms the clock; called on the caller's thread. + internal void Arm() => Volatile.Write(ref caller, Environment.CurrentManagedThreadId); + + internal long Now() + { + long real = Stopwatch.GetTimestamp(); + int armed = Volatile.Read(ref caller); + if (armed < 0) + { + return real; + } + + if (Environment.CurrentManagedThreadId == armed && Interlocked.Increment(ref reads) == 1) + { + cancelAtFirstRead?.Invoke(); + return real; + } + + return jumpAfterFirstRead && Volatile.Read(ref reads) >= 1 ? real + Jump : real; + } + } + + [TestCase(Stop.DeadlineInTheWait)] + [TestCase(Stop.CallerCancelInTheWait)] + [TestCase(Stop.DeadlineAfterTheHandler)] + public void TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks(Stop stop) + { + using var cancel = new CancellationTokenSource(); + using var registered = new ManualResetEventSlim(); + using var callback = new ManualResetEventSlim(); + bool returnsAtOnce = stop == Stop.DeadlineAfterTheHandler; + var clock = new SplitClock(stop == Stop.CallerCancelInTheWait ? cancel.Cancel : null, stop != Stop.CallerCancelInTheWait); + int kills = 0; + bool callbackBeforeKill = false; + Worker.ExpireHook = () => + { + if (Interlocked.Increment(ref kills) == 1) + { + Volatile.Write(ref callbackBeforeKill, callback.Wait(CallbackWait)); + } + }; + + // The caller's thread arms the clock only once the handler has + // registered its callback (and, for the post-handler case, returned), + // so the caller's next clock read is the stopping check. + Host.HandlerThreadStartedHook = () => + { + Assert.That(registered.Wait(TimeSpan.FromSeconds(10)), Is.True, "the handler did not start"); + if (returnsAtOnce) + { + Thread.Sleep(100); + } + + clock.Arm(); + }; + Worker.ClockHook = clock.Now; + InvocationResult result = Within(TimeSpan.FromSeconds(30), () => Replaying("one-once-effect-success", TimeSpan.FromHours(1)).Invoke( + Pinned.Identity("kit-echo"), + Echo(), + new Value[] { new TextValue("ping") }, + new OperationRegistry().Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), (context, args) => + { + context.CancellationToken.Register(callback.Set); + registered.Set(); + if (!returnsAtOnce) + { + context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(15)); + } + + return new TextValue("pong"); + }), + cancel.Token)); + Assert.That(Reason(result), Is.EqualTo(stop == Stop.CallerCancelInTheWait ? "aborted" : "deadline"), result.ToString()); + Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a response was sent after the stop"); + Assert.That(Volatile.Read(ref kills), Is.GreaterThanOrEqualTo(1), "the worker was not killed"); + Assert.That(Volatile.Read(ref callbackBeforeKill), Is.True, "the worker was killed before the token's cancellation started"); + } +} diff --git a/dotnet/test/JacquardHost.Tests/StopNoticeTests.cs b/dotnet/test/JacquardHost.Tests/StopNoticeTests.cs new file mode 100644 index 0000000..9db13fe --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/StopNoticeTests.cs @@ -0,0 +1,108 @@ +using System; +using System.Collections.Concurrent; +using System.Diagnostics; +using System.Linq; +using System.Threading; +using JacquardLang.Host.Carrier; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// +/// The worker gives its stop notice (through which the host starts the +/// cancellation of its handlers' token) before every kill, including a kill +/// by disposal: one from another thread while an operation waits, and one +/// with no operation in progress. Within the worker is +/// disposed only after the invocation has ended, which starts the +/// cancellation itself; these tests cover the worker on its own. +/// +[TestFixture] +public sealed class StopNoticeTests +{ + [TearDown] + public void ClearSeams() + { + Worker.ClockHook = null; + Worker.ExpireHook = null; + } + + private static void AssertNoticeBeforeEveryKill(ConcurrentQueue events) + { + string[] order = events.ToArray(); + Assert.That(order, Does.Contain("kill"), "the worker was not killed"); + int notices = 0; + foreach (string step in order) + { + if (step == "notice") + { + notices++; + } + else + { + Assert.That(notices, Is.GreaterThan(0), "a kill came before any stop notice: " + string.Join(", ", order)); + } + } + } + + [Test] + public void DisposalWithNoOperationGivesTheNoticeBeforeTheKill() + { + var events = new ConcurrentQueue(); + Worker worker = Worker.Start("unused", Standin("silent", string.Empty, TimeSpan.FromSeconds(30)), CancellationToken.None); + worker.StopNotice = () => events.Enqueue("notice"); + Worker.ExpireHook = () => events.Enqueue("kill"); + worker.Dispose(); + AssertNoticeBeforeEveryKill(events); + } + + [Test] + public void DisposalFromAnotherThreadBetweenTwoChecksGivesTheNoticeBeforeTheKill() + { + // The waiting thread's first clock read comes from its stopping + // check, after that check has read the disposal flag. The clock seam + // then starts the disposal on another thread and returns only once + // the flag is set, so the next check, the fault check, sees it. + var events = new ConcurrentQueue(); + Worker worker = Worker.Start("unused", Standin("silent", string.Empty, TimeSpan.FromSeconds(30)), CancellationToken.None); + worker.StopNotice = () => events.Enqueue("notice"); + Worker.ExpireHook = () => events.Enqueue("kill"); + int waiter = -1; + int reads = 0; + Thread? disposer = null; + Worker.ClockHook = () => + { + if (Environment.CurrentManagedThreadId == Volatile.Read(ref waiter) && Interlocked.Increment(ref reads) == 1) + { + disposer = new Thread(worker.Dispose) { IsBackground = true, Name = "test-disposer" }; + disposer.Start(); + var watch = Stopwatch.StartNew(); + while (!worker.Stopping && watch.Elapsed < TimeSpan.FromSeconds(10)) + { + Thread.Yield(); + } + } + + return Stopwatch.GetTimestamp(); + }; + + CarrierException? thrown = Within(TimeSpan.FromSeconds(20), () => + { + Volatile.Write(ref waiter, Environment.CurrentManagedThreadId); + try + { + worker.Await(() => worker.Stopping); + return null; + } + catch (CarrierException error) + { + return error; + } + }); + Assert.That(disposer, Is.Not.Null, "the clock seam did not start the disposal"); + Assert.That(disposer!.Join(TimeSpan.FromSeconds(10)), Is.True, "the disposal did not finish"); + Assert.That(thrown, Is.Not.Null, "the wait ended on its predicate, before the race"); + Assert.That(thrown!.IsAborted, Is.True, thrown.Message); + AssertNoticeBeforeEveryKill(events); + } +} From 52eb5de6cfa3e57be5aff49c68c0b564f9ee99bf Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 03:47:54 -0400 Subject: [PATCH 19/33] Never let starting the .NET token cancellation escape Invoke or skip disposal Starting the cancellation of the handlers' token now catches any failure to create or start its thread, falls back to a thread-pool work item, and if that fails too records InvocationResult.HandlerTokenNotCancelled and gives up; it never runs callbacks on the calling thread. The worker is disposed in a nested finally, so it is released even if ending the call fails. --- dotnet/src/JacquardHost/Host.cs | 70 +++++-- dotnet/src/JacquardHost/InvocationResult.cs | 10 + .../JacquardHost.Tests/CancelStartTests.cs | 175 ++++++++++++++++++ 3 files changed, 244 insertions(+), 11 deletions(-) create mode 100644 dotnet/test/JacquardHost.Tests/CancelStartTests.cs diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index a820219..7cc1a22 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -92,6 +92,15 @@ public Host(string store, HostOptions? options = null) /// internal static volatile Action? HandlerCheckHook; + /// + /// Test seam: runs before the host starts the cancellation of the + /// handlers' token, with "thread" before it creates its dedicated + /// thread and with "pool" before it falls back to a thread-pool + /// work item. A test throws from it to stand for a failure to start + /// either. Null outside tests. + /// + internal static volatile Action? CancelStartHook; + /// /// Runs with . The invoke /// frame is validated before a worker starts. Cancelling @@ -203,9 +212,16 @@ private InvocationResult Run(Func opening, RegistrySnapshot finally { // The handlers' token is cancelled before the worker is disposed, - // which may wait up to a second for a killed worker to exit. - call.End(); - worker.Dispose(); + // which may wait up to a second for a killed worker to exit. The + // worker is disposed even if ending the call fails. + try + { + call.End(); + } + finally + { + worker.Dispose(); + } } result.Requests = session.Requests.ToArray(); @@ -539,21 +555,53 @@ private void StopHandlers() handlersStopped = true; } - var thread = new Thread(static state => CancelQuietly((CancellationTokenSource)state!)) + if (!StartCancellation(source)) { - IsBackground = true, - Name = "jacquard-handler-cancel", - }; + Result.HandlerTokenNotCancelled = true; + } + } + + /// + /// Starts the cancellation of on a new + /// background thread or, if no thread can be created or started, on + /// a thread-pool work item; callbacks never run on the calling + /// thread. Returns false, and leaves the token uncancelled, when + /// neither could be started. Never throws: it runs inside the + /// worker's fault checks and at the end of . + /// + [SuppressMessage("Design", "CA1031", Justification = "Starting the cancellation must not throw into a fault check or out of Invoke; a failure falls back, then is recorded.")] + private static bool StartCancellation(CancellationTokenSource source) + { try { + CancelStartHook?.Invoke("thread"); + var thread = new Thread(static state => CancelQuietly((CancellationTokenSource)state!)) + { + IsBackground = true, + Name = "jacquard-handler-cancel", + }; thread.Start(source); + return true; } - catch (OutOfMemoryException) + catch (Exception) { - // No thread could be created. Fall back to the pool rather - // than run handler callbacks on the caller's thread. - ThreadPool.UnsafeQueueUserWorkItem(static s => CancelQuietly(s), source, preferLocal: false); + // No thread could be created or started (for example + // OutOfMemoryException). Fall back to the pool rather than + // run handler callbacks on this thread. } + + try + { + CancelStartHook?.Invoke("pool"); + return ThreadPool.UnsafeQueueUserWorkItem(static s => CancelQuietly(s), source, preferLocal: false); + } + catch (Exception) + { + // The pool refused the work item as well (for example + // NotSupportedException or OutOfMemoryException): give up. + } + + return false; } /// Cancels a source, discarding whatever its callbacks throw. diff --git a/dotnet/src/JacquardHost/InvocationResult.cs b/dotnet/src/JacquardHost/InvocationResult.cs index fea707f..f0e6e7d 100644 --- a/dotnet/src/JacquardHost/InvocationResult.cs +++ b/dotnet/src/JacquardHost/InvocationResult.cs @@ -89,6 +89,16 @@ internal InvocationResult(ResultKind kind) /// The host's evidence (jacquard-host-dotnet-evidence-v0); empty on success. public JsonObject HostEvidence { get; internal set; } = JsonObject.Empty; + /// + /// True when the host could start neither a thread nor a thread-pool work + /// item to cancel , for + /// example when the process is out of memory. That token is then never + /// cancelled: a handler waiting only on it keeps waiting (until its own + /// timeout, if it has one), while + /// still turns true. False otherwise. + /// + public bool HandlerTokenNotCancelled { get; internal set; } + /// Whether Core returned a successful outcome. public bool Ok => Kind == ResultKind.Ok; diff --git a/dotnet/test/JacquardHost.Tests/CancelStartTests.cs b/dotnet/test/JacquardHost.Tests/CancelStartTests.cs new file mode 100644 index 0000000..7c2db61 --- /dev/null +++ b/dotnet/test/JacquardHost.Tests/CancelStartTests.cs @@ -0,0 +1,175 @@ +using System; +using System.Globalization; +using System.IO; +using System.Threading; +using JacquardLang.Host.Carrier; +using JacquardLang.Host.Protocol; +using NUnit.Framework; +using static JacquardLang.Host.Tests.Support; + +namespace JacquardLang.Host.Tests; + +/// +/// When the host cannot start a thread to cancel the handlers' token it falls +/// back to a thread-pool work item; when that fails too it records the +/// failure and gives up. Either way returns its +/// result, nothing runs a callback on the caller's thread, and the worker is +/// disposed. A seam stands for the failures, as an out-of-memory process +/// would produce them. +/// +[TestFixture] +[Platform("Linux")] +public sealed class CancelStartTests +{ + [TearDown] + public void ClearSeams() => Host.CancelStartHook = null; + + /// Live child processes of this process, from /proc. + private static int Children() + { + string self = Environment.ProcessId.ToString(CultureInfo.InvariantCulture); + int count = 0; + foreach (string dir in Directory.GetDirectories("/proc")) + { + if (!int.TryParse(Path.GetFileName(dir), NumberStyles.None, CultureInfo.InvariantCulture, out _)) + { + continue; + } + + try + { + string stat = File.ReadAllText(Path.Combine(dir, "stat")); + string[] fields = stat[(stat.LastIndexOf(')') + 2)..].Split(' '); + if (fields[1] == self && fields[0] != "Z") + { + count++; + } + } + catch (IOException) + { + // The process ended while it was listed. + } + catch (UnauthorizedAccessException) + { + // Not ours. + } + } + + return count; + } + + private static void Fail(bool pool, string where) + { + if (where == "thread") + { + throw new OutOfMemoryException("seam: no thread"); + } + + if (pool) + { + throw new NotSupportedException("seam: no pool work item"); + } + } + + private sealed class Observed + { + internal readonly ManualResetEventSlim Done = new(); + internal int CallerThread = -1; + internal int CallbackThread = -1; + internal bool CallbackOnPool; + internal bool TokenCancelled = true; + internal bool FlaggedCancelled; + } + + private static InvocationResult InvokeWith(Observed seen, TimeSpan deadline, bool waits, out Exception? thrown) + { + InvocationResult? result = null; + Exception? failure = null; + Host host = Replaying("one-once-effect-success", deadline); + var registry = new OperationRegistry().Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), (context, args) => + { + context.CancellationToken.Register(() => + { + Volatile.Write(ref seen.CallbackThread, Environment.CurrentManagedThreadId); + Volatile.Write(ref seen.CallbackOnPool, Thread.CurrentThread.IsThreadPoolThread); + }); + if (waits) + { + Volatile.Write(ref seen.TokenCancelled, context.CancellationToken.WaitHandle.WaitOne(TimeSpan.FromSeconds(3))); + Volatile.Write(ref seen.FlaggedCancelled, context.IsCancelled); + seen.Done.Set(); + } + + return new TextValue("pong"); + }); + Within(TimeSpan.FromSeconds(20), () => + { + Volatile.Write(ref seen.CallerThread, Environment.CurrentManagedThreadId); + try + { + result = host.Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, registry); + } + catch (Exception error) + { + failure = error; + } + + return 0; + }); + thrown = failure; + return result!; + } + + [Test] + public void WithoutAThreadTheTokenIsCancelledOnAPoolThread() + { + Host.CancelStartHook = where => Fail(pool: false, where); + var seen = new Observed(); + InvocationResult result = InvokeWith(seen, TimeSpan.FromSeconds(1), waits: true, out Exception? thrown); + Assert.That(thrown, Is.Null, "Invoke threw"); + Assert.That(seen.Done.Wait(TimeSpan.FromSeconds(10)), Is.True, "the handler did not return"); + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(result.HandlerTokenNotCancelled, Is.False); + Assert.That(Volatile.Read(ref seen.TokenCancelled), Is.True, "the token was not cancelled"); + Assert.That(Volatile.Read(ref seen.CallbackOnPool), Is.True, "the callback did not run on a pool thread"); + Assert.That(Volatile.Read(ref seen.CallbackThread), Is.Not.EqualTo(Volatile.Read(ref seen.CallerThread))); + } + + [TestCase(true)] + [TestCase(false)] + public void WithNeitherAThreadNorAPoolWorkItemInvokeReturnsAndDisposesTheWorker(bool atTheDeadline) + { + Assert.That(InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))).Ok, Is.True); // warm up lazily opened files + int before = Descriptors(); + Assert.That(Eventually(() => Children() == 0), Is.True, "a worker from an earlier test is still running"); + int attempts = 0; + Host.CancelStartHook = where => + { + Interlocked.Increment(ref attempts); + Fail(pool: true, where); + }; + var seen = new Observed(); + InvocationResult result = InvokeWith(seen, atTheDeadline ? TimeSpan.FromSeconds(1) : TimeSpan.FromSeconds(20), waits: atTheDeadline, out Exception? thrown); + Assert.Multiple((Action)(() => + { + Assert.That(thrown, Is.Null, "Invoke threw: " + thrown); + Assert.That(Eventually(() => Children() == 0), Is.True, "the worker was not disposed: it is still running"); + Assert.That(Eventually(() => Descriptors() <= before), Is.True, "the worker was not disposed: descriptors grew from " + before + " to " + Descriptors()); + })); + Assert.That(Volatile.Read(ref attempts), Is.EqualTo(2), "the host did not try a thread and then the pool, once"); + Assert.That(result.HandlerTokenNotCancelled, Is.True, "the failure was not recorded"); + if (atTheDeadline) + { + Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); + Assert.That(seen.Done.Wait(TimeSpan.FromSeconds(10)), Is.True, "the handler did not return"); + Assert.That(Volatile.Read(ref seen.TokenCancelled), Is.False, "the token was cancelled"); + Assert.That(Volatile.Read(ref seen.FlaggedCancelled), Is.True, "IsCancelled did not report the stop"); + } + else + { + Assert.That(result.Ok, Is.True, result.ToString()); + } + + Assert.That(Volatile.Read(ref seen.CallbackThread), Is.EqualTo(-1), "a callback ran"); + } +} From 1d73c539dd7cf0806f2919c12cf155251758a095 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 03:48:07 -0400 Subject: [PATCH 20/33] Keep the .NET thread count in the resource test to the current invocation --- dotnet/test/JacquardHost.Tests/DescriptorTests.cs | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs index 9a561d5..fdb76f9 100644 --- a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs +++ b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs @@ -14,7 +14,10 @@ namespace JacquardLang.Host.Tests; /// /// Open descriptors, adapter threads, and the heap across invocations and /// failures, read from /proc. The adapter starts no thread of its own -/// except one per handler call, named jacquard-handler. +/// except one per handler call, named jacquard-handler, and one short-lived +/// jacquard-handler-cancel per invocation that gave a handler a token. +/// Both names read as jacquard-handle in /proc (15 bytes), so +/// the counts include both. /// [TestFixture] [Platform("Linux")] @@ -48,7 +51,10 @@ public void InvocationsReleaseTheirPipesAndThreads() Assert.That(threads, Is.EqualTo(0), "escaped descendant: adapter threads remain"); // Handlers run on their own threads, which end with the handler; the - // count sees a handler thread while one runs. + // count sees a handler thread while one runs. Each invocation then + // starts a cancel thread as it ends; the loop waits for it to finish + // before the next invocation, so the handler counts only its own + // thread. int during = -1; OperationRegistry registry = new OperationRegistry().Bind(Pinned.Identity("kit-world"), Pinned.Identity("send"), (context, args) => { @@ -58,6 +64,7 @@ public void InvocationsReleaseTheirPipesAndThreads() Host replaying = Replaying("one-once-effect-success"); for (int i = 0; i < 3; i++) { + Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "an earlier invocation's threads remain"); InvocationResult result = replaying.Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, registry); Assert.That(result.Ok, Is.True, result.ToString()); } From 6f96b93ebca5f36ed8de65f64cfae598b1cb5905 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 04:16:56 -0400 Subject: [PATCH 21/33] Describe the .NET 5 ms poll as a requested timeout and document the cancellation fallbacks The interval between the caller's checks can exceed 5 ms (stderr drains and scheduling), so the docs now call 5 ms the requested poll timeout rather than a wall-clock guarantee. They also state that every kill comes after the token's cancellation starts, and what happens when no thread, and then no pool work item, can be started. --- docs/LIMITS.md | 4 ++-- dotnet/README.md | 29 +++++++++++++++-------- dotnet/src/JacquardHost/Carrier/Worker.cs | 6 +++-- dotnet/src/JacquardHost/Handlers.cs | 24 ++++++++++++------- 4 files changed, 40 insertions(+), 23 deletions(-) diff --git a/docs/LIMITS.md b/docs/LIMITS.md index 88e679a..469a02f 100644 --- a/docs/LIMITS.md +++ b/docs/LIMITS.md @@ -27,8 +27,8 @@ also updates this document. | Java fulfillment is a single-process snapshot example | Atomic JSON replacement demonstrates booking/outbox recovery and deduplication, without cross-process locking or power-loss durability evidence. | A concrete production storage contract and crash/concurrent-writer evidence. | | Java platforms | Linux runs in CI. macOS has no CI; the offline descendant and fake-Core tests need `sh` and `sleep`, and the descriptor tests need `/proc`. Windows is not claimed. | CI, lifecycle, and cleanup evidence on each added platform. | | .NET worker process tree, not group | .NET cannot start the worker in its own process group, so the .NET host kills a process tree. At expiry, cancellation, or disposal of a running worker, it calls `Process.Kill(true)`, which stops the worker, lists its children from the process table, kills it, and does the same for each child. A descendant that was already reparented keeps running: one whose parent exited, or one that daemonized. So does one started after its parent was listed. Descendants are found and signalled by pid, with nothing pinning their identity: one that exits, and is reaped once the worker is killed, between the listing and the signal can have its pid reused by an unrelated process, which is then signalled (the Java host signals process handles, and the Rust and Go hosts a process group). Listing the process table takes time proportional to the number of processes on the machine. The runtime reaps the worker as soon as it exits. A descendant still holding stdout or stderr after the exit is detected and, after a one-second grace, reported as a carrier loss, as the Go and Rust hosts report it; it is not stopped. .NET reports a signal as exit status 128 + its number, so only the host's own SIGKILL is reported as a signal. | An OS-level containment mechanism (for example a cgroup or job object) with its own evidence, or a .NET process-group API. | -| .NET pipe I/O goes through `poll(2)` | .NET's pipe streams have no read or write that a deadline or a cancellation can interrupt, so the .NET host never calls one that can block: the caller's thread asks the C library's `poll(2)` whether a pipe is ready, in waits of at most 5 ms that recheck the deadline, the `CancellationToken`, and disposal, and then reads only bytes already in the pipe or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a writable pipe accepts at once. No thread is left blocked on a worker pipe, even while an escaped descendant holds stdin without reading or floods stdout or stderr. The `pollfd` layout and event bits are the same on Linux and macOS; elsewhere, including Windows, the host does not start a worker and reports a spawn failure. .NET opens its own descriptors close-on-exec, but passes the worker any descriptor the host process itself inherited without that flag. | A .NET API for interruptible or non-blocking process pipes, and CI on each added platform. | -| .NET handler cancellation is cooperative | Each .NET handler runs on its own background thread. At the deadline or on cancellation the host kills the worker's process tree, closes its input, and sends no late response. `HandlerContext.IsCancelled` turns true (it reads the deadline exactly), and stays true once the invocation ends. `HandlerContext.CancellationToken` is cancelled when the call stops or ends, never earlier (a lifetime of any length does not cancel it before its deadline); the caller's thread, which checks the deadline and the caller's token at least every 5 ms while a handler runs, starts its cancellation when it sees either, before it kills the worker, and otherwise shortly after the invocation ends. The cancellation runs on a short-lived thread the host starts for it, one per invocation that gave a handler a token, so it does not wait for the shared thread pool (if no thread can be created, it falls back to a pool work item). Callbacks a handler registers on that token run on that thread, which the host never waits for; a callback that blocks holds that thread, and anything a callback throws is discarded. A callback registered after the token is cancelled runs at once on the handler's own thread and throws there, as with any `CancellationToken`. A handler that ignores both keeps running on its thread until it returns. No outside action is stopped or rolled back. A stack overflow in a handler ends the process, as it does any .NET code. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | +| .NET pipe I/O goes through `poll(2)` | .NET's pipe streams have no read or write that a deadline or a cancellation can interrupt, so the .NET host never calls one that can block: the caller's thread asks the C library's `poll(2)` whether a pipe is ready, with a poll timeout of at most 5 ms, rechecking the deadline, the `CancellationToken`, and disposal between polls, and then reads only bytes already in the pipe or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a writable pipe accepts at once. No thread is left blocked on a worker pipe, even while an escaped descendant holds stdin without reading or floods stdout or stderr. The `pollfd` layout and event bits are the same on Linux and macOS; elsewhere, including Windows, the host does not start a worker and reports a spawn failure. .NET opens its own descriptors close-on-exec, but passes the worker any descriptor the host process itself inherited without that flag. | A .NET API for interruptible or non-blocking process pipes, and CI on each added platform. | +| .NET handler cancellation is cooperative | Each .NET handler runs on its own background thread. At the deadline or on cancellation the host kills the worker's process tree, closes its input, and sends no late response. `HandlerContext.IsCancelled` turns true (it reads the deadline exactly), and stays true once the invocation ends. `HandlerContext.CancellationToken` is cancelled when the call stops or ends, never earlier (a lifetime of any length does not cancel it before its deadline). While a handler runs, the caller's thread rechecks the deadline and the caller's token between poll waits of at most 5 ms; that is the requested poll timeout, not a bound on the interval between checks, which reading pending stderr (up to 1 MiB per pass) and thread scheduling can lengthen. Whichever check sees the stop, the token's cancellation starts before the worker is killed; otherwise it starts shortly after the invocation ends. The cancellation runs on a short-lived thread the host starts for it, one per invocation that gave a handler a token, so it does not wait for the shared thread pool. If no thread can be created or started, the host queues it as a pool work item, which runs only when a pool thread is free, so a starved pool delays it. If the work item cannot be queued either, the host gives up: the token is never cancelled (`HandlerContext.IsCancelled` still turns true), `InvocationResult.HandlerTokenNotCancelled` is set, and a handler waiting only on the token keeps waiting; `Invoke` still returns its result and disposes the worker. Callbacks a handler registers on that token run on that thread, which the host never waits for; a callback that blocks holds that thread, and anything a callback throws is discarded. A callback registered after the token is cancelled runs at once on the handler's own thread and throws there, as with any `CancellationToken`. A handler that ignores both keeps running on its thread until it returns. No outside action is stopped or rolled back. A stack overflow in a handler ends the process, as it does any .NET code. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | | .NET adapter is protocol-level only | .NET callers pass exact identities, boundary types, and protocol values to a prepared store; there is no .NET SDK for program preparation, native type mapping, or caching, and `Invoke` is synchronous. Linux runs in CI on .NET 8 and 10. macOS has no CI; the offline descendant tests need `sh`, `sleep`, and `yes`, and the descriptor tests need `/proc`. Windows is not supported. | A .NET SDK with typed-binding and preparation evidence comparable to the other SDKs, and CI on each added platform. | | Node handler cancellation is cooperative | The Node adapter can expire while awaiting a Promise, signal the handler, and kill the worker; it cannot stop an outside action or preempt JavaScript that blocks the event loop. No late response or automatic retry follows expiration. | A separately enforced operation lifetime with explicit completion and reconciliation evidence. | | Socket host closes without a response on its own failures | A ceiling violation, early disconnect, or failed invocation ends the connection silently; clients see a reset, not an HTTP error. | An owner decision that the host may emit fixed minimal status lines, recorded here with the exact bytes and the rule that they carry no application meaning. | diff --git a/dotnet/README.md b/dotnet/README.md index 9e85494..9a13aac 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -136,15 +136,23 @@ as soon as the call is cancelled or its lifetime passes (it reads the deadline exactly), and stays true once the invocation ends. `HandlerContext.CancellationToken` is cancelled when the call stops or ends, and never earlier: however long the lifetime, the deadline alone does not -cancel it before it passes. While a handler runs, the caller's thread checks -the deadline and the caller's token at least every 5 ms; when it sees -either, it starts the token's cancellation before it kills the worker. -Otherwise the token is cancelled shortly after the invocation ends. The host -cancels it on a short-lived thread of its own, one per invocation, so a busy -thread pool does not delay it. Callbacks registered on the token run on that -thread, never on the caller's thread or the thread that cancelled the -caller's token; the host does not wait for them, and anything they throw is -discarded. A callback that blocks holds only that thread. A callback +cancel it before it passes. While a handler runs, the caller's thread +rechecks the deadline and the caller's token between poll waits of at most +5 ms. That is the requested poll timeout, not a bound on the interval: +reading pending stderr (up to 1 MiB per pass) and thread scheduling can +lengthen it. Whichever check sees the stop, the token's cancellation starts +before the worker is killed. Otherwise the token is cancelled shortly after +the invocation ends. The host cancels it on a short-lived thread of its own, +one per invocation, so a busy thread pool does not delay it. If no thread can +be created, it queues the cancellation as a thread-pool work item instead, +which runs when a pool thread is free; if that cannot be queued either, the +host gives up: the token is never cancelled, `IsCancelled` still turns true, +and the result's `HandlerTokenNotCancelled` is set. `Invoke` still returns +its result and disposes the worker. Callbacks registered on the token run on +that thread (or that pool thread), never on the caller's thread or the thread +that cancelled the caller's token; the host does not wait for them, and +anything they throw is discarded. A callback that blocks holds only that +thread. A callback registered after the token is already cancelled runs at once on the registering (handler) thread, as with any `CancellationToken`, and its exception is thrown there. A handler that ignores both keeps running on its @@ -165,7 +173,8 @@ thread. The carrier is built around both: - **No read or write ever blocks.** The caller's thread asks the C library's `poll(2)` (the one native call the adapter makes) whether a pipe is ready, - in waits of at most 5 ms that never end after the deadline, and only then reads the bytes already in the pipe, + with a poll timeout of at most 5 ms that never ends after the deadline, and + only then reads the bytes already in the pipe, or writes at most `PIPE_BUF` bytes (4096 on Linux, 512 on macOS), which a pipe that poll reports writable accepts at once. Every wait, whether for a frame, a write, a handler, or the exit, rechecks the one lifetime deadline, diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index 55b5ec6..8d1974a 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -20,7 +20,9 @@ namespace JacquardLang.Host.Carrier; /// interrupt (on Unix an anonymous pipe's asynchronous read is a blocking /// read on a pool thread). So the carrier never calls a read or write that /// can block. It asks the C library's poll(2) whether a pipe is ready, -/// on the caller's thread, in waits of at most 5 ms, and only then reads the +/// on the caller's thread, with a poll timeout of at most 5 ms (a requested +/// timeout: reading pending stderr, up to 1 MiB per pass, and scheduling can +/// lengthen the interval between checks), and only then reads the /// bytes already in the pipe or writes at most PIPE_BUF bytes, which /// a writable pipe always accepts at once. Every wait, whether for a frame, a /// write, a handler, or the exit, rechecks the single lifetime deadline, the @@ -837,7 +839,7 @@ private void RecordExit() /// Kills a live worker's process tree, waits up to one second for its /// exit, reads the stderr left in the pipe, and closes the pipes. Safe /// from any thread: an operation in progress on another thread sees the - /// disposal within one poll slice and ends first. + /// disposal at its next check and ends first. /// public void Dispose() { diff --git a/dotnet/src/JacquardHost/Handlers.cs b/dotnet/src/JacquardHost/Handlers.cs index 93723e7..acd19ae 100644 --- a/dotnet/src/JacquardHost/Handlers.cs +++ b/dotnet/src/JacquardHost/Handlers.cs @@ -39,15 +39,21 @@ internal HandlerContext(Func stopping, Func ended, CancellationToken /// A token cancelled when the invocation stops or ends, and never /// earlier: the lifetime alone does not cancel it before its deadline, /// however long it is. While the handler runs, the caller's thread - /// checks the deadline and the caller's token at least every 5 ms; when - /// it sees either, it starts the cancellation before it kills the worker. - /// Otherwise the token is cancelled shortly after the invocation ends. - /// is exact; the token follows it. - /// The host cancels the token on a dedicated thread of its own, not a - /// thread-pool thread, so callbacks registered on it run there, never on - /// the caller's thread or the thread that cancelled the caller's token; - /// the host does not wait for them, and any exception they throw is - /// discarded. A callback registered after the token is cancelled runs at + /// rechecks the deadline and the caller's token between poll waits of at + /// most 5 ms (the requested poll timeout: reading pending stderr and + /// thread scheduling can lengthen the interval between checks). Whichever + /// check sees the stop, the token's cancellation starts before the + /// worker is killed. Otherwise the token is cancelled shortly after the + /// invocation ends. is exact; the token + /// follows it. The host cancels the token on a dedicated thread of its + /// own, not a thread-pool thread, so callbacks registered on it run + /// there, never on the caller's thread or the thread that cancelled the + /// caller's token; the host does not wait for them, and any exception + /// they throw is discarded. If no thread can be created, the host queues + /// the cancellation as a thread-pool work item instead, which runs when a + /// pool thread is free; if that fails too, the token is never cancelled + /// and is set, + /// while still turns true. A callback registered after the token is cancelled runs at /// once on the registering (handler) thread, as for any /// , and its exception is /// thrown there. From 4e32b9ffe829c3440a9597bf2b1b470092451e6b Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 04:16:56 -0400 Subject: [PATCH 22/33] Record the .NET kill-order and cancellation-fallback evidence: 90 offline and 95 live tests --- docs/EVIDENCE.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/EVIDENCE.md b/docs/EVIDENCE.md index 6735c65..af6b3ba 100644 --- a/docs/EVIDENCE.md +++ b/docs/EVIDENCE.md @@ -136,9 +136,9 @@ dependencies. | Lane | Result | |---|---| -| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 82 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 15 handler-token tests, 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline cancels the token; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. With a test clock, 36- and 48-hour lifetimes leave the token uncancelled a day and a minute in and one second before the deadline (`IsCancelled` false too), and cancel it once the clock reaches the deadline, with no late response; the cancellation came 2 to 4 ms after the clock reached it in these runs (an observation, not asserted). With every thread-pool thread and 40 more blocked and a probe work item queued behind them, the token is cancelled at a 1 s deadline, at the caller's cancellation, and at the end of a successful invocation before the probe runs. A seam that holds the worker's kill until the token's callback has run sees the callback first, at the deadline and at caller cancellation. Callbacks run on the host's `jacquard-handler-cancel` thread, not a pool thread; one registered after the cancellation runs on the handler's own thread. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | -| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 87 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 82 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | -| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched`. Against the source with the handler token's timer and pool work items (with the test clock seam copied in, and with the timer's first wait shortened to 100 ms to stand for the day it waits), both `ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore` cases fail (the token is cancelled with 12 and 24 hours left), and the three `UnderAStarvedThreadPool...` tests fail (the token is not cancelled within 10 s). Cancelling through a pool work item instead of the host's thread fails the three starvation tests; killing the worker before starting the cancellation fails both `TheTokenIsCancelledBeforeTheWorkerIsKilled` cases (and the measured delay rose to about 244 ms, the time the process-tree kill took). `ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent` now holds by order: the token is cancelled only after the caller's thread has stopped waiting for the answer | +| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 90 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 15 handler-token tests, 3 kill-order tests, 2 stop-notice tests, 3 cancellation-start tests (Linux), 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline cancels the token; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. With a test clock, 36- and 48-hour lifetimes leave the token uncancelled a day and a minute in and one second before the deadline (`IsCancelled` false too), and cancel it once the clock reaches the deadline, with no late response; the cancellation came 2 to 4 ms after the clock reached it in these runs (an observation, not asserted). With every thread-pool thread and 40 more blocked and a probe work item queued behind them, the token is cancelled at a 1 s deadline, at the caller's cancellation, and at the end of a successful invocation before the probe runs. A seam that holds the worker's kill until the token's callback has run sees the callback first, at the deadline and at caller cancellation; with a test clock that lands the stop exactly between two reads of the stop condition (the deadline or the caller's cancellation between the handler wait's predicate and its fault check, and the deadline between the post-handler stopping check and its fault check), it still sees the callback first. The worker gives its stop notice before every kill, including disposal with no operation running and disposal from another thread landing between a wait's two checks. When the cancellation thread cannot be created (a seam throws `OutOfMemoryException`), the token is cancelled on a pool thread; when the pool work item fails too (`NotSupportedException`), `Invoke` returns its result (at a 1 s deadline, and on success), sets `HandlerTokenNotCancelled`, runs no callback, leaves the token uncancelled while `IsCancelled` is true, and disposes the worker (no child process or descriptor remains). Callbacks run on the host's `jacquard-handler-cancel` thread, not a pool thread; one registered after the cancellation runs on the handler's own thread. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | +| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 95 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 90 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | +| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched`. Against the source with the handler token's timer and pool work items (with the test clock seam copied in, and with the timer's first wait shortened to 100 ms to stand for the day it waits), both `ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore` cases fail (the token is cancelled with 12 and 24 hours left), and the three `UnderAStarvedThreadPool...` tests fail (the token is not cancelled within 10 s). Cancelling through a pool work item instead of the host's thread fails the three starvation tests; killing the worker before starting the cancellation fails both `TheTokenIsCancelledBeforeTheWorkerIsKilled` cases (and the measured delay rose to about 244 ms, the time the process-tree kill took). `ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent` now holds by order: the token is cancelled only after the caller's thread has stopped waiting for the answer. Against the source before the stop notice, all three `TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks` cases fail (the kill came first); removing the stop notice from the kill fails them and both `StopNoticeTests`. Letting the pool fallback's exception escape fails both `WithNeitherAThreadNorAPoolWorkItem...` cases (`Invoke` throws `NotSupportedException`). Repeating `TheTokenIsCancelledBeforeTheWorkerIsKilled` 150 times per case under the real clock passed every run; a reviewer's probe (an 800 ms deadline, or a caller cancellation after 100 to 150 ms, 150 runs each) found the kill first in 8 and 0 runs against the source before the stop notice on this machine, and in 0 and 0 after it | | Build | `TreatWarningsAsErrors`, nullable reference types, and the `8.0-recommended` analyzers are clean with both SDKs; the .NET 8 SDK builds and passes the net8.0 suite on its own | | Pin consistency | `dotnet/src/JacquardHost/Compat.cs` is generated, and the embedded kit copy matches the pinned digests and the Python copy (`scripts/core_pin.py check`); `tests/test_core_pin.py` covers drift in both, and the offline Python suite passes | | Distribution | `scripts/check_distribution.py` passes; the source archive carries the .NET sources, `dotnet/NOTICE`, and the kit copy | From 936421f0264eff2e79a3891fbf450d53b34ac2b8 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 04:40:15 -0400 Subject: [PATCH 23/33] Kill a .NET worker whose pipe setup fails through the common kill path When Process.Start succeeded but taking the worker's pipes failed, Start killed the process directly and disposed it at once: no stop notice, no kill bookkeeping, no bounded exit wait, and a failed kill was swallowed while the last handle was dropped. The Worker now owns the process from the spawn. Attach takes the pipes all or none; on failure FailAfterSpawn kills through KillTree (after the stop notice), waits up to one second for the exit, records it, and reports a spawn failure that says so if the worker had not exited. Disposal releases the handle. The host passes its stop notice to Start, so it is in place before any kill. A test seam (SpawnedHook, null outside tests) fails the setup after the spawn. --- dotnet/src/JacquardHost/Carrier/Worker.cs | 131 ++++++++++++++---- dotnet/src/JacquardHost/Host.cs | 27 ++-- .../JacquardHost.Tests/DescriptorTests.cs | 55 ++++++++ 3 files changed, 172 insertions(+), 41 deletions(-) diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index 8d1974a..dbcd7a8 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -35,8 +35,10 @@ namespace JacquardLang.Host.Carrier; /// calls), so nothing else holds or drains the pipes. /// /// -/// At expiry, cancellation, or disposal while the worker runs, the worker's -/// process tree is killed with : .NET stops +/// At expiry, cancellation, or disposal while the worker runs, and when the +/// spawned worker's pipes cannot be set up, the worker's process tree is +/// killed with , always after the +/// : .NET stops /// the worker, lists its children from the process table, kills it, and /// does the same for each child. A descendant that has already been /// reparented (because its parent exited or it daemonized) is not found and @@ -68,13 +70,15 @@ public sealed class Worker : IDisposable private readonly object gate = new(); private readonly Process? process; - private readonly StreamWriter? stdinWriter; - private readonly Stream? stdin; - private readonly Stream? stdout; - private readonly Stream? stderr; - private readonly int stdinFd = -1; - private readonly int stdoutFd = -1; - private readonly int stderrFd = -1; + + // Set by Attach, before Start returns the worker, and never changed after. + private StreamWriter? stdinWriter; + private Stream? stdin; + private Stream? stdout; + private Stream? stderr; + private int stdinFd = -1; + private int stdoutFd = -1; + private int stderrFd = -1; private readonly long expires; private readonly CancellationToken cancel; private readonly byte[] chunk = new byte[8192]; @@ -97,17 +101,12 @@ private Worker(long expires, string? spawnError, CancellationToken cancel) SpawnError = spawnError; } - private Worker(Process process, long expires, CancellationToken cancel) + /// Takes ownership of a spawned worker; then takes its pipes. + private Worker(Process process, long expires, Action? stopNotice, CancellationToken cancel) : this(expires, null, cancel) { this.process = process; - stdinWriter = process.StandardInput; - stdin = stdinWriter.BaseStream; - stdout = process.StandardOutput.BaseStream; - stderr = process.StandardError.BaseStream; - stdinFd = Descriptor(stdin); - stdoutFd = Descriptor(stdout); - stderrFd = Descriptor(stderr); + this.stopNotice = stopNotice; } /// @@ -135,14 +134,25 @@ private Worker(Process process, long expires, CancellationToken cancel) /// internal static volatile Action? ExpireHook; + /// + /// Test seam: runs once the worker has been spawned and is owned by its + /// , before the carrier takes its pipes. A test + /// throws from it to stand for + /// redirected streams that are not pipe streams. Process-global; null + /// outside tests. + /// + internal static volatile Action? SpawnedHook; + private volatile Action? stopNotice; /// - /// Runs whenever the worker's lifetime ends by expiry, cancellation, or - /// disposal, before its process tree is killed, on the thread that ends - /// it (the caller's thread, or the disposing thread). The host uses it to - /// start the cancellation of its handlers' token before any kill. It must - /// not throw or block, and may run more than once. + /// Runs before every kill of the worker's process tree, on the thread + /// that kills it: at expiry, cancellation, or disposal (the caller's + /// thread, or the disposing thread), and when setting up the spawned + /// worker fails inside . + /// The host passes it to Start, so it is in place before the spawn, + /// and uses it to start the cancellation of its handlers' token before + /// any kill. It must not throw or block, and may run more than once. /// internal Action? StopNotice { @@ -153,8 +163,8 @@ internal Action? StopNotice /// The lifetime clock: outside tests. private static long Now() => ClockHook is { } clock ? clock() : Stopwatch.GetTimestamp(); - /// The spawn error, when the worker never started. - public string? SpawnError { get; } + /// The spawn error, when the worker never started or its pipes could not be set up. + public string? SpawnError { get; private set; } /// The exit status once the worker has exited, unless the host's SIGKILL ended it. public int? ExitStatus { get; private set; } @@ -193,7 +203,16 @@ public byte[] Stderr /// reported by the first operation. /// /// The options are invalid. - public static Worker Start(string store, WorkerOptions options, CancellationToken cancellationToken) + public static Worker Start(string store, WorkerOptions options, CancellationToken cancellationToken) => + Start(store, options, null, cancellationToken); + + /// + /// As , with + /// as the from the + /// moment the worker is spawned. + /// + /// The options are invalid. + internal static Worker Start(string store, WorkerOptions options, Action? stopNotice, CancellationToken cancellationToken) { ArgumentNullException.ThrowIfNull(store); ArgumentNullException.ThrowIfNull(options); @@ -253,16 +272,66 @@ public static Worker Start(string store, WorkerOptions options, CancellationToke cancellationToken); } + // From here the worker owns the process: a failure to set up its + // pipes kills it through the common kill path and keeps the handle, + // which disposal releases. + var worker = new Worker(process, expires, stopNotice, cancellationToken); try { - return new Worker(process, expires, cancellationToken); + worker.Attach(); } catch (Exception error) when (error is InvalidOperationException or NotSupportedException) { // The streams are not the pipes this carrier polls. - KillQuietly(process); - process.Dispose(); - return Failed(expires, error.Message, "could not poll the worker's pipes (" + error.Message + ")", cancellationToken); + worker.FailAfterSpawn(error); + } + + return worker; + } + + /// Takes the spawned worker's pipes, all or none. + /// A redirected stream is not a pipe stream. + private void Attach() + { + SpawnedHook?.Invoke(process!); + StreamWriter writer = process!.StandardInput; + Stream input = writer.BaseStream; + Stream output = process.StandardOutput.BaseStream; + Stream error = process.StandardError.BaseStream; + int inputFd = Descriptor(input); + int outputFd = Descriptor(output); + int errorFd = Descriptor(error); + stdinWriter = writer; + stdin = input; + stdout = output; + stderr = error; + stdinFd = inputFd; + stdoutFd = outputFd; + stderrFd = errorFd; + } + + /// + /// Ends a worker whose pipes could not be set up: kills its process tree + /// through (after the stop notice), waits up to one + /// second for the exit, and records it. The first operation reports the + /// failure as a spawn failure, saying so if the worker had still not + /// exited. releases the process handle. + /// + private void FailAfterSpawn(Exception error) + { + lock (gate) + { + KillTree(); + WaitForExit(ExitGraceMs); + RecordExit(); + string message = "could not poll the worker's pipes (" + error.Message + ")"; + if (!HasExited()) + { + message += "; the worker had not exited one second after the kill and may still be running"; + } + + SpawnError = error.Message; + fault = CarrierException.Loss(message); } } @@ -360,7 +429,9 @@ private bool HasExited() /// /// Kills the worker and the descendants .NET finds, while the worker is - /// alive. Every kill comes through here, after the stop notice. + /// alive. Every kill comes through here, after the stop notice: at expiry, + /// cancellation, disposal, and a failure to set up the spawned worker's + /// pipes. /// private void KillTree() { diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 7cc1a22..0469c7b 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -187,10 +187,14 @@ private InvocationResult Run(Func opening, RegistrySnapshot { var session = new Session(); var result = new InvocationResult(ResultKind.CarrierLoss); + var call = new Call(session, result, registry); Worker worker; try { - worker = Worker.Start(Store, carrier, cancellationToken); + // The stop notice is in place from the spawn, so every kill of the + // worker, including one while Start sets it up, starts the + // handlers' token cancellation first. + worker = Worker.Start(Store, carrier, call.StopHandlers, cancellationToken); } catch (ArgumentException error) { @@ -200,7 +204,7 @@ private InvocationResult Run(Func opening, RegistrySnapshot // The worker observes the caller's token in its waits on this thread; // nothing is registered on it. - var call = new Call(worker, session, result, registry); + call.Attach(worker); try { call.Drive(opening, Limits); @@ -276,20 +280,20 @@ private sealed class Call private CancellationTokenSource? handlerStop; private bool stopRequested; private bool handlersStopped; + private Worker? worker; - internal Call(Worker worker, Session session, InvocationResult result, RegistrySnapshot registry) + internal Call(Session session, InvocationResult result, RegistrySnapshot registry) { - Worker = worker; Session = session; Result = result; this.registry = registry; - - // Every kill of the worker, whichever check or thread sees the - // stop first, starts the token's cancellation before it. - worker.StopNotice = StopHandlers; } - internal Worker Worker { get; } + /// The call's worker, whose stop notice is . + internal Worker Worker => worker ?? throw new InvalidOperationException("the call has no worker"); + + /// Gives the call its worker, started with as its stop notice. + internal void Attach(Worker started) => worker = started; internal Session Session { get; } @@ -534,13 +538,14 @@ private CancellationToken HandlerToken() /// Called when the caller's thread sees the call stopping while a /// handler runs, at , and through the worker's /// before any kill (on the caller's - /// thread, or on a thread disposing the worker): starts, at most + /// thread, including while the worker is being started, or on a + /// thread disposing the worker): starts, at most /// once, a dedicated background thread that cancels the handlers' /// token and discards whatever its callbacks throw. Nothing waits for /// that thread, so a callback that blocks holds only it, and the /// shared thread pool is not involved. /// - private void StopHandlers() + internal void StopHandlers() { CancellationTokenSource? source; lock (stopSync) diff --git a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs index fdb76f9..8030230 100644 --- a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs +++ b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs @@ -25,6 +25,61 @@ public sealed class DescriptorTests { private static bool DescriptorsBackTo(int before) => Eventually(() => Descriptors() <= before); + [TearDown] + public void ClearSeams() + { + Worker.SpawnedHook = null; + Worker.ExpireHook = null; + } + + [Test] + public void AFailureAfterTheSpawnKillsTheWorkerThroughTheKillPathAndReleasesIt() + { + // A seam fails the worker's setup after Process.Start succeeded, as + // redirected streams that are not pipe streams would. + Assert.That(InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))).Ok, Is.True); // warm up + int before = Descriptors(); + var pids = new List(); + int kills = 0; + Worker.SpawnedHook = process => + { + pids.Add(process.Id); + throw new NotSupportedException("seam: worker pipes are not pipe streams"); + }; + Worker.ExpireHook = () => Interlocked.Increment(ref kills); + try + { + for (int i = 0; i < 3; i++) + { + InvocationResult result = Within(TimeSpan.FromSeconds(20), () => InvokeReady(Faulty("silent", TimeSpan.FromSeconds(30)))); + Assert.That(pids, Has.Count.EqualTo(i + 1), "the seam did not run"); + Assert.That(Alive(pids[i]), Is.False, "the spawned worker is still running after Invoke returned"); + Assert.That(result.Kind, Is.EqualTo(ResultKind.CarrierLoss), result.ToString()); + Assert.That(Reason(result), Is.EqualTo("worker_spawn_failed"), result.ToString()); + Assert.That(result.Code, Is.EqualTo("E1611")); + Assert.That(result.HostEvidence.Get("detail")?.AsString(), Is.EqualTo("could not poll the worker's pipes (seam: worker pipes are not pipe streams)")); + Assert.That(result.Signal, Is.EqualTo("SIGKILL"), "the killed worker's exit was not waited for and recorded"); + Assert.That(result.CoreFrames, Is.Empty); + Assert.That(result.HostFrames, Is.Empty); + } + + Assert.That(Volatile.Read(ref kills), Is.EqualTo(3), "the kill did not go through the common kill path once per worker"); + Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "adapter threads remain"); + Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); + } + finally + { + Worker.SpawnedHook = null; + foreach (int pid in pids) + { + if (Alive(pid)) + { + Kill(pid); + } + } + } + } + [Test] public void InvocationsReleaseTheirPipesAndThreads() { From 8fbd2079a15c1552c3e88700c044882e05d6e4fd Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 04:40:15 -0400 Subject: [PATCH 24/33] Assert one .NET stop notice per kill, on the killing thread StopNoticeTests now fails notice, kill, kill: each kill must consume a notice given on its own thread before it. It covers disposal killing on the disposer's own thread and on a waiting operation's thread, and the kill when setting up the spawned worker's pipes fails. --- .../JacquardHost.Tests/StopNoticeTests.cs | 150 +++++++++++++++--- 1 file changed, 125 insertions(+), 25 deletions(-) diff --git a/dotnet/test/JacquardHost.Tests/StopNoticeTests.cs b/dotnet/test/JacquardHost.Tests/StopNoticeTests.cs index 9db13fe..cb436a0 100644 --- a/dotnet/test/JacquardHost.Tests/StopNoticeTests.cs +++ b/dotnet/test/JacquardHost.Tests/StopNoticeTests.cs @@ -1,5 +1,6 @@ using System; using System.Collections.Concurrent; +using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Threading; @@ -11,9 +12,10 @@ namespace JacquardLang.Host.Tests; /// /// The worker gives its stop notice (through which the host starts the -/// cancellation of its handlers' token) before every kill, including a kill -/// by disposal: one from another thread while an operation waits, and one -/// with no operation in progress. Within the worker is +/// cancellation of its handlers' token) before every kill, one notice per +/// kill on the killing thread: a kill by disposal, from the disposer's own +/// thread or from a waiting operation's thread, and the kill when setting up +/// the spawned worker's pipes fails. Within the worker is /// disposed only after the invocation has ended, which starts the /// cancellation itself; these tests cover the worker on its own. /// @@ -25,35 +27,96 @@ public void ClearSeams() { Worker.ClockHook = null; Worker.ExpireHook = null; + Worker.SpawnedHook = null; } - private static void AssertNoticeBeforeEveryKill(ConcurrentQueue events) + /// Records each notice and kill with the thread it ran on. + private sealed class Events { - string[] order = events.ToArray(); - Assert.That(order, Does.Contain("kill"), "the worker was not killed"); - int notices = 0; - foreach (string step in order) + private readonly ConcurrentQueue<(string Step, int Thread)> steps = new(); + + internal Events() { - if (step == "notice") - { - notices++; - } - else + Worker.ExpireHook = () => steps.Enqueue(("kill", Environment.CurrentManagedThreadId)); + } + + internal void Notice() => steps.Enqueue(("notice", Environment.CurrentManagedThreadId)); + + /// + /// Asserts that every kill used a notice of its own, given on the + /// killing thread before it (so notice, kill, kill fails), and + /// that there was a kill. Returns the killing threads, in order. + /// + internal int[] AssertOneNoticePerKill() + { + (string Step, int Thread)[] order = steps.ToArray(); + string shown = string.Join(", ", order.Select(step => step.Step + "@" + step.Thread)); + var unused = new Dictionary(); + var killers = new List(); + foreach ((string step, int thread) in order) { - Assert.That(notices, Is.GreaterThan(0), "a kill came before any stop notice: " + string.Join(", ", order)); + unused.TryGetValue(thread, out int count); + if (step == "notice") + { + unused[thread] = count + 1; + continue; + } + + Assert.That(count, Is.GreaterThan(0), "a kill had no stop notice of its own on its thread: " + shown); + unused[thread] = count - 1; + killers.Add(thread); } + + Assert.That(killers, Is.Not.Empty, "the worker was not killed: " + shown); + return killers.ToArray(); } } + private static WorkerOptions Silent() => Standin("silent", string.Empty, TimeSpan.FromSeconds(30)); + + [Test] + public void TheCheckRefusesAKillWithoutANoticeOfItsOwn() + { + // The assertion itself: two kills after one notice fail it. + var events = new Events(); + Worker.ExpireHook!(); + events.Notice(); + Assert.Throws((Action)(() => events.AssertOneNoticePerKill())); + var paired = new Events(); + paired.Notice(); + Worker.ExpireHook!(); + Worker.ExpireHook!(); + Assert.Throws((Action)(() => paired.AssertOneNoticePerKill())); + } + [Test] public void DisposalWithNoOperationGivesTheNoticeBeforeTheKill() { - var events = new ConcurrentQueue(); - Worker worker = Worker.Start("unused", Standin("silent", string.Empty, TimeSpan.FromSeconds(30)), CancellationToken.None); - worker.StopNotice = () => events.Enqueue("notice"); - Worker.ExpireHook = () => events.Enqueue("kill"); + var events = new Events(); + Worker worker = Worker.Start("unused", Silent(), events.Notice, CancellationToken.None); worker.Dispose(); - AssertNoticeBeforeEveryKill(events); + Assert.That(events.AssertOneNoticePerKill(), Is.EqualTo(new[] { Environment.CurrentManagedThreadId })); + } + + [Test] + public void DisposalOnAnotherThreadKillsOnTheDisposersThreadAfterTheNotice() + { + var events = new Events(); + Worker worker = Worker.Start("unused", Silent(), events.Notice, CancellationToken.None); + int disposerId = -1; + var disposer = new Thread(() => + { + Volatile.Write(ref disposerId, Environment.CurrentManagedThreadId); + worker.Dispose(); + }) + { + IsBackground = true, + Name = "test-disposer", + }; + disposer.Start(); + Assert.That(disposer.Join(TimeSpan.FromSeconds(10)), Is.True, "the disposal did not finish"); + Assert.That(events.AssertOneNoticePerKill(), Is.EqualTo(new[] { Volatile.Read(ref disposerId) })); + Assert.That(worker.Signal, Is.EqualTo("SIGKILL")); } [Test] @@ -62,11 +125,12 @@ public void DisposalFromAnotherThreadBetweenTwoChecksGivesTheNoticeBeforeTheKill // The waiting thread's first clock read comes from its stopping // check, after that check has read the disposal flag. The clock seam // then starts the disposal on another thread and returns only once - // the flag is set, so the next check, the fault check, sees it. - var events = new ConcurrentQueue(); - Worker worker = Worker.Start("unused", Standin("silent", string.Empty, TimeSpan.FromSeconds(30)), CancellationToken.None); - worker.StopNotice = () => events.Enqueue("notice"); - Worker.ExpireHook = () => events.Enqueue("kill"); + // the flag is set, so the next check, the fault check, sees it and + // kills the worker on the waiting thread. The disposer kills again + // only if the killed worker has not exited by the time it holds the + // worker; that kill too must have its own notice. + var events = new Events(); + Worker worker = Worker.Start("unused", Silent(), events.Notice, CancellationToken.None); int waiter = -1; int reads = 0; Thread? disposer = null; @@ -103,6 +167,42 @@ public void DisposalFromAnotherThreadBetweenTwoChecksGivesTheNoticeBeforeTheKill Assert.That(disposer!.Join(TimeSpan.FromSeconds(10)), Is.True, "the disposal did not finish"); Assert.That(thrown, Is.Not.Null, "the wait ended on its predicate, before the race"); Assert.That(thrown!.IsAborted, Is.True, thrown.Message); - AssertNoticeBeforeEveryKill(events); + int[] killers = events.AssertOneNoticePerKill(); + Assert.That(killers[0], Is.EqualTo(Volatile.Read(ref waiter)), "the waiting thread did not kill the worker"); + Assert.That(killers.Skip(1), Is.All.EqualTo(disposer.ManagedThreadId), "a later kill was not the disposer's"); + } + + [Test] + public void AFailureToSetUpTheSpawnedWorkersPipesGivesTheNoticeBeforeTheKill() + { + var events = new Events(); + int pid = -1; + Worker.SpawnedHook = process => + { + pid = process.Id; + throw new NotSupportedException("seam: worker pipes are not pipe streams"); + }; + Worker worker = Worker.Start("unused", Silent(), events.Notice, CancellationToken.None); + try + { + Assert.That(pid, Is.GreaterThan(0), "the seam did not run"); + Assert.That(events.AssertOneNoticePerKill(), Is.EqualTo(new[] { Environment.CurrentManagedThreadId })); + Assert.That(Alive(pid), Is.False, "the worker was still running when Start returned"); + Assert.That(worker.Signal, Is.EqualTo("SIGKILL"), "the killed worker's exit was not recorded"); + Assert.That(worker.SpawnError, Is.EqualTo("seam: worker pipes are not pipe streams")); + CarrierException? fault = worker.Fault(); + Assert.That(fault, Is.Not.Null); + Assert.That(fault!.IsLoss && !fault.IsTimeout && !fault.IsAborted, Is.True, fault.Message); + Assert.That(fault.Message, Is.EqualTo("could not poll the worker's pipes (seam: worker pipes are not pipe streams)")); + Assert.Throws((Action)(() => worker.ReadFrame(JacquardLang.Host.Protocol.Limits.Hard))); + } + finally + { + worker.Dispose(); + if (pid > 0 && Alive(pid)) + { + Kill(pid); + } + } } } From 55ff96d60c21de4b673a3303f0d83844638af5c0 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 04:40:15 -0400 Subject: [PATCH 25/33] Make the .NET post-handler kill-order case deterministic The DeadlineAfterTheHandler case slept 100 ms and assumed the handler's answer was published. A seam (HandlerAnsweredHook, null outside tests) now runs after the answer is published, and the test waits for it before it arms the clock. Each case asserts from the killing thread's stack which check killed the worker: the handler wait's, or the one after the handler returned. --- dotnet/src/JacquardHost/Host.cs | 8 +++++ .../test/JacquardHost.Tests/KillOrderTests.cs | 34 ++++++++++++++++--- 2 files changed, 38 insertions(+), 4 deletions(-) diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 0469c7b..94bb59e 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -92,6 +92,13 @@ public Host(string store, HostOptions? options = null) /// internal static volatile Action? HandlerCheckHook; + /// + /// Test seam: runs on the handler thread just after it publishes its + /// answer, so a test can wait until the caller's wait will see it. Null + /// outside tests. + /// + internal static volatile Action? HandlerAnsweredHook; + /// /// Test seam: runs before the host starts the cancellation of the /// handlers' token, with "thread" before it creates its dedicated @@ -476,6 +483,7 @@ private Answer RunHandler(Handler handler, IReadOnlyList arguments) { HandlerCheckHook?.Invoke(); completion.Answer = Worker.Stopping ? NotStarted : CallHandler(handler, context, arguments); + HandlerAnsweredHook?.Invoke(); }) { IsBackground = true, diff --git a/dotnet/test/JacquardHost.Tests/KillOrderTests.cs b/dotnet/test/JacquardHost.Tests/KillOrderTests.cs index 96b3475..608b8f2 100644 --- a/dotnet/test/JacquardHost.Tests/KillOrderTests.cs +++ b/dotnet/test/JacquardHost.Tests/KillOrderTests.cs @@ -1,5 +1,6 @@ using System; using System.Diagnostics; +using System.Linq; using System.Threading; using JacquardLang.Host.Carrier; using JacquardLang.Host.Protocol; @@ -15,7 +16,9 @@ namespace JacquardLang.Host.Tests; /// slice (its own predicate, then the fault check), and the call checks again /// after the handler returns; these tests use the clock seam to land the /// stop exactly between two such reads, so the outcome does not depend on -/// timing. +/// timing. Each case also asserts which check killed the worker, from the +/// killing thread's stack: the handler wait's fault check, or the call's +/// check after the handler returned. /// [TestFixture] public sealed class KillOrderTests @@ -29,8 +32,13 @@ public void ClearSeams() Worker.ClockHook = null; Worker.ExpireHook = null; Host.HandlerThreadStartedHook = null; + Host.HandlerAnsweredHook = null; } + /// The names of the methods on the current thread's stack. + private static string[] StackMethods() => + new StackTrace(false).GetFrames().Select(frame => frame.GetMethod()?.Name ?? string.Empty).ToArray(); + /// What the stop is, and when it lands. public enum Stop { @@ -95,27 +103,33 @@ public void TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks(Stop stop) using var cancel = new CancellationTokenSource(); using var registered = new ManualResetEventSlim(); using var callback = new ManualResetEventSlim(); + using var answered = new ManualResetEventSlim(); bool returnsAtOnce = stop == Stop.DeadlineAfterTheHandler; var clock = new SplitClock(stop == Stop.CallerCancelInTheWait ? cancel.Cancel : null, stop != Stop.CallerCancelInTheWait); int kills = 0; bool callbackBeforeKill = false; + string[] killer = Array.Empty(); Worker.ExpireHook = () => { if (Interlocked.Increment(ref kills) == 1) { + Volatile.Write(ref killer, StackMethods()); Volatile.Write(ref callbackBeforeKill, callback.Wait(CallbackWait)); } }; // The caller's thread arms the clock only once the handler has - // registered its callback (and, for the post-handler case, returned), - // so the caller's next clock read is the stopping check. + // registered its callback (and, for the post-handler case, returned + // and published its answer, so the wait's predicate sees the answer + // without reading the clock), so the caller's next clock read is the + // stopping check. + Host.HandlerAnsweredHook = answered.Set; Host.HandlerThreadStartedHook = () => { Assert.That(registered.Wait(TimeSpan.FromSeconds(10)), Is.True, "the handler did not start"); if (returnsAtOnce) { - Thread.Sleep(100); + Assert.That(answered.Wait(TimeSpan.FromSeconds(10)), Is.True, "the handler did not publish its answer"); } clock.Arm(); @@ -141,5 +155,17 @@ public void TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks(Stop stop) Assert.That(result.HostFrames, Has.Count.EqualTo(2), "a response was sent after the stop"); Assert.That(Volatile.Read(ref kills), Is.GreaterThanOrEqualTo(1), "the worker was not killed"); Assert.That(Volatile.Read(ref callbackBeforeKill), Is.True, "the worker was killed before the token's cancellation started"); + string[] stack = Volatile.Read(ref killer); + string path = string.Join(" < ", stack); + if (returnsAtOnce) + { + // The check in Respond after RunHandler returned killed it. + Assert.That(stack, Does.Contain("Respond").And.Not.Contain("RunHandler").And.Not.Contain("Await").And.Not.Contain("WriteBytes"), path); + } + else + { + // The fault check in the handler wait killed it. + Assert.That(stack, Does.Contain("Await").And.Contain("RunHandler"), path); + } } } From f3e9e9098c3c0a3a280b13b221bd8ad3afe56334 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 04:40:15 -0400 Subject: [PATCH 26/33] Show HandlerTokenNotCancelled in the .NET result's ToString The flag is a volatile field, since a thread disposing the worker may set it. It is shown by ToString when true and stays out of HostEvidence, which keeps the shared evidence schema and stays empty on success. --- dotnet/src/JacquardHost/InvocationResult.cs | 19 ++++++++++++++++--- .../JacquardHost.Tests/CancelStartTests.cs | 6 ++++++ 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/dotnet/src/JacquardHost/InvocationResult.cs b/dotnet/src/JacquardHost/InvocationResult.cs index f0e6e7d..5b805fc 100644 --- a/dotnet/src/JacquardHost/InvocationResult.cs +++ b/dotnet/src/JacquardHost/InvocationResult.cs @@ -48,6 +48,8 @@ public enum Failure /// Core's frames and evidence beside the host's observations. public sealed class InvocationResult { + private volatile bool handlerTokenNotCancelled; + internal InvocationResult(ResultKind kind) { Kind = kind; @@ -95,9 +97,19 @@ internal InvocationResult(ResultKind kind) /// example when the process is out of memory. That token is then never /// cancelled: a handler waiting only on it keeps waiting (until its own /// timeout, if it has one), while - /// still turns true. False otherwise. + /// still turns true. False otherwise. It may be set on a thread disposing + /// the worker, so it is read and written as a volatile field. It is shown + /// by when true, and is not part of + /// : that evidence describes how the invocation + /// ended, in a schema the adapters share, while this flag reports the + /// .NET host's own failure to start the token's cancellation, which can + /// follow any outcome, including a successful one. /// - public bool HandlerTokenNotCancelled { get; internal set; } + public bool HandlerTokenNotCancelled + { + get => handlerTokenNotCancelled; + internal set => handlerTokenNotCancelled = value; + } /// Whether Core returned a successful outcome. public bool Ok => Kind == ResultKind.Ok; @@ -164,7 +176,8 @@ public Value ValueOrThrow() } /// - public override string ToString() => "InvocationResult[" + KindName(Kind) + ", " + Message + "]"; + public override string ToString() => + "InvocationResult[" + KindName(Kind) + ", " + Message + (HandlerTokenNotCancelled ? ", handler token not cancelled" : string.Empty) + "]"; } /// An unsuccessful invocation. diff --git a/dotnet/test/JacquardHost.Tests/CancelStartTests.cs b/dotnet/test/JacquardHost.Tests/CancelStartTests.cs index 7c2db61..c14ea32 100644 --- a/dotnet/test/JacquardHost.Tests/CancelStartTests.cs +++ b/dotnet/test/JacquardHost.Tests/CancelStartTests.cs @@ -130,6 +130,7 @@ public void WithoutAThreadTheTokenIsCancelledOnAPoolThread() Assert.That(seen.Done.Wait(TimeSpan.FromSeconds(10)), Is.True, "the handler did not return"); Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); Assert.That(result.HandlerTokenNotCancelled, Is.False); + Assert.That(result.ToString(), Does.Not.Contain("handler token"), "an unset flag was shown"); Assert.That(Volatile.Read(ref seen.TokenCancelled), Is.True, "the token was not cancelled"); Assert.That(Volatile.Read(ref seen.CallbackOnPool), Is.True, "the callback did not run on a pool thread"); Assert.That(Volatile.Read(ref seen.CallbackThread), Is.Not.EqualTo(Volatile.Read(ref seen.CallerThread))); @@ -158,6 +159,7 @@ public void WithNeitherAThreadNorAPoolWorkItemInvokeReturnsAndDisposesTheWorker( })); Assert.That(Volatile.Read(ref attempts), Is.EqualTo(2), "the host did not try a thread and then the pool, once"); Assert.That(result.HandlerTokenNotCancelled, Is.True, "the failure was not recorded"); + Assert.That(result.ToString(), Does.EndWith(", handler token not cancelled]"), "the flag is not shown"); if (atTheDeadline) { Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); @@ -168,6 +170,10 @@ public void WithNeitherAThreadNorAPoolWorkItemInvokeReturnsAndDisposesTheWorker( else { Assert.That(result.Ok, Is.True, result.ToString()); + + // The flag is not host evidence: the evidence stays empty on success. + Assert.That(result.HostEvidence, Is.Empty, "a successful invocation has host evidence"); + Assert.That(result.ToString(), Is.EqualTo("InvocationResult[ok, Jacquard invocation succeeded, handler token not cancelled]")); } Assert.That(Volatile.Read(ref seen.CallbackThread), Is.EqualTo(-1), "a callback ran"); From e0a01ac40eb0416e1a0b975ebb18e8f99a60609e Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 04:49:44 -0400 Subject: [PATCH 27/33] Leave runtime assembly files out of the .NET descriptor counts On .NET 10 the full suite saw System.Security.Claims.dll opened during the resource test's loop, after the added spawn-failure test ran first: the runtime loads an assembly when it first needs it and keeps the file open. The counts now skip descriptors on .dll files, which the adapter never opens. --- dotnet/test/JacquardHost.Tests/Support.cs | 33 +++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/dotnet/test/JacquardHost.Tests/Support.cs b/dotnet/test/JacquardHost.Tests/Support.cs index f5acecd..26d1c22 100644 --- a/dotnet/test/JacquardHost.Tests/Support.cs +++ b/dotnet/test/JacquardHost.Tests/Support.cs @@ -243,8 +243,37 @@ internal static T Within(TimeSpan limit, Func body) return value!; } - /// Open descriptors of this process (Linux). - internal static int Descriptors() => Directory.GetFiles("/proc/self/fd").Length; + /// + /// Open descriptors of this process (Linux), except those on assembly + /// files: the runtime opens an assembly when it first loads it, which + /// can happen in the middle of a test (for example when tiered + /// compilation recompiles a method), and keeps it open. The adapter opens + /// no assembly files. + /// + internal static int Descriptors() + { + int count = 0; + foreach (string fd in Directory.GetFiles("/proc/self/fd")) + { + string? target; + try + { + target = new FileInfo(fd).LinkTarget; + } + catch (IOException) + { + // Closed while it was listed. + continue; + } + + if (target is null || !target.EndsWith(".dll", StringComparison.Ordinal)) + { + count++; + } + } + + return count; + } /// Live threads of this process named by the adapter (Linux, names truncated to 15 bytes). internal static int AdapterThreads() From 66eb43d8ce101087f2dcfb707a5cf0a5c7f0ac63 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 05:06:18 -0400 Subject: [PATCH 28/33] Record the .NET post-spawn kill and stop-notice evidence: 94 offline and 99 live tests The README and EVIDENCE now say that a failure to set up the spawned worker's pipes kills it through the common kill path, after the stop notice, with a bounded exit wait; that every kill has a notice of its own; and that HandlerTokenNotCancelled is shown by ToString and kept out of HostEvidence. EVIDENCE records which new tests fail against 4e32b9f with only the seams added, the two kill-order mutations, and 50 clean runs of the kill-order and stop-notice tests on each runtime. --- docs/EVIDENCE.md | 6 +++--- dotnet/README.md | 17 +++++++++++++---- 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/docs/EVIDENCE.md b/docs/EVIDENCE.md index af6b3ba..739d2d0 100644 --- a/docs/EVIDENCE.md +++ b/docs/EVIDENCE.md @@ -136,9 +136,9 @@ dependencies. | Lane | Result | |---|---| -| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 90 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 15 handler-token tests, 3 kill-order tests, 2 stop-notice tests, 3 cancellation-start tests (Linux), 8 process-tree tests (Unix), 7 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline cancels the token; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. With a test clock, 36- and 48-hour lifetimes leave the token uncancelled a day and a minute in and one second before the deadline (`IsCancelled` false too), and cancel it once the clock reaches the deadline, with no late response; the cancellation came 2 to 4 ms after the clock reached it in these runs (an observation, not asserted). With every thread-pool thread and 40 more blocked and a probe work item queued behind them, the token is cancelled at a 1 s deadline, at the caller's cancellation, and at the end of a successful invocation before the probe runs. A seam that holds the worker's kill until the token's callback has run sees the callback first, at the deadline and at caller cancellation; with a test clock that lands the stop exactly between two reads of the stop condition (the deadline or the caller's cancellation between the handler wait's predicate and its fault check, and the deadline between the post-handler stopping check and its fault check), it still sees the callback first. The worker gives its stop notice before every kill, including disposal with no operation running and disposal from another thread landing between a wait's two checks. When the cancellation thread cannot be created (a seam throws `OutOfMemoryException`), the token is cancelled on a pool thread; when the pool work item fails too (`NotSupportedException`), `Invoke` returns its result (at a 1 s deadline, and on success), sets `HandlerTokenNotCancelled`, runs no callback, leaves the token uncancelled while `IsCancelled` is true, and disposes the worker (no child process or descriptor remains). Callbacks run on the host's `jacquard-handler-cancel` thread, not a pool thread; one registered after the cancellation runs on the handler's own thread. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors: no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | -| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 95 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 90 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | -| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched`. Against the source with the handler token's timer and pool work items (with the test clock seam copied in, and with the timer's first wait shortened to 100 ms to stand for the day it waits), both `ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore` cases fail (the token is cancelled with 12 and 24 hours left), and the three `UnderAStarvedThreadPool...` tests fail (the token is not cancelled within 10 s). Cancelling through a pool work item instead of the host's thread fails the three starvation tests; killing the worker before starting the cancellation fails both `TheTokenIsCancelledBeforeTheWorkerIsKilled` cases (and the measured delay rose to about 244 ms, the time the process-tree kill took). `ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent` now holds by order: the token is cancelled only after the caller's thread has stopped waiting for the answer. Against the source before the stop notice, all three `TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks` cases fail (the kill came first); removing the stop notice from the kill fails them and both `StopNoticeTests`. Letting the pool fallback's exception escape fails both `WithNeitherAThreadNorAPoolWorkItem...` cases (`Invoke` throws `NotSupportedException`). Repeating `TheTokenIsCancelledBeforeTheWorkerIsKilled` 150 times per case under the real clock passed every run; a reviewer's probe (an 800 ms deadline, or a caller cancellation after 100 to 150 ms, 150 runs each) found the kill first in 8 and 0 runs against the source before the stop notice on this machine, and in 0 and 0 after it | +| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 94 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 15 handler-token tests, 3 kill-order tests, 5 stop-notice tests, 3 cancellation-start tests (Linux), 8 process-tree tests (Unix), 8 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline cancels the token; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. With a test clock, 36- and 48-hour lifetimes leave the token uncancelled a day and a minute in and one second before the deadline (`IsCancelled` false too), and cancel it once the clock reaches the deadline, with no late response; the cancellation came 2 to 4 ms after the clock reached it in these runs (an observation, not asserted). With every thread-pool thread and 40 more blocked and a probe work item queued behind them, the token is cancelled at a 1 s deadline, at the caller's cancellation, and at the end of a successful invocation before the probe runs. A seam that holds the worker's kill until the token's callback has run sees the callback first, at the deadline and at caller cancellation; with a test clock that lands the stop exactly between two reads of the stop condition (the deadline or the caller's cancellation between the handler wait's predicate and its fault check, and the deadline between the post-handler stopping check and its fault check), it still sees the callback first; the post-handler case arms the clock only after a seam reports the handler's answer published, and each case asserts from the killing thread's stack which check killed the worker (the handler wait's, or the one after the handler returned). Every kill of the worker uses a stop notice of its own, given before it on the killing thread (a check that `notice, kill, kill` fails is itself tested): disposal with no operation running, on the test's thread and on a separate disposer thread (which does the kill); disposal from another thread landing between a wait's two checks (the waiting thread kills, and any later kill is the disposer's); and a failure to set up the spawned worker's pipes. For that last case a seam throws `NotSupportedException` after `Process.Start`, as non-pipe streams would: `Start` returns with the worker already exited (SIGKILL recorded, its pid no longer alive), the first operation reports the spawn failure, and through `Invoke` the result is a carrier loss with reason `worker_spawn_failed`, code E1611, no frames, and no growth in descriptors or adapter threads over three runs. When the cancellation thread cannot be created (a seam throws `OutOfMemoryException`), the token is cancelled on a pool thread; when the pool work item fails too (`NotSupportedException`), `Invoke` returns its result (at a 1 s deadline, and on success), sets `HandlerTokenNotCancelled` (shown by the result's `ToString()`; on success `HostEvidence` stays empty), runs no callback, leaves the token uncancelled while `IsCancelled` is true, and disposes the worker (no child process or descriptor remains). Callbacks run on the host's `jacquard-handler-cancel` thread, not a pool thread; one registered after the cancellation runs on the handler's own thread. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors (descriptors on the runtime's own assembly files, which it opens when it first loads an assembly, are not counted): no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | +| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 99 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 94 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | +| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched`. Against the source with the handler token's timer and pool work items (with the test clock seam copied in, and with the timer's first wait shortened to 100 ms to stand for the day it waits), both `ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore` cases fail (the token is cancelled with 12 and 24 hours left), and the three `UnderAStarvedThreadPool...` tests fail (the token is not cancelled within 10 s). Cancelling through a pool work item instead of the host's thread fails the three starvation tests; killing the worker before starting the cancellation fails both `TheTokenIsCancelledBeforeTheWorkerIsKilled` cases (and the measured delay rose to about 244 ms, the time the process-tree kill took). `ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent` now holds by order: the token is cancelled only after the caller's thread has stopped waiting for the answer. Against the source before the stop notice, all three `TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks` cases fail (the kill came first); removing the stop notice from the kill fails them and both `StopNoticeTests`. Letting the pool fallback's exception escape fails both `WithNeitherAThreadNorAPoolWorkItem...` cases (`Invoke` throws `NotSupportedException`). Against 4e32b9f with only the seams added (the setup failure seam where the constructor took the pipes, the answer-published seam, and a `Start` overload that assigns the notice after `Start` returns, as that host did), both post-spawn tests fail (no kill went through the kill path, and no SIGKILL exit was recorded; the killed worker was gone by the time `Invoke` returned, so the pid check alone passed there) and both `WithNeitherAThreadNorAPoolWorkItem...` cases fail (`ToString()` did not show the flag); the other kill-order and stop-notice tests pass. Removing the post-handler fault check fails the `DeadlineAfterTheHandler` case (the kill came from the frame write's check); a handler thread slowed by 300 ms passes with the answer-published wait and fails without it (the handler wait's check killed the worker). The 3 kill-order and 5 stop-notice tests passed 50 consecutive runs on each runtime. Repeating `TheTokenIsCancelledBeforeTheWorkerIsKilled` 150 times per case under the real clock passed every run; a reviewer's probe (an 800 ms deadline, or a caller cancellation after 100 to 150 ms, 150 runs each) found the kill first in 8 and 0 runs against the source before the stop notice on this machine, and in 0 and 0 after it | | Build | `TreatWarningsAsErrors`, nullable reference types, and the `8.0-recommended` analyzers are clean with both SDKs; the .NET 8 SDK builds and passes the net8.0 suite on its own | | Pin consistency | `dotnet/src/JacquardHost/Compat.cs` is generated, and the embedded kit copy matches the pinned digests and the Python copy (`scripts/core_pin.py check`); `tests/test_core_pin.py` covers drift in both, and the offline Python suite passes | | Distribution | `scripts/check_distribution.py` passes; the source archive carries the .NET sources, `dotnet/NOTICE`, and the kit copy | diff --git a/dotnet/README.md b/dotnet/README.md index 9a13aac..6edbe11 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -147,8 +147,12 @@ one per invocation, so a busy thread pool does not delay it. If no thread can be created, it queues the cancellation as a thread-pool work item instead, which runs when a pool thread is free; if that cannot be queued either, the host gives up: the token is never cancelled, `IsCancelled` still turns true, -and the result's `HandlerTokenNotCancelled` is set. `Invoke` still returns -its result and disposes the worker. Callbacks registered on the token run on +and the result's `HandlerTokenNotCancelled` is set (its `ToString()` then +ends with `handler token not cancelled`). The flag is not part of +`HostEvidence`, which describes how the invocation ended in the schema the +adapters share and stays empty on success; giving up can follow any outcome, +including a successful one. `Invoke` still returns its result and disposes +the worker. Callbacks registered on the token run on that thread (or that pool thread), never on the caller's thread or the thread that cancelled the caller's token; the host does not wait for them, and anything they throw is discarded. A callback that blocks holds only that @@ -188,8 +192,13 @@ thread. The carrier is built around both: most 1 MiB per read pass, and only a bounded prefix (`max_stderr_bytes`) is kept. - **Expiry kills a process tree, not a group.** At expiry, cancellation, or - disposal of a running worker, the host calls `Process.Kill(true)`: .NET - stops the worker, lists its children from the process table, kills it, and + disposal of a running worker, and when the spawned worker's pipes cannot + be set up (redirected streams that are not pipe streams), the host calls + `Process.Kill(true)` through one kill path, which first starts the + cancellation of the handlers' token. Disposal waits up to one second for a + killed worker to exit. So does a failed setup, which reports a spawn + failure and says so if the worker was still running then. .NET stops the + worker, lists its children from the process table, kills it, and does the same for each child. A descendant that was already reparented is not found and keeps running: one whose parent exited, or one that daemonized. So is one started after its parent was listed. Descendants are From 093022dc265f974b5d6d1c2994a0931123c7a69e Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 05:26:03 -0400 Subject: [PATCH 29/33] Close the .NET worker's obtained pipe streams when checking them fails --- dotnet/src/JacquardHost/Carrier/Worker.cs | 57 +++++++++++++++---- .../JacquardHost.Tests/DescriptorTests.cs | 46 +++++++++++++-- 2 files changed, 88 insertions(+), 15 deletions(-) diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index dbcd7a8..0ee2943 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -143,6 +143,15 @@ private Worker(Process process, long expires, Action? stopNotice, CancellationTo /// internal static volatile Action? SpawnedHook; + /// + /// Test seam: runs for each redirected stream (stdin, stdout, then + /// stderr) as the carrier checks that it is a pipe stream, after all + /// three were obtained from the . A test throws + /// from it to stand for a stream + /// that is not a pipe stream. Process-global; null outside tests. + /// + internal static volatile Action? DescriptorHook; + private volatile Action? stopNotice; /// @@ -289,18 +298,43 @@ internal static Worker Start(string store, WorkerOptions options, Action? stopNo return worker; } - /// Takes the spawned worker's pipes, all or none. + /// + /// Takes the spawned worker's pipes, all or none. On a failure it closes + /// every stream it already obtained: once the caller has read them, + /// Process.Dispose leaves them open, and nothing else + /// would close them before finalization. + /// /// A redirected stream is not a pipe stream. private void Attach() { SpawnedHook?.Invoke(process!); - StreamWriter writer = process!.StandardInput; - Stream input = writer.BaseStream; - Stream output = process.StandardOutput.BaseStream; - Stream error = process.StandardError.BaseStream; - int inputFd = Descriptor(input); - int outputFd = Descriptor(output); - int errorFd = Descriptor(error); + Stream? input = null; + Stream? output = null; + Stream? error = null; + StreamWriter writer; + int inputFd; + int outputFd; + int errorFd; + try + { + writer = process!.StandardInput; + input = writer.BaseStream; + output = process.StandardOutput.BaseStream; + error = process.StandardError.BaseStream; + inputFd = Descriptor(input); + outputFd = Descriptor(output); + errorFd = Descriptor(error); + } + catch + { + // Close the base streams rather than the writer, whose disposal + // would flush into a worker about to be killed. + input?.Dispose(); + output?.Dispose(); + error?.Dispose(); + throw; + } + stdinWriter = writer; stdin = input; stdout = output; @@ -346,10 +380,13 @@ internal static long Expires(long now, TimeSpan deadline) return ticks >= long.MaxValue - now ? long.MaxValue : now + (long)ticks; } - private static int Descriptor(Stream stream) => - stream is PipeStream pipe + private static int Descriptor(Stream stream) + { + DescriptorHook?.Invoke(stream); + return stream is PipeStream pipe ? (int)pipe.SafePipeHandle.DangerousGetHandle() : throw new NotSupportedException("worker pipes are " + stream.GetType().Name + ", not pipe streams"); + } private static bool Stopped(long expires, CancellationToken cancel) => cancel.IsCancellationRequested || Now() >= expires; diff --git a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs index 8030230..648bc14 100644 --- a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs +++ b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs @@ -29,30 +29,66 @@ public sealed class DescriptorTests public void ClearSeams() { Worker.SpawnedHook = null; + Worker.DescriptorHook = null; Worker.ExpireHook = null; } - [Test] - public void AFailureAfterTheSpawnKillsTheWorkerThroughTheKillPathAndReleasesIt() + /// Where the seam fails the worker's setup after Process.Start succeeded. + public enum SetupFailure + { + /// Before the carrier obtains any redirected stream. + BeforeThePipes, + + /// While the carrier checks the streams it obtained: stderr is not a pipe stream. + WhileCheckingThePipes, + } + + [TestCase(SetupFailure.BeforeThePipes)] + [TestCase(SetupFailure.WhileCheckingThePipes)] + public void AFailureAfterTheSpawnKillsTheWorkerThroughTheKillPathAndReleasesIt(SetupFailure failure) { // A seam fails the worker's setup after Process.Start succeeded, as - // redirected streams that are not pipe streams would. + // redirected streams that are not pipe streams would. In the second + // case all three streams were obtained, and stdin and stdout were + // checked, before the stderr check fails. Assert.That(InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))).Ok, Is.True); // warm up int before = Descriptors(); var pids = new List(); int kills = 0; + int checkedStreams = 0; Worker.SpawnedHook = process => { pids.Add(process.Id); - throw new NotSupportedException("seam: worker pipes are not pipe streams"); + if (failure == SetupFailure.BeforeThePipes) + { + throw new NotSupportedException("seam: worker pipes are not pipe streams"); + } }; + if (failure == SetupFailure.WhileCheckingThePipes) + { + Worker.DescriptorHook = _ => + { + if (Interlocked.Increment(ref checkedStreams) % 3 == 0) + { + throw new NotSupportedException("seam: worker pipes are not pipe streams"); + } + }; + } + Worker.ExpireHook = () => Interlocked.Increment(ref kills); try { for (int i = 0; i < 3; i++) { InvocationResult result = Within(TimeSpan.FromSeconds(20), () => InvokeReady(Faulty("silent", TimeSpan.FromSeconds(30)))); + + // Release is synchronous here: nothing outlives Invoke that + // holds a host pipe, so the count is read at once, with no + // window in which a collection could finalize a lost stream. + int after = Descriptors(); + Assert.That(after, Is.LessThanOrEqualTo(before), "descriptors grew from " + before + " to " + after + " after invocation " + (i + 1)); Assert.That(pids, Has.Count.EqualTo(i + 1), "the seam did not run"); + Assert.That(Volatile.Read(ref checkedStreams), Is.EqualTo(failure == SetupFailure.BeforeThePipes ? 0 : 3 * (i + 1)), "the streams were not checked as the seam expects"); Assert.That(Alive(pids[i]), Is.False, "the spawned worker is still running after Invoke returned"); Assert.That(result.Kind, Is.EqualTo(ResultKind.CarrierLoss), result.ToString()); Assert.That(Reason(result), Is.EqualTo("worker_spawn_failed"), result.ToString()); @@ -65,11 +101,11 @@ public void AFailureAfterTheSpawnKillsTheWorkerThroughTheKillPathAndReleasesIt() Assert.That(Volatile.Read(ref kills), Is.EqualTo(3), "the kill did not go through the common kill path once per worker"); Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "adapter threads remain"); - Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); } finally { Worker.SpawnedHook = null; + Worker.DescriptorHook = null; foreach (int pid in pids) { if (Alive(pid)) From abf75cc1b663b6b97eb702a2544f7bc2ff6cdcde Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 05:26:24 -0400 Subject: [PATCH 30/33] Leave only the runtime's and test output's assemblies out of the .NET descriptor counts --- .../JacquardHost.Tests/DescriptorTests.cs | 25 +++++++++++ dotnet/test/JacquardHost.Tests/Support.cs | 41 ++++++++++++++++--- 2 files changed, 60 insertions(+), 6 deletions(-) diff --git a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs index 648bc14..15d9eb4 100644 --- a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs +++ b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs @@ -33,6 +33,31 @@ public void ClearSeams() Worker.ExpireHook = null; } + [Test] + public void ADescriptorOnADllOutsideTheRuntimeAndTestDirectoriesIsCounted() + { + // Only assembly files the runtime loads from its own directory or the + // test output are left out of the count; any other .dll is counted. + string dir = Directory.CreateTempSubdirectory("jacquard-dll-").FullName; + string path = Path.Combine(dir, "Leak.dll"); + try + { + File.WriteAllBytes(path, new byte[] { 0 }); + int before = Descriptors(); + using (File.OpenRead(path)) + { + Assert.That(Descriptors(), Is.EqualTo(before + 1), "a descriptor on " + path + " was not counted"); + } + + Assert.That(Descriptors(), Is.EqualTo(before)); + } + finally + { + File.Delete(path); + Directory.Delete(dir); + } + } + /// Where the seam fails the worker's setup after Process.Start succeeded. public enum SetupFailure { diff --git a/dotnet/test/JacquardHost.Tests/Support.cs b/dotnet/test/JacquardHost.Tests/Support.cs index 26d1c22..34ed3a8 100644 --- a/dotnet/test/JacquardHost.Tests/Support.cs +++ b/dotnet/test/JacquardHost.Tests/Support.cs @@ -4,6 +4,7 @@ using System.Globalization; using System.IO; using System.Linq; +using System.Runtime.InteropServices; using System.Text; using System.Threading; using JacquardLang.Host.Carrier; @@ -244,11 +245,20 @@ internal static T Within(TimeSpan limit, Func body) } /// - /// Open descriptors of this process (Linux), except those on assembly - /// files: the runtime opens an assembly when it first loads it, which - /// can happen in the middle of a test (for example when tiered - /// compilation recompiles a method), and keeps it open. The adapter opens - /// no assembly files. + /// The directories whose assembly files the runtime may open in the + /// middle of a test (for example when tiered compilation recompiles a + /// method) and keep open: the shared runtime's and the test output's. + /// + private static readonly string[] AssemblyDirectories = + { + Path.TrimEndingDirectorySeparator(Path.GetFullPath(RuntimeEnvironment.GetRuntimeDirectory())) + Path.DirectorySeparatorChar, + Path.TrimEndingDirectorySeparator(Path.GetFullPath(AppContext.BaseDirectory)) + Path.DirectorySeparatorChar, + }; + + /// + /// Open descriptors of this process (Linux), except those on + /// .dll files directly in . The + /// adapter opens no assembly files; any other .dll is counted. /// internal static int Descriptors() { @@ -266,7 +276,7 @@ internal static int Descriptors() continue; } - if (target is null || !target.EndsWith(".dll", StringComparison.Ordinal)) + if (target is null || !IsRuntimeAssembly(target)) { count++; } @@ -275,6 +285,25 @@ internal static int Descriptors() return count; } + private static bool IsRuntimeAssembly(string target) + { + if (!target.EndsWith(".dll", StringComparison.Ordinal)) + { + return false; + } + + string directory = Path.GetDirectoryName(target) + Path.DirectorySeparatorChar; + foreach (string allowed in AssemblyDirectories) + { + if (string.Equals(directory, allowed, StringComparison.Ordinal)) + { + return true; + } + } + + return false; + } + /// Live threads of this process named by the adapter (Linux, names truncated to 15 bytes). internal static int AdapterThreads() { From 1eeb122616e455d68d0d2dbd4ad5aba7bbbe7969 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 05:26:24 -0400 Subject: [PATCH 31/33] Read the .NET descriptor counts at once after each release instead of polling --- .../JacquardHost.Tests/CancelStartTests.cs | 5 +- .../JacquardHost.Tests/DescriptorTests.cs | 47 ++++++++++++++----- 2 files changed, 39 insertions(+), 13 deletions(-) diff --git a/dotnet/test/JacquardHost.Tests/CancelStartTests.cs b/dotnet/test/JacquardHost.Tests/CancelStartTests.cs index c14ea32..fc005c3 100644 --- a/dotnet/test/JacquardHost.Tests/CancelStartTests.cs +++ b/dotnet/test/JacquardHost.Tests/CancelStartTests.cs @@ -155,7 +155,10 @@ public void WithNeitherAThreadNorAPoolWorkItemInvokeReturnsAndDisposesTheWorker( { Assert.That(thrown, Is.Null, "Invoke threw: " + thrown); Assert.That(Eventually(() => Children() == 0), Is.True, "the worker was not disposed: it is still running"); - Assert.That(Eventually(() => Descriptors() <= before), Is.True, "the worker was not disposed: descriptors grew from " + before + " to " + Descriptors()); + // Disposal closes the pipes before Invoke returns, so the count + // is read at once rather than polled. + int after = Descriptors(); + Assert.That(after, Is.LessThanOrEqualTo(before), "the worker was not disposed: descriptors grew from " + before + " to " + after); })); Assert.That(Volatile.Read(ref attempts), Is.EqualTo(2), "the host did not try a thread and then the pool, once"); Assert.That(result.HandlerTokenNotCancelled, Is.True, "the failure was not recorded"); diff --git a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs index 15d9eb4..8b9b2e7 100644 --- a/dotnet/test/JacquardHost.Tests/DescriptorTests.cs +++ b/dotnet/test/JacquardHost.Tests/DescriptorTests.cs @@ -23,7 +23,21 @@ namespace JacquardLang.Host.Tests; [Platform("Linux")] public sealed class DescriptorTests { - private static bool DescriptorsBackTo(int before) => Eventually(() => Descriptors() <= before); + /// + /// Asserts the count is back to , read once + /// with no polling, right after the call that should have released it. Every path these tests take releases the host's + /// descriptors synchronously before Invoke (or the probe) returns: + /// disposal closes the pipes on the calling thread, and a descendant + /// that still holds the worker's ends holds its own copies, not the + /// host's. A poll would give a collection time to finalize a lost + /// stream and hide the leak. Only thread counts are polled, since the + /// short-lived cancel thread ends after Invoke returns. + /// + private static void AssertReleased(int before, string what) + { + int after = Descriptors(); + Assert.That(after, Is.LessThanOrEqualTo(before), what + ": descriptors grew from " + before + " to " + after); + } [TearDown] public void ClearSeams() @@ -150,20 +164,20 @@ public void InvocationsReleaseTheirPipesAndThreads() for (int i = 0; i < 5; i++) { InvocationResult result = InvokeReady(Faulty("ok", TimeSpan.FromSeconds(10))); + AssertReleased(before, "invocation " + (i + 1)); Assert.That(result.Ok, Is.True, result.ToString()); } - Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); Assert.That(AdapterThreads(), Is.EqualTo(0), "adapter threads outlived their invocations"); // A daemonized descendant keeps holding stderr after the invocation // returns; the host does not wait for it, and its pipes are closed. InvocationResult escaped = InvokeReady(Faulty("escaped-descendant", TimeSpan.FromSeconds(10))); + int released = Descriptors(); int pid = Descendant("escaped-descendant", escaped.Stderr); - bool released = DescriptorsBackTo(before); int threads = AdapterThreads(); Kill(pid); - Assert.That(released, Is.True, "escaped descendant: descriptors grew from " + before + " to " + Descriptors()); + Assert.That(released, Is.LessThanOrEqualTo(before), "escaped descendant: descriptors grew from " + before + " to " + released); Assert.That(threads, Is.EqualTo(0), "escaped descendant: adapter threads remain"); // Handlers run on their own threads, which end with the handler; the @@ -182,27 +196,34 @@ public void InvocationsReleaseTheirPipesAndThreads() { Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "an earlier invocation's threads remain"); InvocationResult result = replaying.Invoke(Pinned.Identity("kit-echo"), Echo(), new Value[] { new TextValue("ping") }, registry); + AssertReleased(before, "handler invocation " + (i + 1)); Assert.That(result.Ok, Is.True, result.ToString()); } Assert.That(during, Is.EqualTo(1), "the handler thread was not seen while it ran"); Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "handler threads remain"); - Assert.That(DescriptorsBackTo(before), Is.True, "handlers: descriptors grew from " + before + " to " + Descriptors()); // Failure paths release the same resources: expiry, cancellation, a // truncated frame, and a spawn failure. for (int i = 0; i < 3; i++) { - Assert.That(Reason(InvokeReady(Faulty("silent", TimeSpan.FromMilliseconds(500)))), Is.EqualTo("deadline")); + InvocationResult expired = InvokeReady(Faulty("silent", TimeSpan.FromMilliseconds(500))); + AssertReleased(before, "expiry"); + Assert.That(Reason(expired), Is.EqualTo("deadline")); using var cancelled = new CancellationTokenSource(); After(TimeSpan.FromMilliseconds(200), cancelled.Cancel); - Assert.That(Reason(InvokeReady(Faulty("silent", TimeSpan.FromSeconds(10)), cancelled.Token)), Is.EqualTo("aborted")); - Assert.That(InvokeReady(Faulty("truncated", TimeSpan.FromSeconds(10))).Ok, Is.False); - Assert.That(Reason(InvokeReady(new Host("unused", new HostOptions { Carrier = new WorkerOptions { Command = new[] { "/no/such/executable" } } }))), Is.EqualTo("worker_spawn_failed")); + InvocationResult aborted = InvokeReady(Faulty("silent", TimeSpan.FromSeconds(10)), cancelled.Token); + AssertReleased(before, "cancellation"); + Assert.That(Reason(aborted), Is.EqualTo("aborted")); + InvocationResult truncated = InvokeReady(Faulty("truncated", TimeSpan.FromSeconds(10))); + AssertReleased(before, "truncated frame"); + Assert.That(truncated.Ok, Is.False); + InvocationResult unspawned = InvokeReady(new Host("unused", new HostOptions { Carrier = new WorkerOptions { Command = new[] { "/no/such/executable" } } })); + AssertReleased(before, "spawn failure"); + Assert.That(Reason(unspawned), Is.EqualTo("worker_spawn_failed")); } Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "failure paths: adapter threads remain"); - Assert.That(DescriptorsBackTo(before), Is.True, "failure paths: descriptors grew from " + before + " to " + Descriptors()); } [Test] @@ -216,13 +237,14 @@ public void AnEscapedDescendantHoldingStdinLeavesNoThreadOrDescriptor() int before = Descriptors(); InvocationResult result = Within(TimeSpan.FromSeconds(15), () => Faulty("stdin-descendant", TimeSpan.FromSeconds(2)) .Invoke(Pinned.Identity("kit-ready"), Echo(withWorld: false), new Value[] { new TextValue(new string('x', 220_000)) }, new OperationRegistry())); + int after = Descriptors(); int pid = Descendant("stdin-descendant", result.Stderr); try { Assert.That(Reason(result), Is.EqualTo("deadline"), result.ToString()); Assert.That(Alive(pid), Is.True); Assert.That(AdapterThreads(), Is.EqualTo(0)); - Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); + Assert.That(after, Is.LessThanOrEqualTo(before), "descriptors grew from " + before + " to " + after); } finally { @@ -278,6 +300,7 @@ private static void AssertFloodDoesNotHold(string script, TimeSpan deadline, Tim var watch = Stopwatch.StartNew(); InvocationResult result = Within(TimeSpan.FromSeconds(30), () => new Host("unused", new HostOptions { Carrier = options }).ShutdownProbe(cancel.Token)); TimeSpan elapsed = watch.Elapsed; + int after = Descriptors(); long heapAfter = GC.GetTotalMemory(forceFullCollection: false); Assert.That(Eventually(() => File.Exists(pidFile) && File.ReadAllText(pidFile).Trim().Length > 0), Is.True, "the descendant never started"); int pid = int.Parse(File.ReadAllText(pidFile).Trim(), CultureInfo.InvariantCulture); @@ -301,6 +324,6 @@ private static void AssertFloodDoesNotHold(string script, TimeSpan deadline, Tim } Assert.That(Eventually(() => AdapterThreads() == 0), Is.True, "adapter threads remain"); - Assert.That(DescriptorsBackTo(before), Is.True, "descriptors grew from " + before + " to " + Descriptors()); + Assert.That(after, Is.LessThanOrEqualTo(before), "descriptors grew from " + before + " to " + after); } } From 03eca5dd438a1d669eb906aa0f9da696076b6331 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 05:26:59 -0400 Subject: [PATCH 32/33] Keep the .NET kill-path methods out of inlining so the kill-order stacks hold in Release --- dotnet/src/JacquardHost/Carrier/Worker.cs | 5 +++++ dotnet/src/JacquardHost/Host.cs | 5 +++++ dotnet/test/JacquardHost.Tests/KillOrderTests.cs | 4 +++- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/dotnet/src/JacquardHost/Carrier/Worker.cs b/dotnet/src/JacquardHost/Carrier/Worker.cs index 0ee2943..0153467 100644 --- a/dotnet/src/JacquardHost/Carrier/Worker.cs +++ b/dotnet/src/JacquardHost/Carrier/Worker.cs @@ -5,6 +5,7 @@ using System.Diagnostics; using System.IO; using System.IO.Pipes; +using System.Runtime.CompilerServices; using System.Threading; using JacquardLang.Host.Protocol; @@ -737,6 +738,8 @@ private int Fill(byte[] buffer) /// blocks; between writes the caller waits under the lifetime. /// /// A loss: the input is closed, the write failed, or the lifetime ended. + // Not inlined, so the kill-order tests can name it on the killing thread's stack in any build. + [MethodImpl(MethodImplOptions.NoInlining)] public void WriteBytes(ReadOnlySpan bytes) { lock (gate) @@ -809,6 +812,8 @@ public void WriteFrame(IReadOnlyDictionary frame, Limits limits) /// before it kills the worker. /// /// The lifetime ended first. + // Not inlined, so the kill-order tests can name it on the killing thread's stack in any build. + [MethodImpl(MethodImplOptions.NoInlining)] public void Await(Func isDone) { ArgumentNullException.ThrowIfNull(isDone); diff --git a/dotnet/src/JacquardHost/Host.cs b/dotnet/src/JacquardHost/Host.cs index 94bb59e..d3c4308 100644 --- a/dotnet/src/JacquardHost/Host.cs +++ b/dotnet/src/JacquardHost/Host.cs @@ -2,6 +2,7 @@ using System.Collections.Generic; using System.Diagnostics.CodeAnalysis; using System.Linq; +using System.Runtime.CompilerServices; using System.Text; using System.Threading; using JacquardLang.Host.Carrier; @@ -422,6 +423,8 @@ internal void Drive(Func opening, Limits limits) /// and renders the response. A handler still running at the deadline is /// abandoned: no late response is sent and nothing is retried. /// + // Not inlined, so the kill-order tests can name it on the killing thread's stack in any build. + [MethodImpl(MethodImplOptions.NoInlining)] private JsonObject Respond(EffectRequest request) { RequestDecodedHook?.Invoke(); @@ -475,6 +478,8 @@ private JsonObject Respond(EffectRequest request) } } + // Not inlined, so the kill-order tests can name it on the killing thread's stack in any build. + [MethodImpl(MethodImplOptions.NoInlining)] private Answer RunHandler(Handler handler, IReadOnlyList arguments) { HandlerContext context = new(() => Worker.Stopping, () => ended, HandlerToken()); diff --git a/dotnet/test/JacquardHost.Tests/KillOrderTests.cs b/dotnet/test/JacquardHost.Tests/KillOrderTests.cs index 608b8f2..1c96550 100644 --- a/dotnet/test/JacquardHost.Tests/KillOrderTests.cs +++ b/dotnet/test/JacquardHost.Tests/KillOrderTests.cs @@ -18,7 +18,9 @@ namespace JacquardLang.Host.Tests; /// stop exactly between two such reads, so the outcome does not depend on /// timing. Each case also asserts which check killed the worker, from the /// killing thread's stack: the handler wait's fault check, or the call's -/// check after the handler returned. +/// check after the handler returned. The methods named (Respond, +/// RunHandler, Await, WriteBytes) are marked +/// NoInlining, so their frames are on the stack in a Release build too. /// [TestFixture] public sealed class KillOrderTests From d53f719de14c7c17890ea8230e1514f365bc9cd3 Mon Sep 17 00:00:00 2001 From: Josh Winters Date: Thu, 29 Oct 2026 05:39:48 -0400 Subject: [PATCH 33/33] Record the .NET pipe-check cleanup evidence: 96 offline and 101 live tests --- docs/EVIDENCE.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/EVIDENCE.md b/docs/EVIDENCE.md index 739d2d0..a7f73fd 100644 --- a/docs/EVIDENCE.md +++ b/docs/EVIDENCE.md @@ -136,9 +136,9 @@ dependencies. | Lane | Result | |---|---| -| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 94 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 15 handler-token tests, 3 kill-order tests, 5 stop-notice tests, 3 cancellation-start tests (Linux), 8 process-tree tests (Unix), 8 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline cancels the token; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. With a test clock, 36- and 48-hour lifetimes leave the token uncancelled a day and a minute in and one second before the deadline (`IsCancelled` false too), and cancel it once the clock reaches the deadline, with no late response; the cancellation came 2 to 4 ms after the clock reached it in these runs (an observation, not asserted). With every thread-pool thread and 40 more blocked and a probe work item queued behind them, the token is cancelled at a 1 s deadline, at the caller's cancellation, and at the end of a successful invocation before the probe runs. A seam that holds the worker's kill until the token's callback has run sees the callback first, at the deadline and at caller cancellation; with a test clock that lands the stop exactly between two reads of the stop condition (the deadline or the caller's cancellation between the handler wait's predicate and its fault check, and the deadline between the post-handler stopping check and its fault check), it still sees the callback first; the post-handler case arms the clock only after a seam reports the handler's answer published, and each case asserts from the killing thread's stack which check killed the worker (the handler wait's, or the one after the handler returned). Every kill of the worker uses a stop notice of its own, given before it on the killing thread (a check that `notice, kill, kill` fails is itself tested): disposal with no operation running, on the test's thread and on a separate disposer thread (which does the kill); disposal from another thread landing between a wait's two checks (the waiting thread kills, and any later kill is the disposer's); and a failure to set up the spawned worker's pipes. For that last case a seam throws `NotSupportedException` after `Process.Start`, as non-pipe streams would: `Start` returns with the worker already exited (SIGKILL recorded, its pid no longer alive), the first operation reports the spawn failure, and through `Invoke` the result is a carrier loss with reason `worker_spawn_failed`, code E1611, no frames, and no growth in descriptors or adapter threads over three runs. When the cancellation thread cannot be created (a seam throws `OutOfMemoryException`), the token is cancelled on a pool thread; when the pool work item fails too (`NotSupportedException`), `Invoke` returns its result (at a 1 s deadline, and on success), sets `HandlerTokenNotCancelled` (shown by the result's `ToString()`; on success `HostEvidence` stays empty), runs no callback, leaves the token uncancelled while `IsCancelled` is true, and disposes the worker (no child process or descriptor remains). Callbacks run on the host's `jacquard-handler-cancel` thread, not a pool thread; one registered after the cancellation runs on the handler's own thread. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors (descriptors on the runtime's own assembly files, which it opens when it first loads an assembly, are not counted): no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | -| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 99 tests pass on .NET 8 (built with the .NET 8 SDK alone, and with the .NET 10 SDK) and on .NET 10, none ignored: the 94 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | -| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched`. Against the source with the handler token's timer and pool work items (with the test clock seam copied in, and with the timer's first wait shortened to 100 ms to stand for the day it waits), both `ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore` cases fail (the token is cancelled with 12 and 24 hours left), and the three `UnderAStarvedThreadPool...` tests fail (the token is not cancelled within 10 s). Cancelling through a pool work item instead of the host's thread fails the three starvation tests; killing the worker before starting the cancellation fails both `TheTokenIsCancelledBeforeTheWorkerIsKilled` cases (and the measured delay rose to about 244 ms, the time the process-tree kill took). `ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent` now holds by order: the token is cancelled only after the caller's thread has stopped waiting for the answer. Against the source before the stop notice, all three `TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks` cases fail (the kill came first); removing the stop notice from the kill fails them and both `StopNoticeTests`. Letting the pool fallback's exception escape fails both `WithNeitherAThreadNorAPoolWorkItem...` cases (`Invoke` throws `NotSupportedException`). Against 4e32b9f with only the seams added (the setup failure seam where the constructor took the pipes, the answer-published seam, and a `Start` overload that assigns the notice after `Start` returns, as that host did), both post-spawn tests fail (no kill went through the kill path, and no SIGKILL exit was recorded; the killed worker was gone by the time `Invoke` returned, so the pid check alone passed there) and both `WithNeitherAThreadNorAPoolWorkItem...` cases fail (`ToString()` did not show the flag); the other kill-order and stop-notice tests pass. Removing the post-handler fault check fails the `DeadlineAfterTheHandler` case (the kill came from the frame write's check); a handler thread slowed by 300 ms passes with the answer-published wait and fails without it (the handler wait's check killed the worker). The 3 kill-order and 5 stop-notice tests passed 50 consecutive runs on each runtime. Repeating `TheTokenIsCancelledBeforeTheWorkerIsKilled` 150 times per case under the real clock passed every run; a reviewer's probe (an 800 ms deadline, or a caller cancellation after 100 to 150 ms, 150 runs each) found the kill first in 8 and 0 runs against the source before the stop notice on this machine, and in 0 and 0 after it | +| .NET offline (`dotnet test test/JacquardHost.Tests -f net8.0`, and `-f net10.0`, without Core) | 96 tests pass on each runtime, and the 5 live tests are ignored: 18 codec and session tests, 32 kit-replay and host-fault tests, 15 handler-token tests, 3 kill-order tests, 5 stop-notice tests, 3 cancellation-start tests (Linux), 8 process-tree tests (Unix), 10 descriptor tests (Linux), and the NOTICE and README-example copy checks. Coverage: strict JSON, Unicode, and duplicate keys, including invalid, overlong, and encoded-surrogate UTF-8, a byte-order mark, and non-ASCII `\u` digits; integers parsed exactly from their text (2^53 + 1 and both 63-bit bounds round-trip; `-0` is 0 and tokens over 20 characters are refused, as in Rust and Java); code-point key order; exact Int and Real codecs; limits; Core-order descriptor checks; hello and selection; invoke-frame validation; session ordering and one response slot; the shared outcome node budget. Fault cases run against a stand-in program: malformed, extra, and noncanonical Core frames; EOF, truncated, silent, stalled, and nonzero workers; 2 MiB of stderr before the hello, drained and cut to the selected ceiling; frames sent before selection read under the selected limits (E1602); a worker that never reads its input, so the invoke frame cannot be written, ends at the 2 s deadline; cancellation; a stop before the spawn starts no worker; local refusal before spawn; null arguments refused before spawn; a registry over `max_operations` refused locally; handler answers (structured failures, exceptions, `null`, an out-of-range Int, and a message cut at a code point with a lone surrogate replaced); a handler that ignores cancellation for 3 s does not hold a call with a 1 s deadline; a cooperative handler sees its token cancelled and no late response is sent; disposal from another thread ends a waiting read as an abort and kills the worker; poll timeouts are never negative, never over 5 ms, and never end after the deadline, for deadlines up to `TimeSpan.MaxValue`; with a 365-day or `TimeSpan.MaxValue` deadline, cancellation after 300 ms ends a waiting read, and an invocation, within 5 s; an undefined `FailureCategory` is refused by `HandlerException`, and one forged past the constructor is answered as completion unknown; an operation bound after the invoke frame advertised none is answered as unsupported and its handler does not run. Handler token: a callback registered on `HandlerContext.CancellationToken` that throws does not escape `Invoke` on success, does not reach the thread that cancels the caller's token, and does not end the process when the deadline cancels the token; one that blocks holds neither `Invoke` (on success, at caller cancellation, at a 1 s deadline) nor the canceller's thread; under a 365-day deadline the caller's cancellation still cancels the token. With a test clock, 36- and 48-hour lifetimes leave the token uncancelled a day and a minute in and one second before the deadline (`IsCancelled` false too), and cancel it once the clock reaches the deadline, with no late response; the cancellation came 2 to 4 ms after the clock reached it in these runs (an observation, not asserted). With every thread-pool thread and 40 more blocked and a probe work item queued behind them, the token is cancelled at a 1 s deadline, at the caller's cancellation, and at the end of a successful invocation before the probe runs. A seam that holds the worker's kill until the token's callback has run sees the callback first, at the deadline and at caller cancellation; with a test clock that lands the stop exactly between two reads of the stop condition (the deadline or the caller's cancellation between the handler wait's predicate and its fault check, and the deadline between the post-handler stopping check and its fault check), it still sees the callback first; the post-handler case arms the clock only after a seam reports the handler's answer published, and each case asserts from the killing thread's stack which check killed the worker (the handler wait's, or the one after the handler returned). Every kill of the worker uses a stop notice of its own, given before it on the killing thread (a check that `notice, kill, kill` fails is itself tested): disposal with no operation running, on the test's thread and on a separate disposer thread (which does the kill); disposal from another thread landing between a wait's two checks (the waiting thread kills, and any later kill is the disposer's); and a failure to set up the spawned worker's pipes. For that last case a seam throws `NotSupportedException` after `Process.Start`, as non-pipe streams would, either before the carrier obtains any redirected stream or while it checks the three it obtained (the stderr check fails); the carrier then closes every stream it obtained, since `Process.Dispose` leaves streams the caller has read open: `Start` returns with the worker already exited (SIGKILL recorded, its pid no longer alive), the first operation reports the spawn failure, and through `Invoke` the result is a carrier loss with reason `worker_spawn_failed`, code E1611, no frames, and no growth in descriptors (read once, right after each `Invoke` returns) or adapter threads over three runs. When the cancellation thread cannot be created (a seam throws `OutOfMemoryException`), the token is cancelled on a pool thread; when the pool work item fails too (`NotSupportedException`), `Invoke` returns its result (at a 1 s deadline, and on success), sets `HandlerTokenNotCancelled` (shown by the result's `ToString()`; on success `HostEvidence` stays empty), runs no callback, leaves the token uncancelled while `IsCancelled` is true, and disposes the worker (no child process or descriptor remains). Callbacks run on the host's `jacquard-handler-cancel` thread, not a pool thread; one registered after the cancellation runs on the handler's own thread. Process tree: a child, and a grandchild under a waiting shell, holding the pipes are killed at the deadline; with the worker alive and its child holding stdin without reading, expiry and cancellation each return within 4 s and kill the child; a descendant that daemonized while the worker ran, one left by a launcher that exited, and one holding stdin after the worker exited are not found and survive; a descendant holding stdout or stderr after the worker exits is reported as a carrier loss after the one-second grace, with the exit status kept (0, or 74); the exit wait obeys the deadline and reports SIGKILL; a slow clean exit succeeds; cleanup after an early protocol error is not held by an escaped descendant. Descriptors (descriptors on `.dll` files directly in the shared runtime's directory or the test output, which the runtime opens when it first loads an assembly, are not counted; a `.dll` elsewhere is, which a test checks): each count is read once, right after the call that releases the host's pipes returns, with no polling window in which a collection could finalize a lost stream (only adapter-thread counts are polled, for the cancel thread that ends after `Invoke` returns); no descriptor or adapter-thread growth across successful invocations, an escaped descendant, handler calls (the handler thread is seen while it runs), and repeated expiry, cancellation, truncated-frame, and spawn failures; an escaped descendant holding stdin leaves no thread or descriptor behind; a reparented `yes` flooding stderr at the worker's own exit, at a 500 ms deadline, and at cancellation after 500 ms, one flooding stdout, and a silent descendant holding both, each return within bound, grow the heap by less than 64 MiB, and (for the floods) leave the writer without a reader so it exits. Kit: all 21 recordings replay through the stand-in, absent expectations diverge as in the reference, prose drift is reported without normalization, and a worker that sends 300 frames of 600 KB after the terminal is read no further than one frame past the recording | +| .NET live (`JACQUARD_REQUIRE_LIVE=1`, with `JACQUARD_BIN` and `JACQUARD_PRELUDE` set) | 101 tests pass on .NET 8 and on .NET 10, none ignored: the 96 offline tests and 5 live ones. All 21 recordings match against Core with no prose drift; pure and effectful invocations keep exact evidence, and a shutdown probe is acknowledged; the README example returns `ping!`; the four failure pairs, and other exceptions, are reported without retry or private detail; cooperative and abandoned handlers at a 500 ms deadline send no late response, and both see the context cancelled. With `JACQUARD_REQUIRE_LIVE=1` and no Core configured, the live tests fail rather than skip | +| Regression tests fail without their fix | Each guard was removed, alone, in a scratch copy, and the named test run on .NET 8: frame encoding without the byte ceiling (`EncodingAFrameFarOverTheCeilingStopsAtTheCeiling`: over 384 MiB would be allocated, and `TheEncoderStopsAtTheFrameCeiling`); no text-length check before scanning (`ATextFarOverTheTextCeilingIsRefusedBeforeItIsScanned` sees E1601 from the scan); no frame charge for texts (`ValidatingManyReferencesToALongTextChargesTheFrame`: 4 x 1022 references to one 256 KiB text are accepted); no caller check after decoding (`ACancellationWhileTheRequestIsDecodedStartsNoHandlerThread`: a handler thread starts); no check on the handler thread (`ACancellationJustBeforeTheHandlerIsCalledRunsNoHandler` and `ADeadlineJustBeforeTheHandlerIsCalledRunsNoHandler`: the handler runs); unbounded replay retention (`ReplayStopsReadingAWorkerThatSendsPastTheRecording`); a write not gated by `poll` (`AWorkerThatNeverReadsCannotHoldAWritePastTheDeadline`: the call does not return within 15 s); no bound on a caller's raw map (`ACallersRawMapIsBoundedBeforeItIsCopiedOrSorted`: all 60,000 keys are copied); a handler wait or exit wait that ignores the deadline (`AnAbandonedHandlerDoesNotHoldTheCaller`, `TheExitWaitObeysTheDeadline`); disposal not observed (`DisposalFromAnotherThreadEndsAWaitingRead`); a descendant holding stdout or stderr after the exit not detected (`DescendantsReparentedBeforeTheKillAreNotFound`, `ADescendantHoldingStderrAfterTheExitIsALoss`); an unchecked frame length (`FramesAfterSelectionAreReadUnderTheSelectedLimits`). All 15 mutations fail their tests. Against the source before the handler-token fix, `AThrowingCallbackDoesNotEscapeASuccessfulInvoke` (AggregateException out of `Invoke`), `ABlockingCallbackDoesNotHoldASuccessfulInvoke` and `ABlockingCallbackHoldsNeitherTheCancellerNorInvoke` (held 15 s), and `AThrowingCallbackDoesNotEscapeToTheCanceller` fail, and `AThrowingCallbackAtTheDeadlineDoesNotCrashTheProcess` crashes the test host; `ABlockingCallbackAtTheDeadlineDoesNotHoldInvoke` passed there too (the timer, not the caller, was held). Restoring the old poll-timeout arithmetic fails `PollTimeoutsAreNeverNegativeNorPastTheDeadline`, `CancellationEndsAReadUnderTheLongestDeadlines`, `ALongDeadlineStillEndsAnInvocationAtCancellation`, and `AHandlerTokenUnderAYearLongDeadlineIsCancelledByTheCaller` (each call waits forever). Without the category checks, both `AnUndefinedFailureCategory...` tests fail. Dispatching through the live registry instead of the snapshot fails `AnOperationBoundAfterTheInvokeFrameIsNotDispatched`. Against the source with the handler token's timer and pool work items (with the test clock seam copied in, and with the timer's first wait shortened to 100 ms to stand for the day it waits), both `ATokenUnderALifetimeOverADayIsCancelledAtTheDeadlineAndNotBefore` cases fail (the token is cancelled with 12 and 24 hours left), and the three `UnderAStarvedThreadPool...` tests fail (the token is not cancelled within 10 s). Cancelling through a pool work item instead of the host's thread fails the three starvation tests; killing the worker before starting the cancellation fails both `TheTokenIsCancelledBeforeTheWorkerIsKilled` cases (and the measured delay rose to about 244 ms, the time the process-tree kill took). `ACooperativeHandlerReturningAtTheDeadlineHasNoAnswerSent` now holds by order: the token is cancelled only after the caller's thread has stopped waiting for the answer. Against the source before the stop notice, all three `TheTokenIsCancelledBeforeAKillThatLandsBetweenTwoChecks` cases fail (the kill came first); removing the stop notice from the kill fails them and both `StopNoticeTests`. Letting the pool fallback's exception escape fails both `WithNeitherAThreadNorAPoolWorkItem...` cases (`Invoke` throws `NotSupportedException`). Against 4e32b9f with only the seams added (the setup failure seam where the constructor took the pipes, the answer-published seam, and a `Start` overload that assigns the notice after `Start` returns, as that host did), both post-spawn tests fail (no kill went through the kill path, and no SIGKILL exit was recorded; the killed worker was gone by the time `Invoke` returned, so the pid check alone passed there) and both `WithNeitherAThreadNorAPoolWorkItem...` cases fail (`ToString()` did not show the flag); the other kill-order and stop-notice tests pass. Removing the post-handler fault check fails the `DeadlineAfterTheHandler` case (the kill came from the frame write's check); a handler thread slowed by 300 ms passes with the answer-published wait and fails without it (the handler wait's check killed the worker). The 3 kill-order and 5 stop-notice tests passed 50 consecutive runs on each runtime. Repeating `TheTokenIsCancelledBeforeTheWorkerIsKilled` 150 times per case under the real clock passed every run; a reviewer's probe (an 800 ms deadline, or a caller cancellation after 100 to 150 ms, 150 runs each) found the kill first in 8 and 0 runs against the source before the stop notice on this machine, and in 0 and 0 after it. Against 66eb43d with only the stream-check seam added, the `WhileCheckingThePipes` post-spawn case fails on .NET 8 and 10 (descriptors grew by 3, from 21 to 24 and from 20 to 23, after the first invocation: the three obtained streams were left for finalization); the `BeforeThePipes` case passes there. With the earlier filter that left every `.dll` out of the count, the `.dll` counting test fails. The descriptor and cancellation-start tests, now reading counts at once, passed 10 consecutive runs on each runtime. The kill-order methods the stack assertions name (`Respond`, `RunHandler`, `Await`, `WriteBytes`) are marked `NoInlining`; the 3 kill-order tests pass in a Release build on .NET 8 and 10 (they passed in Release on 66eb43d too, so no inlining was seen on this JIT; the attribute keeps that from depending on it) | | Build | `TreatWarningsAsErrors`, nullable reference types, and the `8.0-recommended` analyzers are clean with both SDKs; the .NET 8 SDK builds and passes the net8.0 suite on its own | | Pin consistency | `dotnet/src/JacquardHost/Compat.cs` is generated, and the embedded kit copy matches the pinned digests and the Python copy (`scripts/core_pin.py check`); `tests/test_core_pin.py` covers drift in both, and the offline Python suite passes | | Distribution | `scripts/check_distribution.py` passes; the source archive carries the .NET sources, `dotnet/NOTICE`, and the kit copy |