Support JSON.jl 1.x (and add a Nix dev shell for local testing)#66
Merged
Conversation
Provides a reproducible environment (kubectl, kind) plus helper scripts (kuber-cluster-up/proxy/test/cluster-down) to run the integration suite against an isolated, pinned kind cluster. KUBECONFIG is isolated so it never reads or clobbers a system cluster (e.g. k3s) config. Julia is not pinned by Nix; it is taken from the host PATH (juliaup), since nix develop / nix-shell are impure by default. flake.nix is the primary entry point (nix develop); shell.nix is a flake-compat shim for non-flake nix-shell users. Verified end-to-end: suite passes 69/69 against the pinned kindest/node:v1.35.0 using juliaup's julia.
JSON 1.x parses objects to JSON.Object rather than the concrete
Dict{String,Any} that OpenAPI's from_json and Kuber's own helpers
dispatch on. Force dicttype=Dict{String,Any} at Kuber's parse sites
(a no-op on 0.21, the fix on 1.x) and widen the compat bound.
Verified against a kind cluster (k8s v1.35.0): 64/64 tests pass on
both JSON 1.6.1 and 0.21.4.
krynju
approved these changes
Jul 2, 2026
tanmaykm
pushed a commit
that referenced
this pull request
Jul 3, 2026
…eam processor death (#67) * fix: JSON.jl 1.x watch events + fail-fast on stream processor death Two related fixes for silent k8s watch stalls: 1. `kuber_obj` / `_kuber_obj` accepted only `Dict{String,Any}`. Under JSON.jl 1.x, watch event objects parse to `JSON.Object{String,Any}` (an `AbstractDict`), so the first event of every watch threw a MethodError inside the stream processor. Widen to `AbstractDict` — `convert(::Type{<:APIModel}, j)` already handles any AbstractDict. Follow-up to the JSON 1.x support in #66, which missed this entry point. 2. `watch(streamprocessor, ctx, watched, ...)` ran the processor in a bare `@async` under `@sync`. If the processor threw (e.g. the MethodError above), its task died but `@sync` kept waiting on the still-running HTTP watch task — a deaf watch that surfaces no error until the apiserver drops the connection (potentially 30+ minutes), while events pile up unconsumed in the buffered stream. Close the stream in a `finally` on the processor task too — symmetric with the watcher task — so processor death aborts the watch promptly and the caller sees the real exception. This mirrors client-go's reflector, where any watchHandler error exits ListAndWatch and triggers a fresh relist+rewatch. Diagnosed live: JobLoops' pod reflector went permanently stale (jobs stuck in Submitted while pods ran) because every watch stream died silently on its first event under JSON 1.x. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: regression tests for deaf-watch fixes - Watch Events: convert each received event object with kuber_obj — exercises the AbstractDict path (JSON.Object under JSON.jl 1.x) that previously threw MethodError on the first event of every watch. - New "Watch processor failure aborts watch" testset (no cluster needed): a throwing streamprocessor must propagate its error out of watch() promptly instead of leaving @sync waiting on the producer task forever. Verified to hang on the unfixed code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch contains two independent changes plus a patch version bump:
nix develop/nix-shellenvironment for running the integration test suite against a localkindcluster.v0.7.9for tagging.1. JSON.jl 1.x compatibility
JSON.jl 1.0 is a StructUtils-based rewrite with one behavioral change that affects Kuber:
JSON.parse(str)now returns objects asJSON.Object{String,Any}instead of the concreteDict{String,Any}. Kuber's conversion helpers (and OpenAPI.jl'sfrom_json) dispatch on the concreteDict{String,Any}, so parsed values would miss those methods under 1.x.Fix: route Kuber's five
JSON.parsecall sites through a small helper that pins the output type:dicttypeis a no-op on 0.21 (already the default) and restoresDict{String,Any}on 1.x — verified to apply at all nesting levels — so a single code path serves both versions. The serialize path (JSON.json(model)) needs no change:JSON.lowerisStructUtils.loweron 1.x and OpenAPI.jl (>= 0.2.2) already extends it.[compat]:JSON = "0.21"->"0.21, 1".OpenAPIis left at"0.1,0.2"— OpenAPI < 0.2.2 forbids JSON 1, so the resolver cannot mis-pair.2. Nix dev shell
flake.nix(+flake.lock) and ashell.nixcompat shim providekubectl+kindand four helper commands —kuber-cluster-up,kuber-proxy,kuber-test,kuber-cluster-down— with an isolatedKUBECONFIGso a host cluster (e.g. system k3s) is never read or clobbered. Julia is intentionally not pinned by Nix; it comes from the host (juliaup). Requires a host Docker daemon.Testing
Ran the full integration suite against a
kindcluster (k8s v1.35.0), via the new dev shell:The
put!/get/update!/watchround-trips exercise both the parse (dict dispatch) and serialize (JSON.lower) paths.