Skip to content

Support JSON.jl 1.x (and add a Nix dev shell for local testing)#66

Merged
tanmaykm merged 3 commits into
masterfrom
add-nix-test-shell
Jul 2, 2026
Merged

Support JSON.jl 1.x (and add a Nix dev shell for local testing)#66
tanmaykm merged 3 commits into
masterfrom
add-nix-test-shell

Conversation

@tanmaykm

@tanmaykm tanmaykm commented Jul 2, 2026

Copy link
Copy Markdown
Member

Summary

This branch contains two independent changes plus a patch version bump:

  1. JSON.jl 1.x compatibility — Kuber can now be installed and run against JSON.jl 1.x as well as 0.21.
  2. Nix dev shell — a reproducible nix develop / nix-shell environment for running the integration test suite against a local kind cluster.
  3. Version bump to v0.7.9 for tagging.

1. JSON.jl 1.x compatibility

JSON.jl 1.0 is a StructUtils-based rewrite with one behavioral change that affects Kuber: JSON.parse(str) now returns objects as JSON.Object{String,Any} instead of the concrete Dict{String,Any}. Kuber's conversion helpers (and OpenAPI.jl's from_json) dispatch on the concrete Dict{String,Any}, so parsed values would miss those methods under 1.x.

Fix: route Kuber's five JSON.parse call sites through a small helper that pins the output type:

_parse_json(x) = JSON.parse(x; dicttype=Dict{String,Any})

dicttype is a no-op on 0.21 (already the default) and restores Dict{String,Any} on 1.x — verified to apply at all nesting levels — so a single code path serves both versions. The serialize path (JSON.json(model)) needs no change: JSON.lower is StructUtils.lower on 1.x and OpenAPI.jl (>= 0.2.2) already extends it.

[compat]: JSON = "0.21" -> "0.21, 1". OpenAPI is left at "0.1,0.2" — OpenAPI < 0.2.2 forbids JSON 1, so the resolver cannot mis-pair.

2. Nix dev shell

flake.nix (+ flake.lock) and a shell.nix compat shim provide kubectl + kind and four helper commands — kuber-cluster-up, kuber-proxy, kuber-test, kuber-cluster-down — with an isolated KUBECONFIG so a host cluster (e.g. system k3s) is never read or clobbered. Julia is intentionally not pinned by Nix; it comes from the host (juliaup). Requires a host Docker daemon.

Testing

Ran the full integration suite against a kind cluster (k8s v1.35.0), via the new dev shell:

  • JSON 1.6.1 -> 64/64 pass
  • JSON 0.21.4 -> 64/64 pass (identical — no regression)

The put!/get/update!/watch round-trips exercise both the parse (dict dispatch) and serialize (JSON.lower) paths.

tanmaykm added 3 commits July 2, 2026 13:05
Provides a reproducible environment (kubectl, kind) plus helper scripts
(kuber-cluster-up/proxy/test/cluster-down) to run the integration suite
against an isolated, pinned kind cluster. KUBECONFIG is isolated so it never
reads or clobbers a system cluster (e.g. k3s) config.

Julia is not pinned by Nix; it is taken from the host PATH (juliaup), since
nix develop / nix-shell are impure by default.

flake.nix is the primary entry point (nix develop); shell.nix is a
flake-compat shim for non-flake nix-shell users. Verified end-to-end: suite
passes 69/69 against the pinned kindest/node:v1.35.0 using juliaup's julia.
JSON 1.x parses objects to JSON.Object rather than the concrete
Dict{String,Any} that OpenAPI's from_json and Kuber's own helpers
dispatch on. Force dicttype=Dict{String,Any} at Kuber's parse sites
(a no-op on 0.21, the fix on 1.x) and widen the compat bound.

Verified against a kind cluster (k8s v1.35.0): 64/64 tests pass on
both JSON 1.6.1 and 0.21.4.
@tanmaykm
tanmaykm requested a review from a team July 2, 2026 08:08
@tanmaykm
tanmaykm merged commit cb36a5a into master Jul 2, 2026
3 checks passed
tanmaykm pushed a commit that referenced this pull request Jul 3, 2026
…eam processor death (#67)

* fix: JSON.jl 1.x watch events + fail-fast on stream processor death

Two related fixes for silent k8s watch stalls:

1. `kuber_obj` / `_kuber_obj` accepted only `Dict{String,Any}`. Under
   JSON.jl 1.x, watch event objects parse to `JSON.Object{String,Any}`
   (an `AbstractDict`), so the first event of every watch threw a
   MethodError inside the stream processor. Widen to `AbstractDict` —
   `convert(::Type{<:APIModel}, j)` already handles any AbstractDict.
   Follow-up to the JSON 1.x support in #66, which missed this entry point.

2. `watch(streamprocessor, ctx, watched, ...)` ran the processor in a bare
   `@async` under `@sync`. If the processor threw (e.g. the MethodError
   above), its task died but `@sync` kept waiting on the still-running HTTP
   watch task — a deaf watch that surfaces no error until the apiserver
   drops the connection (potentially 30+ minutes), while events pile up
   unconsumed in the buffered stream. Close the stream in a `finally` on
   the processor task too — symmetric with the watcher task — so processor
   death aborts the watch promptly and the caller sees the real exception.
   This mirrors client-go's reflector, where any watchHandler error exits
   ListAndWatch and triggers a fresh relist+rewatch.

Diagnosed live: JobLoops' pod reflector went permanently stale (jobs stuck
in Submitted while pods ran) because every watch stream died silently on
its first event under JSON 1.x.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: regression tests for deaf-watch fixes

- Watch Events: convert each received event object with kuber_obj —
  exercises the AbstractDict path (JSON.Object under JSON.jl 1.x) that
  previously threw MethodError on the first event of every watch.
- New "Watch processor failure aborts watch" testset (no cluster needed):
  a throwing streamprocessor must propagate its error out of watch()
  promptly instead of leaving @sync waiting on the producer task forever.
  Verified to hang on the unfixed code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants