fix: cache importstr and importbin values - #1094
Open
He-Pin wants to merge 3 commits into
Open
Conversation
Motivation: Repeated raw imports called ResolvedFile readers and rebuilt Jsonnet values on every visit. Large importstr inputs could be read from disk repeatedly, while importbin eagerly allocated one Eval entry per byte. This also failed to keep a filename-stable raw-value snapshot for an evaluator. Modification: Cache importstr and importbin values independently by resolved path for the lifetime of an Evaluator. Clone importer-owned binary data before storing it in the compact ByteArr representation. Add snapshot, aliasing, cross-kind cache tests, and controlled JMH workloads. Result: Stable importstr output remains identical across sjsonnet, go-jsonnet, and jrsonnet; direct importbin output remains identical across sjsonnet and go-jsonnet. Repeated 256 KiB importbin x300 improves from 1062.890 to 0.199 ms/op, and repeated >1 MiB importstr x200 improves from 25.394 to 0.232 ms/op in the local JMH runs. Caches remain evaluator-local, so independent Interpreter instances do not share raw imported values. References: - databricks#1094 - https://jsonnet.org/ref/spec.html
He-Pin
force-pushed
the
perf/cache-raw-import-values
branch
from
July 29, 2026 08:27
f14ad61 to
c1cfb5c
Compare
He-Pin
changed the title
He-Pin
marked this pull request as draft
July 29, 2026 08:53
Motivation: The import cache shares ByteArr instances across all import sites. rawBytes exposes the internal backing array by reference, so any mutation would corrupt the cached value for all subsequent imports. Modification: Add CONTRACT comments to both ByteArr and LinearModByteArr rawBytes overrides documenting that callers must not mutate the returned array. Result: Future contributors are warned about the shared-state invariant.
Motivation: CI checkFormat failed on all platforms due to scaladoc comment style. Modification: Reformat multi-line scaladoc to match scalafmt 3.11.5 rules. Result: checkFormat passes on JVM/JS/Wasm/Native.
He-Pin
marked this pull request as ready for review
July 29, 2026 10:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
importstrandimportbinrebuilt raw Jsonnet values on every visit, even though the resolved file was already cached. For large files this meant repeated disk reads and allocations.Modification
importstr/importbinvalue caches keyed by resolvedPath.ByteArr.Result
importbinx300importstrx200importbinimportstrx300Output correctness verified against go-jsonnet v0.22.0 and jrsonnet.
References