Performance regression between v0.20.0 and master

[igorwwwwwwwwwwwwwwwwwwww]

I ran into this while debugging an other mystery. Master at the time of writing is 923f51b. I tested this on Apple M1 hardware.

It appears that one of the recent changes has had a considerable performance impact, it is reproducible with the cpp benchmarks.

$ for i in v0.20.0 master; do git checkout $i; go build -o jsonnet-$i ./cmd/jsonnet; done

$ hyperfine --warmup 3 --parameter-list binary './jsonnet-v0.20.0,./jsonnet-master' '{binary} cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000'

Benchmark #1: ./jsonnet-v0.20.0 cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000
  Time (mean ± σ):     201.7 ms ±   3.8 ms    [User: 169.0 ms, System: 46.8 ms]
  Range (min … max):   196.3 ms … 208.5 ms    13 runs

Benchmark #2: ./jsonnet-master cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000
  Time (mean ± σ):     290.9 ms ±   2.3 ms    [User: 239.1 ms, System: 71.4 ms]
  Range (min … max):   287.6 ms … 294.1 ms    10 runs

Summary
  './jsonnet-v0.20.0 cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000' ran
    1.44 ± 0.03 times faster than './jsonnet-master cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000'

Profiling:

$ for i in v0.20.0 master; do JSONNET_CPU_PROFILE=cpu-$i.pprof ./jsonnet-$i cpp-jsonnet/benchmarks/bench.07.jsonnet --max-stack 1000000; go tool pprof -svg -output cpu-$i.pprof.svg cpu-$i.pprof; done

These are short-lived runs, so I'm not sure how accurate the profiles are.

• cpu-v0.20.0.pprof.svg
• cpu-master.pprof.svg

I also noticed during a larger build that jsonnet-master was using 3.5 cores, whereas jsonnet-v0.20.0 only used a single core.