Enhance testing capabilities#2509
Conversation
| }; | ||
|
|
||
| const optionMap: Record<string, TestOptions> = { | ||
| "test/inputs/json/misc/227e2.in.json": { |
There was a problem hiding this comment.
This means, for 227e2.in.json, the test will be run with omit-empty set to true, strict comparison of input and expected output, and will only run for Golang language.
|
Thanks @Adam724 — this PR pointed at a real gap, and both of its ideas ended up in the codebase in some form. The per-test CLI options half has since been superseded: The other half — expected-output files so a test can assert output that legitimately differs from the input (your Go |
test: expected-output fixtures for JSON inputs (salvaged from #2509)
The JSON fixture harness asserts round-trip output ~ input, with unconditional leniency: extra null properties in the output are always allowed, and allowMissingNull languages may drop nulls. Correct behaviors where the output legitimately differs from the input were therefore untestable -- concretely, that Go's omitempty drops null fields. Port the JSON Schema fixtures' `.out.<feature>.json` convention to JSON inputs: an input `foo.json` may come with an expected-output file `foo.out.<key>.json`, which applies to a run when `<key>` is one of the language's `features` or the name of a renderer option that the particular run sets (via a pinned `quickTestRendererOptions` entry). When it applies, the output must match strictly, without the null leniency; when absent, behavior is unchanged. `.out.` files are excluded from test-input enumeration. Use it to assert the omit-empty behavior from glideapps#2509: a pinned Go run of priority/omit-empty.json with omit-empty enabled must drop the null field, per omit-empty.out.omit-empty.json. The input is in priority/ (not misc/) so it also runs under QUICKTEST in CI. Salvaged from glideapps#2509; the test case is Adam724's. Co-authored-by: Adam724 <alaspina6@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3 participants
Currently, there is no way to pass cliOptions in the automated tests, making it difficult to test these flags. This PR aims to solve that with the following:
test/lib/optionMap.ts. This allows you to specify a test file path as key, and a configuration object as the value to customize the cliOptions, comparisonArgs, and target language for that test./^(.*)\.in\.(.*)$/, and output files matching this regex:/^\1\.out\.\2$/. For examplemyCustomTest.in.jsonandmyCustomTest.out.json. If an input file is found with no matching output, an error is thrown with a message indicating the issue.No changes are necessary for any existing tests, this just adds the ability to write more customizable additional tests. Unsure if there is a better way to do this, any feedback is welcome!