Repository navigation
Conversation
f7878fe to
276925e
Compare
wgtmac
left a comment
There was a problem hiding this comment.
Generally looks good! Please rebase to resolve ci failure.
| git submodule update --init testing/iceberg-verification | ||
| ``` | ||
|
|
||
| Pass `-DICEBERG_VERIFICATION_DIR=/path/to/iceberg-verification` to run against |
There was a problem hiding this comment.
FYI I'm not sure if you've noticed https://github.com/apache/iceberg-verification/blob/main/reference-tables/README.md#paths.
Pin apache/iceberg-verification as a git submodule at testing/iceberg-verification and add verification_test, which runs the schema and types surfaces of its table-spec fixtures through SchemaFromJson and TypeFromJson. The test is built only with -DICEBERG_BUILD_VERIFICATION_TESTS=ON (off by default) and is not run in CI yet. With the option on, CMake fails at configure time if the fixtures are missing. Every case that diverges from its fixture fails; advisory (normative_level: should) cases skip. Four cases currently fail on implementation gaps: - schema unknown/reject-unknown-required: SchemaFromJson accepts a required field of type unknown. - types primitive/decimal-9-2 and decimal-9-2-spaced: ToJson writes decimal(P,S) without the space the fixtures expect. - types primitive/decimal-precision-over-38: TypeFromJson throws from the DecimalType constructor instead of returning an error.
276925e to
912ccb9
Compare
|
@manuzhang @zhjwpku Do you want to take a look? |
| std::vector<VerificationCase> LoadCases(std::string_view surface, | ||
| std::span<const std::string_view> subdirs) { | ||
| std::vector<VerificationCase> cases; | ||
| if (!FixturesAvailable()) { |
There was a problem hiding this comment.
nit: Can we replace return and continue in this class with if ... else ...? We only need to return cases in the end.
| load_error(e.what()); | ||
| continue; | ||
| } | ||
| if (!fixture.contains(kCases) || !fixture[kCases].is_array()) { |
There was a problem hiding this comment.
what about empty fixture array?
| load_error("no `cases` array"); | ||
| continue; | ||
| } | ||
| for (auto& json : fixture[kCases]) { |
There was a problem hiding this comment.
A null case entry would abort the test initialization. It would be better to explicitly check and report the error to identify the fixture file.
| DiffJson(c.at(kDecoded), ProjectType(type), "", &diffs); | ||
| if (c.contains(kCanonical)) { | ||
| auto written = ToJson(type); | ||
| if (!written.has_value()) { |
There was a problem hiding this comment.
nit: Not sure why the negative case is checked. It doesn't go well with my human mind.
| diffs->push_back(std::format("{}: unexpected {}", child, value.dump())); | ||
| } | ||
| } | ||
| return; |
There was a problem hiding this comment.
Bear with me but someone just forgets there's such thing as if ... else if ... else. 😅
Phase 1 of #909: pin apache/iceberg-verification as a git submodule at
testing/iceberg-verification, and addverification_test, which runs thetypesandschemasurfaces of its table-spec fixtures.Each fixture case becomes one GTest parameterized test named after
<subdir>/<case-id>. The types cases go throughTypeFromJsonand the schema cases throughSchemaFromJson. Each result is then projected into the fixture'sdecodedshape through public accessors and compared field by field. Every difference is reported with its path, for examplefields[2].parent. The subscribed subdirectories aretypes/{primitive,nested,variant,geospatial}andschema/{core,unknown}.The test is behind a new
ICEBERG_BUILD_VERIFICATION_TESTSoption, off by default. To run it:git submodule update --init testing/iceberg-verification cmake -S . -B build -DICEBERG_BUILD_VERIFICATION_TESTS=ON cmake --build build --target verification_test ./build/src/iceberg/test/verification_testICEBERG_VERIFICATION_DIRpoints the test at another checkout of the fixtures.Two things differ from the plan in #909:
normative_level: should) are reported as skips. The test is not added to CI until all cases pass.Currently 77 cases pass and 4 fail:
schema/unknown/reject-unknown-requiredtypes/primitive/decimal-9-2,types/primitive/decimal-9-2-spacedtypes/primitive/decimal-precision-over-38Validation: built and ran
verification_testlocally on macOS with the results above. Configuring with a nonexistentICEBERG_VERIFICATION_DIRfails as expected. pre-commit passes.