Repository navigation
refactor(datamodel): Drop sql_type/is_categorical from the CDE DTO - #40
Closed
KFilippopolitis wants to merge 14 commits into
Closed
KFilippopolitis wants to merge 14 commits into
KFilippopolitis wants to merge 14 commits into
Conversation
ci: run vulnerability scans on push to master
DataModelService now omits datasets whose variable list is null or [] and drops null entries inside a list instead of failing the request, so the UI only grays out variables for datasets whose variables are actually known.
ClaimUtils no longer adds an [] entry for an authorized dataset without a variable list; it applies the same rule as DataModelService.
…available-variables fix(datamodels): send only known dataset variable lists
… patches - spring-boot-starter-parent 4.1.1 -> 4.1.2 (Spring Framework 7.0.10, Security 7.1.2, Data 2026.0.2, Tomcat 11.0.27) - drop the tomcat.version 11.0.25 override: Boot 4.1.2 manages 11.0.27, which is the newest release and includes the CVE fixes it was added for - hibernate-core 7.4.8.Final -> 7.4.13.Final (matches the Boot 4.1.2 BOM) - postgresql 42.7.13 -> 42.7.14 - flyway-core / flyway-database-postgresql 13.6.0 -> 13.11.1 - nimbus-jose-jwt 10.9.1 -> 10.10 - jackson 2.22.3 -> 2.22.4, jackson 3.2.3 -> 3.2.4
GitHub Actions (SHA-pinned with a version comment): - github/codeql-action/upload-sarif v4.31.2 -> v4.39.0; the gitleaks workflow was still on the deprecated v2 line (v2.25.6) and moves to v4.39.0 - actions/checkout v6.0.3 -> v7.0.1 in secrets-scan (others already on v7.0.1) - actions/upload-artifact v7.0.1 -> v7.0.2 - docker/setup-buildx-action v4.3.0 -> v4.4.1 - docker/build-push-action v7.3.0 -> v7.4.0 - Python 3.14.7 -> 3.14.8 for the scan scripts Scanners in ci/setup-tools.sh (SHA256 values checked against the downloaded release assets): - trivy v0.74.0 -> v0.76.0 - osv-scanner v2.5.1 -> v2.7.0 - opengrep v1.30.0 -> v1.31.0 - semgrep-rules develop ref -> 17e31e11 (2026-10-15) Versions respect the repo's 7-day cooldown (dependabot.yml / renovate.json): codeql-action v4.39.1, setup-python v7.1.0 / Python 3.14.9, trivy v0.76.1, opengrep v1.32.0 and newer semgrep-rules commits are held back. hadolint v2.15.1, gitleaks v8.30.1 and cyclonedx-npm 6.0.1 are already latest.
Add a decision-log entry with versions, cooldown hold-backs, scan results and validation; drop the resolved Tomcat-override risk; bump the Boot version in AGENTS.md.
…-ci-latest chore(deps): upgrade dependencies and CI to latest
…hon-3.x chore(deps): update dependency python to v3.15.0
…ngrep-opengrep-1.x chore(deps): update dependency opengrep/opengrep to v1.31.0
The MIP variable-type refactor removes sql_type and isCategorical from CDEs; `type` (continuous, discrete, nominal, ordinal, binary) alone carries the classification. Exaflow no longer sends either field, so the CDE DTO stops declaring them and /data-models returns `type` only. Compatibility: breaking for API consumers that read sql_type or is_categorical from /data-models; the UI and the notebook client are updated on the same branch name.
This was referenced Oct 26, 2026
KFilippopolitis
commented
Oct 27, 2026
KFilippopolitis
left a comment
Contributor
Author
There was a problem hiding this comment.
Contract review (CDEs JSON passthrough)
This is the only backend change needed: DataModelDTO.CommonDataElementDTO drops sql_type and is_categorical. type, enumerations, minValue, and maxValue still serialize. minValue/maxValue are String here, so continuous floats from the catalog are not truncated (unlike datacatalog’s Integer DTO — flagged on datacatalog#22).
Grep of this worktree: no remaining production reads of sql_type / is_categorical / isCategorical. Algorithm tests that mention real / text are Exaflow parameter types, not CDE type. Keep String type; this layer does not switch on it.
Verdict: ready to merge with UI#57, mip-jupyter#15, and Exaflow#658. Current UI/client still read the dropped fields from /data-models.
KFilippopolitis
force-pushed
the
master
branch
2 times, most recently
from
November 5, 2026 12:48
aab8297 to
03d18bf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The MIP variable-type refactor removes
sql_typeandisCategoricalfrom CDEs;type(continuous,discrete,nominal,ordinal,binary) alone carries the classification. Exaflow no longer sends either field, soDataModelDTO.CommonDataElementDTOstops declaringsql_typeandis_categorical, andGET /services/data-modelsreturnstypeonly.Tests
mvn test: 89 passedCompatibility
Breaking for consumers that read
sql_typeoris_categoricalfrom/data-models; the UI and notebook client PRs below switch totype. Deploy together with the Exaflow PR. Rollback: revert this PR together with the related PRs.Related PRs
Same branch name
refactor/cde-variable-typesin every repo; merge and deploy together.