Repository navigation
Conversation
…porary-rules-to-decision-lint' into akhilpindiprolu/eng-8873-add-temporary-rules-to-decision-lint
afeba8f to
2532c6c
Compare
narsaynorath
left a comment
There was a problem hiding this comment.
Just some small comments, r.e. diffing and one of the variables, otherwise looks good to me
| PR: ${{ github.event.number }} | ||
| LABEL: 'Trigger: Override Decision Model Lint' | ||
| # Rules diff each file against the PR's base, it must match --base below | ||
| DECISION_MODEL_LINT_BASE: HEAD^1 |
There was a problem hiding this comment.
If I understand correctly, HEAD^1 should just get the previous commit, right? Would we prefer to diff between the current HEAD and the target branch HEAD? There's a merge-base command that will return the common ancestor comment, i.e. git merge-base master branchA will get you the commit in master that was last common with branchA, which I imagine would be a better point to do the diff if I'm understanding the intent correctly
There was a problem hiding this comment.
Good thought, but here HEAD^1 already gives the merge-base result. On pull_request runs, actions/checkout doesn't check out the PR's last commit. It checks out a merge commit GitHub makes by merging the PR into master. That commit's first parent (so HEAD^1) is master's tip, so HEAD^1..HEAD is exactly what the PR changes, even if master has moved on. It also has to match the --base HEAD^1 the linter already gets, and with fetch-depth: 2 there isn't enough history for git merge-base to work anyway. For local runs I'd set it to git merge-base origin/master HEAD
| // renamed files have the same file path before the rename | ||
| let renames: Promise<Map<string, string>> | undefined; |
There was a problem hiding this comment.
It's a little weird that this variable is outside of the renamedFrom function and not used globally, can we push it into the renamedFrom function body?
There was a problem hiding this comment.
Yeah I can see why you think that, but it has to stay outside because it's a cache cuz diff() runs only once per changed file, and renamedFrom returns the saved result after the first call, so git diff --name-status runs once per lint run instead of once per file. If it were inside the function, it would reset to undefined on every call and git would run for every file. I think I'll add a comment above the line to make that clear
Linear issue
This PR adds 5 decision-model lint rules as a temporary safety net while we build deterministic API checks in CI. All five are warn only, so they show up in the lint comment but never fail the check. Each check though will be removed once its deterministic check is enforced
Rules:
api suite
security suite
The api suite rules use threshold: 0.8, like the existing rules that had false positives. returnsNumericIdAsInteger keeps the default
The linter shows the model only the lines a PR changed, never what they were before. So it couldn't tell new code from code that was already there. Touching a line next to an old integer ID got a comment, and the fix it suggested would break clients obviosuly. The stricter input rule also couldn't see that
required=Falsebecamerequired=True.I made it so that the api suite now also gets each file's git diff against the PR's base, with removed and added lines only, so unchanged code stays hidden as before. The workflow sets
DECISION_MODEL_LINT_BASE: HEAD^1for this so that local runs get no diff unless you set it (ideally togit merge-base origin/master HEAD). The three design rules now answerINTRODUCED,PREEXISTING, orNONE, and onlyINTRODUCEDis reported, so existing problems never get a comment. The stricter input rule compares the removed and added lines, which also lets it catch default changes again.I tested this in #129449 on six cases, with the old and new config. The old config made three mistakes: it flagged an existing integer ID on a reformatted line, flagged an integer ID in a file with none, and missed a field made required. The new config got all six right, for more input tokens but the cost is insignificant.
For whole-file runs: running the CLI on whole files gives no diff, so the three
INTRODUCED/PREEXISTINGrules now treat the whole file as under review and report any problem they find. I added this since before, they split their answer betweenINTRODUCEDandPREEXISTINGand reported nothing.For renamed files: a renamed file is diffed against its old path, so its real changes show up instead of the whole file looking new. This fixes a missed case where a field was made required in a renamed file