Remove two committed page dumps, and close the hole that let them through - #2
Merged
Merged
Conversation
… that let them through
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The v0.1.0 squash merge committed
live_results.page1andlive_results.page2— 1.5 MB each, raw--dump-htmloutput from a live run.Content, read before deciding: each is one motors listing page as the site served it — 26 per-seller UUIDs, 12 copies of dubizzle's public Algolia search key, 2 Sentry public keys, ~820 of its own ad identifiers. All of it data the site ships to every visitor. Nothing of ours: no 2Captcha key, no proxy credential, no session cookie, no Bearer token, no email. A motors page carries no
agent_profile, so no individual is named.Why the guards missed it
.gitignorehadlive_results/— the DIRECTORY.--dump-html live_resultswriteslive_results.page1, a FILE.ci_checks.pyscanned six suffixes (.py .md .txt .yml .yaml .example)..page1was not one of them, so the check ran, passed, and never opened them. Neither had it ever openedfixtures_generated.json,sample_output.jsonorsample_output.csv—.jsonand.csvwere not on the list either.Fixes
.gitignorenow coverslive_results.*and*.pageN.git ls-files --cached --others --exclude-standard— so a developer's own.envand captures stay invisible while a new file is caught before it is added.*.pageN,*_debug.html,live_results.*,captures/) whether or not its content looks dangerous — the two that got through would have passed any content rule.KEY_SHAPED_FIELDrule applies everywhere, generated data included. The bare-32-hex rule is deliberately not applied to the three generated data files, with the reason written down: this site emits 32-hex ids in five public contexts and the rule fired 221 times on correct data. Those files are covered by that rule plusmake_fixtures.py's scrub and the suite's corpus scan.--history-checknow reports any capture that has ever been committed, with its size, and fails unless a decision is recorded — because a later commit cannot reach the blob.Verified
Both new rules were checked by planting a violation and watching them fail, and the history rule by removing the recorded decision (rc=1) and putting it back (rc=0).
520 offline checks, pytest green,
ci_checks --alland--history-checkclean.