fix: Baseline subtraction for --git mode diffs#27
Merged
Conversation
When using --git in CI with staged/delayed deployments, fleet-plan reported changes already merged to main but not yet deployed. This trained users to ignore fleet-plan comments. Now in --git mode, fleet-plan extracts the base branch YAML via git show, diffs it against live Fleet, and subtracts those pre-existing deltas. Only the incremental changes from the current MR are reported. Ref #23
The baseline diff was using raw API software state without fleet-maintained app script enrichment, causing fleet-maintained app diffs (e.g. blender uninstall_script) to not be subtracted properly.
When an MR adds a path: reference to teams/foo.yml, the referenced resource's SourceFile (e.g. queries/windows/hardware.yml) isn't in the MR's changed files list. The filter dropped these resources because it only checked the individual SourceFile, not the team file that includes them. Now buildSourceMap registers the team's SourceFile for all resource types (policies, queries, software, profiles, scripts), matching the existing behavior for fleet-maintained apps.
Global config changes (default.yml/base.yml) were not subtracted, causing MRs touching global config to show all pending undeployed changes. Now applies subtractConfigChanges and subtractResourceDiff to the global diff block. Also merges base branch base.yml with env overlay for baseline global parsing. Ref #25
collectBaselineFiles only scanned team files for path: references. Global config files (base.yml, default.yml) also have path: refs to queries and policies that need to be in the baseline temp dir for the parser to resolve them.
The merged base.yml+env overlay was written to /tmp, causing path: references (./queries/...) to resolve against /tmp instead of the baseline temp dir where the query files live.
ParseRepo bails early if teams/ doesn't exist, preventing global config parsing. For global-only baselines (MR touches base.yml but no team files), the teams/ dir must exist as an empty directory.
…l config Two changes: 1. --verbose (-v) now logs baseline subtraction details to stderr: proposed/fleet counts, MR diff, baseline diff, after subtraction, and changedFiles filter results for each team and global scope. 2. collectBaselineFiles no longer unconditionally extracts base.yml and default.yml. Only files in the MR's changedFiles (plus their path: references) are extracted. This fixes over-subtraction for MRs that don't touch global config. Ref #25
Two pre-existing bugs found via --verbose: 1. resolveDefaultFile placed the merged base.yml+env overlay in /tmp. The parser resolves path: refs relative to the file's directory, so refs like ./queries/... resolved to /tmp/queries/... which doesn't exist. Now places the merged file inside the repo dir. 2. ResolveScope set IncludeGlobal for base.yml changes but didn't add base.yml to scope.ChangedFiles. The baseline extraction and changedFiles filter couldn't see it. Ref #25
When --git mode detects both global (base.yml) and team changes, the auto-detected team filter suppressed the global diff block. Now WithIncludeGlobal overrides the teamFilters check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In
--gitmode, fleet-plan now subtracts changes already merged to main but not yet deployed. Only the incremental changes from the current MR are reported. Applies to both per-team and global diffs.Changes
internal/git/baseline.goCheckoutBaseline()extracts base-branch files viagit show,collectBaselineFiles()resolves path refsinternal/git/baseline_test.gointernal/git/scope.gobase.yml/default.ymlinchangedFilesinternal/diff/differ.goWithBaseline(),WithVerbose(),WithIncludeGlobal()options,subtractResourceDiff(),subtractConfigChanges(), verbose logginginternal/diff/differ_test.goTestSubtractResourceDiff,TestDiffWithBaselineSubtractioncmd/fleet-plan/main.goincludeGlobalpassthroughBugs fixed
resolveDefaultFileplaced merged file in/tmp, breakingpath:resolution for global queriesResolveScopedidn't addbase.yml/default.ymltochangedFilesbuildSourceMapdidn't register team source file for all resource typesTest plan
TestSubtractResourceDiff(5 subtests),TestDiffWithBaselineSubtractionTestCheckoutBaseline_ExtractsFiles,TestCollectBaselineFiles_OnlyIncludesChangedFilesRef #23