🌱 New check: protected tags

[AdamKorcz]

What kind of change does this PR introduce?

Fixes #10 and #2476

New feature

• PR title follows the guidelines defined in our pull request documentation

This PR adds a new check for tag protection. It is very similar to the Branch Protection check but for tags. At a high level, the check does these things:

1. First, it fetches release tags from the repository to identify which tags need protection. In the case of Gitlab, it will also get all branches.
2. Next, it checks the protection rules applied to release tags, analyzing their levels of tag protection. In the case of Gitlab, it will also check whether any tags are named the same as any branches. The check does that because Gitlab has much fewer options for protecting tags and because the documentation specifically mentions it (ref)

The check specifically focuses on release tags only and not necessarily all tags in the repository. It identifies release tags by calling ListReleases() on the repository client, which returns the tags associated with formal releases.

How these tiers are satisfied differs significantly between GitHub and GitLab:

Scoring

GitHub (Tiers: 0/3/6/8/10)
Tier 1 (3 points): All release tags are protected and tag creation is restricted. The repository meets the minimum requirements for basic tag protection.

Tier 2 (6 points): Includes Tier 1 protections plus force-push blocking and deletion blocking on all tags. Tags cannot be force-pushed or deleted once created.

Tier 3 (8 points): Includes Tier 2 protections plus tag update blocking and protections apply to admin users. All modification operations are blocked for all user roles including admins.

Tier 4 (10 points - Maximum): Includes all Tier 3 protections plus signed tags requirement. Tags must be cryptographically signed in addition to having all other protections enabled.

GitLab (Component-Based: 0-2-4-8-10)
Branch Shadowing Component (0/1/2 points): Scores based on protection preventing tags with branch names: 1 point when all branches require Maintainer+ access, 2 points when "No one" access level is set for all branches, meaning that no one can create tags with the same names as existing branch names.

Release Tag Protection Component (0/4/8 points): Scores based on who can create release tags: 4 points when all release tags require Maintainer+ access to create, 8 points when "No one" access level is set for all release tags. The two component scores are added together for the final score.

This aligns with Branch Protection scoring in that they both follow a tier type of scoring. However, branch protection rules and tag protection rules differ slightly on GitHub; for example, tags can be protected whereas branches can't, and branches can require conversation resolution whereas tags can't.

Gitlab and GitHub: Differences in tag protections

At a bit of a lower level, the check integrates with both GitHub and GitLab, but the implementation differs significantly due to platform. GitLab has a separate protected tags feature distinct from (its and GitHub's) protected branches. It has fewer options than GitHub tag protection rules. Essentially, Gitlab allows projects to specify which roles, users or user groups can create tags - that's it. Both the GitHub and GitLab handlers support pattern matching for tags: On both platforms you can specify with patterns which tag names protection rules apply to.

A big part of this PR was to refactor the existing Branch Protection handlers to reuse much of their logic to also handle tag protections. Branch Protection is still not as mature as other checks, and I have not worked on maturing it in this PR. I will be happy to look into that later though since I have spent some time in the code.

Testing

I have created some test repositories to test this one. See below how to run the check on them and to check their tag protection rules manually.

GitHub Test Repositories

1. scorecard-tag-test-gh-none

URL: https://github.com/AdamKorcz/scorecard-tag-test-gh-none
Configuration: No tag protection rules configured.
Expected Score: 0/10

Test with Scorecard:

go run main.go --repo=github.com/AdamKorcz/scorecard-tag-test-gh-none --checks=Tag-Protection

Verify rules manually:

curl -H "Authorization: token ${GITHUB_AUTH_TOKEN}" \
  https://api.github.com/repos/AdamKorcz/scorecard-tag-test-gh-none/rulesets

2. scorecard-tag-test-gh-basic

URL: https://github.com/AdamKorcz/scorecard-tag-test-gh-basic
Configuration: Tags are protected but no specific restrictions are enabled. No admin enforcement.
Expected Score: 3/10

Test with Scorecard:

go run main.go --repo=github.com/AdamKorcz/scorecard-tag-test-gh-basic --checks=Tag-Protection

Verify rules manually:

curl -H "Authorization: token ${GITHUB_AUTH_TOKEN}" \
  https://api.github.com/repos/AdamKorcz/scorecard-tag-test-gh-basic/rulesets

3. scorecard-tag-test-gh-delete-only

URL: https://github.com/AdamKorcz/scorecard-tag-test-gh-delete-only
Configuration: Has Restrict Delete protection only. Force push and updates are allowed. No admin enforcement.
Expected Score: 8/10

Test with Scorecard:

go run main.go --repo=github.com/AdamKorcz/scorecard-tag-test-gh-delete-only --checks=Tag-Protection

Verify rules manually:

curl -H "Authorization: token ${GITHUB_AUTH_TOKEN}" \
  https://api.github.com/repos/AdamKorcz/scorecard-tag-test-gh-delete-only/rulesets

4. scorecard-tag-test-gh-no-updates

URL: https://github.com/AdamKorcz/scorecard-tag-test-gh-no-updates
Configuration: Has Restrict Delete and Restrict Force Push protections. Updates are automatically restricted by the non_fast_forward rule. No admin enforcement.
Expected Score: 8/10

Test with Scorecard:

go run main.go --repo=github.com/AdamKorcz/scorecard-tag-test-gh-no-updates --checks=Tag-Protection

Verify rules manually:

curl -H "Authorization: token ${GITHUB_AUTH_TOKEN}" \
  https://api.github.com/repos/AdamKorcz/scorecard-tag-test-gh-no-updates/rulesets

5. scorecard-tag-test-gh-admin-bypass

URL: https://github.com/AdamKorcz/scorecard-tag-test-gh-admin-bypass
Configuration: Has Restrict Delete and Restrict Force Push protections. Updates are automatically restricted. Admin bypass is enabled OR tag creation is not restricted.
Expected Score: 8/10

Test with Scorecard:

go run main.go --repo=github.com/AdamKorcz/scorecard-tag-test-gh-admin-bypass --checks=Tag-Protection

Verify rules manually:

curl -H "Authorization: token ${GITHUB_AUTH_TOKEN}" \
  https://api.github.com/repos/AdamKorcz/scorecard-tag-test-gh-admin-bypass/rulesets

6. scorecard-tag-test-gh-full

URL: https://github.com/AdamKorcz/scorecard-tag-test-gh-full
Configuration: Has Restrict Delete and Restrict Force Push protections. Updates are automatically restricted. Tag creation is restricted. There are no bypasses (admin enforcement enabled).
Expected Score: 10/10

Test with Scorecard:

go run main.go --repo=github.com/AdamKorcz/scorecard-tag-test-gh-full --checks=Tag-Protection

Verify rules manually:

curl -H "Authorization: token ${GITHUB_AUTH_TOKEN}" \
  https://api.github.com/repos/AdamKorcz/scorecard-tag-test-gh-full/rulesets

GitLab Test Repositories

1. scorecard-tag-test-gl-exact-match

URL: https://gitlab.com/adamkorcz/scorecard-tag-test-gl-exact-match
Expected Score: 4/10

Test with Scorecard:

go run main.go --repo=gitlab.com/adamkorcz/scorecard-tag-test-gl-exact-match --checks=Tag-Protection

Verify rules manually:

curl -H "PRIVATE-TOKEN: ${GITLAB_AUTH_TOKEN}" \
https://gitlab.com/api/v4/projects/adamkorcz%2Fscorecard-tag-test-gl-exact-match/protected_tags

2. scorecard-tag-test-gl-partial-match

URL: https://gitlab.com/adamkorcz/scorecard-tag-test-gl-partial-match
Expected Score: 0/10

Test with Scorecard:

go run main.go --repo=gitlab.com/adamkorcz/scorecard-tag-test-gl-partial-match --checks=Tag-Protection

Verify rules manually:

curl -H "PRIVATE-TOKEN: ${GITLAB_AUTH_TOKEN}" \
https://gitlab.com/api/v4/projects/adamkorcz%2Fscorecard-tag-test-gl-partial-match/protected_tags

3. scorecard-tag-test-gl-multi-pattern-merge

URL: https://gitlab.com/adamkorcz/scorecard-tag-test-gl-multi-pattern-merge
Expected Score: 0/10

Test with Scorecard:

go run main.go --repo=gitlab.com/adamkorcz/scorecard-tag-test-gl-multi-pattern-merge --checks=Tag-Protection

Verify rules manually:

curl -H "PRIVATE-TOKEN: ${GITLAB_AUTH_TOKEN}" \
https://gitlab.com/api/v4/projects/adamkorcz%2Fscorecard-tag-test-gl-multi-pattern-merge/protected_tags

4. scorecard-tag-test-gl-full

URL: https://gitlab.com/adamkorcz/scorecard-tag-test-gl-full
Expected Score: 4/10

Test with Scorecard:

go run main.go --repo=gitlab.com/adamkorcz/scorecard-tag-test-gl-full --checks=Tag-Protection

Verify rules manually:

curl -H "PRIVATE-TOKEN: ${GITLAB_AUTH_TOKEN}" \
  https://gitlab.com/api/v4/projects/adamkorcz%2Fscorecard-tag-test-gl-full/protected_tags

Does this PR introduce a user-facing change?

Yes.

Add new check for tag protection.

[codecov]

Codecov Report

❌ Patch coverage is 72.68003% with 368 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.92%. Comparing base (353ed60) to head (1aa6a10).
⚠️ Report is 298 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4894      +/-   ##
==========================================
+ Coverage   66.80%   69.92%   +3.12%     
==========================================
  Files         230      267      +37     
  Lines       16602    16978     +376     
==========================================
+ Hits        11091    11872     +781     
+ Misses       4808     4176     -632     
- Partials      703      930     +227     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.