Skip to content

Commit 33b2b37

Browse files
committed
fix: close final catalog audit gaps
1 parent 31d23c9 commit 33b2b37

13 files changed

Lines changed: 130 additions & 49 deletions

.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,11 @@ List the actual results, not only the commands you intended to run.
2828
- [ ] `python -m pytest`
2929
- [ ] `python tools/validate_catalog.py`
3030
- [ ] `bundle exec jekyll build`
31+
- [ ] `bundle exec ruby tests/render_security_test.rb`
3132
- [ ] English and Chinese output remain consistent
3233
- [ ] No secrets, generated reports, local build output, or analytics IDs were committed
3334

34-
External links are fetched only by the scheduled or manually dispatched
35+
Catalog links are fetched only by the scheduled or manually dispatched
3536
workflow, not by pull-request CI.
3637

3738
## Permissions and review

.github/workflows/link-audit.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: External link audit
1+
name: Catalog link audit
22

33
on:
44
schedule:
@@ -9,7 +9,7 @@ permissions:
99
contents: read
1010

1111
concurrency:
12-
group: external-link-audit
12+
group: catalog-link-audit
1313
cancel-in-progress: false
1414

1515
jobs:
@@ -32,14 +32,14 @@ jobs:
3232
- name: Prepare report directory
3333
run: mkdir -p reports
3434

35-
- name: Check external links
36-
run: python tools/check_links.py --mode external --output reports/link-check.json
35+
- name: Check every catalog link
36+
run: python tools/check_links.py --mode all --output reports/link-check.json
3737

3838
- name: Upload link report
3939
if: ${{ always() }}
4040
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
4141
with:
42-
name: external-link-report-${{ github.run_id }}
42+
name: catalog-link-report-${{ github.run_id }}
4343
path: reports/link-check.json
4444
if-no-files-found: error
4545
retention-days: 14

.github/workflows/validate.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,6 @@ jobs:
4343

4444
- name: Build Jekyll site
4545
run: bundle exec jekyll build
46+
47+
- name: Verify catalog output escaping
48+
run: bundle exec ruby tests/render_security_test.rb

CHANGELOG.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ history remains the source for changes made before this changelog was added.
1616
- Contribution, conduct, security, and resource-curation policies.
1717
- Structured issue forms and a pull request review checklist.
1818
- Dependency update configuration and validation workflows.
19-
- Scheduled, manually dispatchable external-link auditing that does not run on
19+
- Scheduled, manually dispatchable catalog-link auditing that does not run on
2020
untrusted pull requests.
2121

2222
### Changed
@@ -30,6 +30,8 @@ history remains the source for changes made before this changelog was added.
3030

3131
- Corrected repository links, dynamic resource counts, bilingual metadata,
3232
canonical and hreflang output, robots rules, and duplicate heading/meta tags.
33+
- Escaped catalog-controlled HTML output, audited same-origin catalog links,
34+
and made review-needed network results fail the scheduled audit.
3335

3436
### Removed
3537

CONTRIBUTING.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,17 +67,21 @@ Run the same checks as CI before requesting review:
6767
python -m pytest
6868
python tools/validate_catalog.py
6969
bundle exec jekyll build
70+
bundle exec ruby tests/render_security_test.rb
7071
```
7172

7273
Maintainers can run the networked link audit from GitHub Actions with the
73-
scheduled **External link audit** workflow. For a deliberate local audit:
74+
scheduled **Catalog link audit** workflow. For a deliberate local audit of
75+
every catalog entry, including same-origin URLs:
7476

7577
```bash
76-
python tools/check_links.py --mode external --output reports/link-check.json
78+
python tools/check_links.py --mode all --output reports/link-check.json
7779
```
7880

79-
External-link fetching is intentionally excluded from pull-request CI. A pull
81+
Network link fetching is intentionally excluded from pull-request CI. A pull
8082
request can contain untrusted URLs, and status checks must remain deterministic.
83+
Any review-needed link result fails the scheduled audit so a maintainer must
84+
inspect it; it does not by itself justify removing the resource.
8185

8286
## Pull request checklist
8387

_includes/catalog-grid.html

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,15 @@
22
<nav class="path-grid" aria-label="{% if page.lang == 'zh-CN' %}学习路径{% else %}Learning paths{% endif %}">
33
{% for path in ordered_paths %}
44
{% assign path_resources = include.resources | where: "path", path.id %}
5-
<a class="path-card" href="#{{ path.id }}">
6-
<span class="path-number">0{{ path.order }}</span>
5+
<a class="path-card" href="#{{ path.id | escape }}">
6+
<span class="path-number">0{{ path.order | escape }}</span>
77
{% if page.lang == "zh-CN" %}
8-
<strong>{{ path.title_zh }}</strong>
9-
<span>{{ path.summary_zh }}</span>
8+
<strong>{{ path.title_zh | escape }}</strong>
9+
<span>{{ path.summary_zh | escape }}</span>
1010
<small>{{ path_resources.size }} 条资源</small>
1111
{% else %}
12-
<strong>{{ path.title_en }}</strong>
13-
<span>{{ path.summary_en }}</span>
12+
<strong>{{ path.title_en | escape }}</strong>
13+
<span>{{ path.summary_en | escape }}</span>
1414
<small>{{ path_resources.size }} resources</small>
1515
{% endif %}
1616
</a>
@@ -19,13 +19,13 @@
1919

2020
{% for path in ordered_paths %}
2121
{% assign path_resources = include.resources | where: "path", path.id %}
22-
<section class="path-section" id="{{ path.id }}" aria-labelledby="{{ path.id }}-title">
22+
<section class="path-section" id="{{ path.id | escape }}" aria-labelledby="{{ path.id | escape }}-title">
2323
<header class="section-heading">
2424
<div>
25-
<span class="section-eyebrow">{% if page.lang == "zh-CN" %}路径{% else %}Path{% endif %} 0{{ path.order }}</span>
26-
<h2 id="{{ path.id }}-title">{% if page.lang == "zh-CN" %}{{ path.title_zh }}{% else %}{{ path.title_en }}{% endif %}</h2>
25+
<span class="section-eyebrow">{% if page.lang == "zh-CN" %}路径{% else %}Path{% endif %} 0{{ path.order | escape }}</span>
26+
<h2 id="{{ path.id | escape }}-title">{% if page.lang == "zh-CN" %}{{ path.title_zh | escape }}{% else %}{{ path.title_en | escape }}{% endif %}</h2>
2727
</div>
28-
<p>{% if page.lang == "zh-CN" %}{{ path.summary_zh }}{% else %}{{ path.summary_en }}{% endif %}</p>
28+
<p>{% if page.lang == "zh-CN" %}{{ path.summary_zh | escape }}{% else %}{{ path.summary_en | escape }}{% endif %}</p>
2929
</header>
3030

3131
<div class="resource-grid">
@@ -39,12 +39,12 @@ <h2 id="{{ path.id }}-title">{% if page.lang == "zh-CN" %}{{ path.title_zh }}{%
3939
{% when "official-project" %}<span class="source-badge">官方项目</span>
4040
{% endcase %}
4141
{% else %}
42-
<span class="source-badge">{{ resource.source_type | replace: "-", " " }}</span>
42+
<span class="source-badge">{{ resource.source_type | replace: "-", " " | escape }}</span>
4343
{% endif %}
4444
{% if resource.featured %}<span class="featured-badge">{% if page.lang == "zh-CN" %}精选{% else %}Featured{% endif %}</span>{% endif %}
4545
</div>
46-
<h3><a href="{{ resource.url }}">{{ resource.title }}</a></h3>
47-
<p>{% if page.lang == "zh-CN" %}{{ resource.why_zh }}{% else %}{{ resource.why_en }}{% endif %}</p>
46+
<h3><a href="{{ resource.url | escape }}">{{ resource.title | escape }}</a></h3>
47+
<p>{% if page.lang == "zh-CN" %}{{ resource.why_zh | escape }}{% else %}{{ resource.why_en | escape }}{% endif %}</p>
4848
<ul class="resource-meta" aria-label="{% if page.lang == 'zh-CN' %}资源元数据{% else %}Resource metadata{% endif %}">
4949
{% if page.lang == "zh-CN" %}
5050
{% case resource.level %}
@@ -58,15 +58,15 @@ <h3><a href="{{ resource.url }}">{{ resource.title }}</a></h3>
5858
{% when "zh" %}<li>来源语言:中文</li>
5959
{% when "multilingual" %}<li>来源语言:多语言</li>
6060
{% endcase %}
61-
<li>审核于 {{ resource.reviewed_on }}</li>
61+
<li>审核于 {{ resource.reviewed_on | escape }}</li>
6262
{% else %}
63-
<li>{{ resource.level | replace: "-", " " }}</li>
63+
<li>{{ resource.level | replace: "-", " " | escape }}</li>
6464
{% case resource.language %}
6565
{% when "en" %}<li>English</li>
6666
{% when "zh" %}<li>Chinese</li>
6767
{% when "multilingual" %}<li>Multilingual</li>
6868
{% endcase %}
69-
<li>Reviewed {{ resource.reviewed_on }}</li>
69+
<li>Reviewed {{ resource.reviewed_on | escape }}</li>
7070
{% endif %}
7171
</ul>
7272
{% if resource.requires_key or resource.risk == "medium" %}

_layouts/default.html

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!DOCTYPE html>
2-
<html lang="{{ page.lang | default: site.lang | default: 'en' }}">
2+
<html lang="{{ page.lang | default: site.lang | default: 'en' | escape }}">
33
<head>
44
<meta charset="UTF-8">
55
<meta name="viewport" content="width=device-width, initial-scale=1.0">
@@ -34,14 +34,14 @@
3434
<p>社区公开维护的 Python 一手资源目录,也是 FlyPython 学习路径的资源基础。</p>
3535
<ul aria-label="目录状态">
3636
<li>{{ active_resources.size }} 条有效资源</li>
37-
<li>审核于 {{ site.data.resources.catalog.reviewed_on }}</li>
37+
<li>审核于 {{ site.data.resources.catalog.reviewed_on | escape }}</li>
3838
<li><a href="https://github.com/flypythoncom/python">GitHub 仓库 ↗</a></li>
3939
</ul>
4040
{% else %}
4141
<p>A public, community-maintained catalog of primary Python sources behind the FlyPython learning paths.</p>
4242
<ul aria-label="Catalog status">
4343
<li>{{ active_resources.size }} active resources</li>
44-
<li>Reviewed {{ site.data.resources.catalog.reviewed_on }}</li>
44+
<li>Reviewed {{ site.data.resources.catalog.reviewed_on | escape }}</li>
4545
<li><a href="https://github.com/flypythoncom/python">GitHub repository ↗</a></li>
4646
</ul>
4747
{% endif %}

docs/CURATION_POLICY.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -100,20 +100,21 @@ as every other proposal.
100100

101101
## Link verification
102102

103-
Internal validation runs on every pull request. External-link fetching runs only
103+
Internal validation runs on every pull request. Network link fetching runs only
104104
on the scheduled or manually dispatched GitHub Actions workflow.
105105

106106
A 404 or 410 from the canonical resource is strong removal evidence. A 403, 429,
107107
timeout, or transient 5xx is a review-needed result, not proof that the resource
108-
is gone. DNS, connection, TLS, invalid-URL, and redirect-protocol failures make
109-
the automated audit fail, but they are still not enough on their own to remove a
110-
resource. Retry with rate limits and record the observation date. Link checks
111-
must refuse private, loopback, link-local, multicast, cloud-platform, and cloud
112-
metadata targets, including redirect destinations.
108+
is gone. Review-needed results fail the automated audit so they cannot remain
109+
hidden behind a green run. DNS, connection, TLS, invalid-URL, and
110+
redirect-protocol failures also fail the audit, but none of these results alone
111+
is enough to remove a resource. Retry with rate limits and record the observation
112+
date. Link checks must refuse private, loopback, link-local, multicast,
113+
cloud-platform, and cloud metadata targets, including redirect destinations.
113114

114115
## Review cadence and removal
115116

116-
Run the external-link audit weekly. Perform an editorial review when a resource
117+
Run the catalog-link audit weekly. Perform an editorial review when a resource
117118
changes ownership, becomes unmaintained, introduces a material safety concern,
118119
or receives a substantiated report.
119120

index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ source, then return to the main learning hub for guided projects and context.
2727
<p class="catalog-note">
2828
<strong>{{ catalog_resources.size }} active resources</strong>
2929
<span aria-hidden="true">·</span>
30-
Catalog reviewed {{ site.data.resources.catalog.reviewed_on }}
30+
Catalog reviewed {{ site.data.resources.catalog.reviewed_on | escape }}
3131
<span aria-hidden="true">·</span>
3232
Primary sources first
3333
</p>

tests/render_security_test.rb

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# frozen_string_literal: true
2+
3+
require "jekyll"
4+
require "tmpdir"
5+
6+
source = File.expand_path("..", __dir__)
7+
8+
Dir.mktmpdir("flypython-render-security") do |destination|
9+
config = Jekyll.configuration(
10+
"source" => source,
11+
"destination" => destination,
12+
"quiet" => true,
13+
"disable_disk_cache" => true
14+
)
15+
site = Jekyll::Site.new(config)
16+
site.reset
17+
site.read
18+
19+
path = site.data.fetch("resources").fetch("catalog").fetch("paths").first
20+
path["title_en"] = "<img src=x onerror=alert(1)>"
21+
path["title_zh"] = "<img src=x onerror=alert(1)>"
22+
path["summary_en"] = "<svg onload=alert(2)>"
23+
path["summary_zh"] = "<svg onload=alert(2)>"
24+
25+
resource = site.data.fetch("resources").fetch("resources").first
26+
resource["title"] = "</a><script>alert(3)</script>"
27+
resource["why_en"] = "<math href=x onmouseover=alert(4)>"
28+
resource["why_zh"] = "<math href=x onmouseover=alert(4)>"
29+
resource["url"] = 'https://example.com/" onmouseover="alert(5)'
30+
31+
site.generate
32+
site.render
33+
site.cleanup
34+
site.write
35+
36+
html = ["index.html", File.join("zh", "index.html")].map do |relative_path|
37+
File.read(File.join(destination, relative_path), encoding: "UTF-8")
38+
end.join("\n")
39+
40+
forbidden = [
41+
"<img src=x onerror=alert(1)>",
42+
"<svg onload=alert(2)>",
43+
"</a><script>alert(3)</script>",
44+
"<math href=x onmouseover=alert(4)>",
45+
'href="https://example.com/" onmouseover="alert(5)"'
46+
]
47+
leaked = forbidden.select { |payload| html.include?(payload) }
48+
abort "unescaped catalog payloads: #{leaked.join(', ')}" unless leaked.empty?
49+
50+
expected = [
51+
"&lt;img src=x onerror=alert(1)&gt;",
52+
"&lt;svg onload=alert(2)&gt;",
53+
"&lt;/a&gt;&lt;script&gt;alert(3)&lt;/script&gt;",
54+
"&lt;math href=x onmouseover=alert(4)&gt;",
55+
"https://example.com/&quot; onmouseover=&quot;alert(5)"
56+
]
57+
missing = expected.reject { |payload| html.include?(payload) }
58+
abort "escaped catalog payloads missing: #{missing.join(', ')}" unless missing.empty?
59+
end
60+
61+
puts "catalog output escaping valid"

0 commit comments

Comments
 (0)