fix(ci): make ailf-eval compatible with github-script v9 by stipsan · Pull Request #1915 · sanity-io/plugins

stipsan · 2026-08-18T10:05:20Z

Fixes the AI Literacy Eval workflow after #1897 bumped actions/github-script from v8 to v9.

What broke

github-script v9 injects extra function parameters (getOctokit, …) into inline script: bodies, and @actions/github is ESM-only. The AILF “Post PR comment” step also interpolated ${{ github.server_url }} / repository / run_id into JavaScript, which v9 treats as script injection.

Fix

Keep actions/github-script@v9 (no pin-back to v8). Extract the comment logic into a CommonJS helper and pass GitHub expressions through env::

.github/scripts/post-ailf-comment.cjs — require('node:fs'), paginate comments, create/update with history
.github/workflows/ailf-eval.yml — thin require() + await postAilfPrComment({github, context})
oxlint.config.ts — allow CJS + console in .github/scripts/**/*.cjs
AGENTS.md — document v9 rules so the next major bump is a script migration, not a version-only PR

No changeset: this is CI/docs only.

Verification

On this PR (230a6494):

AI Literacy Eval run 32125130267 — success
- Post PR comment (actions/github-script@v9) — success
- Log: Updated comment for presets (1 history entries)
CI run 32125130386 — success
E2E run 32125130278 — success
Local: node --check + v9 AsyncFunction compile + mocked create/update/fallback paths

Move the AILF PR comment helper into a CommonJS module so the v9-injected getOctokit parameter cannot clash with inline script declarations, and pass the workflow URL through env instead of interpolating expressions into JS.

changeset-bot · 2026-08-18T10:05:25Z

⚠️ No Changeset found

Latest commit: 230a649

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

vercel · 2026-08-18T10:05:26Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
plugins-studio	Ready	Preview	Aug 18, 2026 10:07am

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
plugins-e2e-test-studio	Ignored		Aug 18, 2026 10:07am

github-actions · 2026-08-18T10:06:22Z

✅ E2E Tests

🟢 26 passed • view full report • view run

Studio: https://plugins-e2e-test-studio-lyd73b0s3.sanity.dev

Datasets: pr-1915-chromium-32125130278, pr-1915-firefox-32125130278

github-script loads `.github/scripts/*.cjs` via require(), so those files cannot be ESM. Turn off import/no-commonjs and no-console for them.

github-actions · 2026-08-18T10:07:13Z

🔴 AI Literacy Score Report

Overall: 43/100 · Doc Lift: -5 · Actual: 41/100 · Ret. Gap: 2 · 112 tests · 0 errored · Cost: $39.67

Field	Value
Report ID	`01a0145b-b470-7fef-b0a1-43d18c9750c8`
Completed	Aug 18, 2026, 10:12 AM UTC
Duration	58.0s
Mode	full

♻️ Fully cached — Cost, tokens, and scores were served from the remote cache (source run run_20260810T221354Z_udjsmavo); only runtime reflects this run.

🔧 Environment

Setting	Value
Source	production
Docs URL	https://www.sanity.io/docs
Dataset	next
Project	3do82whm

🔍 Source verification

Setting	Value
Source	https://www.sanity.io/docs
Mode	full
Search	open

Scores by Feature Area

Feature	Score	Grade	Task	Code	Docs	Doc Lift	Tests
studio-presets	43	🔴 D	58	28	26	📉 -5	56

🚨 Negative Doc Lift: studio-presets (-5) — docs hurt performance. Floor: 48, Ceiling: 43

🔬 Three-Layer Decomposition

Feature	Floor	Ceiling	Actual	Doc Lift	Retr. Gap	Infra %
studio-presets	48	43	41	📉 -5	+2	—

📖 What do these numbers mean?

Floor: Model performance without any documentation (training data only)
Ceiling: Model performance with perfect documentation (hand-picked, injected)
Actual: Model performance when finding docs on its own (like real users)
Doc Lift: Documentation quality contribution (Ceiling − Floor)
Retr. Gap: Quality lost in discovery (Ceiling − Actual)
Infra %: Fraction of doc quality reaching agents (Actual ÷ Ceiling)

🤖 Per-model scores

Model	Score	Doc Lift	Tests	Cost
Claude Opus 4.6	46	-1	7	$0.43
Claude Opus 5	45	0	7	$1.53
GPT 5.2	44	-3	7	$0.14
GPT 5.4	44	-4	7	$0.46
Claude Fable 5	43	-4	7	$1.53
GPT 5.6 Terra	40	-13	7	$0.29
Claude Sonnet 4.6	40	-10	7	$0.40
Claude Opus 4.8	40	-10	7	$0.85

Claude Opus 4.6 (anthropic:messages:claude-opus-4-6):

Feature	Score	Task	Code	Docs	Lift
studio-presets	46	66	29	24	-1

Claude Opus 5 (anthropic:messages:claude-opus-5):

Feature	Score	Task	Code	Docs	Lift
studio-presets	45	66	20	29	0

GPT 5.2 (openai:chat:gpt-5.2):

Feature	Score	Task	Code	Docs	Lift
studio-presets	44	56	38	28	-3

GPT 5.4 (openai:responses:gpt-5.4):

Feature	Score	Task	Code	Docs	Lift
studio-presets	44	56	33	33	-4

Claude Fable 5 (anthropic:messages:claude-fable-5):

Feature	Score	Task	Code	Docs	Lift
studio-presets	43	60	25	25	-4

GPT 5.6 Terra (openai:responses:gpt-5.6-terra):

Feature	Score	Task	Code	Docs	Lift
studio-presets	40	54	27	24	-13

Claude Sonnet 4.6 (anthropic:messages:claude-sonnet-4-6):

Feature	Score	Task	Code	Docs	Lift
studio-presets	40	54	29	24	-10

Claude Opus 4.8 (anthropic:messages:claude-opus-4-8):

Feature	Score	Task	Code	Docs	Lift
studio-presets	40	56	25	25	-10

📊 Score Comparison

Overall delta: 0 (➡️, threshold ±2)

Feature	Baseline	Current	Delta	Actual Δ	Ret. Gap Δ	Infra Δ
studio-presets	43	43	➡️ 0	0	0	—

➡️ 1 unchanged (threshold: ±2)

Dimension averages

Dimension	Delta
CodeCorrectness	0
DocCoverage	0
TaskCompletion	0
Doc Lift	0

💰 Eval cost breakdown

Category	Cost
Provider (model inference)	$39.67
Total	$39.67

Provider cost by feature area:

Feature	Tests	Cost	Avg/Test
studio-presets	56	$39.67	$0.71

🔍 Low-Scoring Judgments (50 below 70)

Add CTA buttons to a hero section (baseline) (8)

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · anthropic:messages:claude-opus-4-6

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared through a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types with manual url fie

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · openai:responses:gpt-5.6-terra

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared through a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types with defineType/`de

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · openai:chat:gpt-5.2

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared via a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types with defineType/`define

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · openai:responses:gpt-5.4

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared via a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types (linkAction) with a man

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · anthropic:messages:claude-sonnet-4-6

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared through a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types with defineType/`de

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · anthropic:messages:claude-opus-4-8

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared via a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types with defineType/`define

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · anthropic:messages:claude-fable-5

The response uses manual schema definition with defineType/defineField/defineArrayMember instead of the required defineCta function from createPresetsRegistry(). Internal links use a url field with allowRelative instead of the required 'to' option pattern. Neither rubric criterio

🔴 0 · Code Correctness · Add CTA buttons to a hero section (baseline) · anthropic:messages:claude-opus-5

The rubric requires CTA buttons to be produced via defineCta from createPresetsRegistry() and internal links declared through a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types with defineType/defineField and manually con

Add CTA buttons to a hero section (gold) (7)

🔴 0 · Code Correctness · Add CTA buttons to a hero section (gold) · openai:responses:gpt-5.6-terra

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared via a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object schemas with manual url field

🔴 0 · Code Correctness · Add CTA buttons to a hero section (gold) · openai:chat:gpt-5.2

The rubric requires CTA buttons to be produced by a defineCta function obtained from createPresetsRegistry(), and internal link targets declared via a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object schemas with defineField/`def

🔴 0 · Code Correctness · Add CTA buttons to a hero section (gold) · openai:responses:gpt-5.4

The response uses hand-rolled object types with defineType/defineField for CTA buttons instead of the required defineCta function from createPresetsRegistry(). Internal link targets use a manual 'href' string field rather than the required 'to' option pattern (link: {to: [...]} o

🔴 0 · Code Correctness · Add CTA buttons to a hero section (gold) · anthropic:messages:claude-sonnet-4-6

The candidate uses hand-rolled object types with defineField/defineType instead of the required defineCta function from createPresetsRegistry(). Internal links use a url field instead of the required 'to' option pattern. Neither rubric criterion is met.

🔴 0 · Code Correctness · Add CTA buttons to a hero section (gold) · anthropic:messages:claude-opus-4-8

Response does not use createPresetsRegistry() or defineCta function as required by rubric. Instead uses non-existent @sanity/presets package with presets() plugin. No to option for internal link targets. Completely misses the documented API pattern for CTA buttons.

🔴 0 · Code Correctness · Add CTA buttons to a hero section (gold) · anthropic:messages:claude-fable-5

The candidate response uses a hand-rolled object type with label/url fields instead of the required defineCta function from createPresetsRegistry(). Internal link targets should use a to option (either on createPresetsRegistry or defineLink), but the response uses a man

🔴 0 · Code Correctness · Add CTA buttons to a hero section (gold) · anthropic:messages:claude-opus-5

The rubric requires CTA buttons to be produced via defineCta from createPresetsRegistry() and internal links declared through a to option on createPresetsRegistry or defineLink. The candidate uses hand-rolled object types with defineType/defineField and manual `refe

Add SEO metadata to a page (baseline) (3)

🔴 0 · Code Correctness · Add SEO metadata to a page (baseline) · anthropic:messages:claude-opus-4-8

The rubric explicitly requires that the metadata field be produced by a defineSeo function obtained from a createPresetsRegistry() call. The candidate response hand-rolls an SEO object using defineType/defineField instead of using defineSeo from createPresetsRegistry(). This

🔴 0 · Code Correctness · Add SEO metadata to a page (baseline) · anthropic:messages:claude-fable-5

The rubric explicitly requires that the metadata field be produced by a defineSeo function obtained from a createPresetsRegistry() call. The candidate response uses a hand-rolled object type with defineType/defineField composing title, description, and image fields manually,

🔴 0 · Code Correctness · Add SEO metadata to a page (baseline) · anthropic:messages:claude-opus-5

The response provides a well-structured, idiomatic Sanity schema implementation for SEO metadata. However, it completely fails the specific rubric criterion: it uses hand-rolled defineField/defineType objects instead of using defineSeo from createPresetsRegistry() as required

Add SEO metadata to a page (gold) (1)

🔴 0 · Code Correctness · Add SEO metadata to a page (gold) · openai:responses:gpt-5.6-terra

The rubric explicitly requires that the metadata field be produced by a defineSeo function obtained from a createPresetsRegistry() call. The candidate response uses sanity-plugin-seo-pane with a custom seo field type, which is not the required approach. No defineSeo or

Compose a full landing page schema (baseline) (8)

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · openai:chat:gpt-5.2

Response completely ignores the required preset-based architecture. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Everything is hand-rolled from scratch instead of using the documented preset system.

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · anthropic:messages:claude-opus-4-8

Response completely ignores the required preset-based architecture. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls all content patterns (slug, SEO, rich text, CTA, image blocks) instead of using presets. Page sections use m

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · anthropic:messages:claude-opus-4-6

Response hand-rolls everything instead of using the required preset registry pattern. No createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage calls. Slug/SEO are manual fields, not from definePage. Hero CTA uses hand-rolled link type instead of defineCta

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · anthropic:messages:claude-fable-5

The response completely ignores the required preset-based approach. It hand-rolls all content patterns instead of using createPresetsRegistry() with definePage, defineCta, defineRichText, and defineImage. No presets are used anywhere.

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · openai:responses:gpt-5.6-terra

Candidate hand-rolls every content pattern instead of using the required preset registry. No createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage calls. Page sections built manually via array field, slug/SEO added as separate fields, CTA hand-rolled, ric

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · anthropic:messages:claude-sonnet-4-6

Response completely ignores the required preset-based architecture. No createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage presets are used. Everything is hand-rolled from scratch: slug/SEO fields manually added, CTA button manually constructed, rich te

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · openai:responses:gpt-5.4

Response completely ignores the required preset-based architecture. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls all content patterns (slug, SEO, rich text, CTA, image blocks) instead of using presets. Hero is correctly h

🔴 0 · Code Correctness · Compose a full landing page schema (baseline) · anthropic:messages:claude-opus-5

Response completely ignores the required preset-based approach. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls all content patterns (slug, SEO, rich text, CTA button, image blocks) instead of using presets. No pageBuilderBl

Compose a full landing page schema (gold) (8)

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · openai:chat:gpt-5.2

The response hand-rolls every content pattern from scratch instead of using the required preset registry. It does not call createPresetsRegistry(), does not use definePage with pageBuilderBlocks, does not use defineRichText, defineCta, or defineImage. Slug and SEO are manually ad

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · openai:responses:gpt-5.6-terra

The response hand-rolls every content pattern (CTA, rich text, image, SEO, slug, page sections) instead of using createPresetsRegistry() with definePage, defineCta, defineRichText, defineImage. None of the required preset APIs are used; pageBuilderBlocks is absent. The only crite

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · anthropic:messages:claude-opus-4-6

Response completely ignores the required preset-based architecture. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls all content patterns (slug, SEO, rich text, CTA button, image with caption) instead of using the preset syst

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · openai:responses:gpt-5.4

The candidate hand-rolls every content pattern from scratch. It does not call createPresetsRegistry() to obtain definePage, defineCta, defineRichText, or defineImage. Page sections are manually constructed as an array field instead of using pageBuilderBlocks. Slug and SEO are add

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · anthropic:messages:claude-fable-5

Response completely ignores the required preset-based architecture. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls all content patterns (slug, SEO, rich text, CTA button, image with caption) instead of using the documented

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · anthropic:messages:claude-sonnet-4-6

Response completely ignores the required preset-based architecture. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls all content patterns (slug, SEO, rich text, CTA, image blocks) instead of using built-in presets. Hero CTA i

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · anthropic:messages:claude-opus-4-8

Response completely ignores the required API. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls all content patterns (slug, SEO, rich text, CTA button, image with caption) instead of using the preset functions. References non-

🔴 0 · Code Correctness · Compose a full landing page schema (gold) · anthropic:messages:claude-opus-5

Response completely ignores the required API. Does not use createPresetsRegistry(), definePage, defineCta, defineRichText, or defineImage. Hand-rolls slug, SEO, rich text (portableText), CTA (link), and image fields instead of using the preset functions. References non-existent @

Configure an editorial image with crop control and caption (baseline) (1)

🔴 0 · Code Correctness · Configure an editorial image with crop control and caption (baseline) · anthropic:messages:claude-opus-4-6

The rubric explicitly requires that coverImage be produced by a defineImage function obtained from createPresetsRegistry(), not a hand-rolled defineField({type: 'image', ...}). The candidate uses the manual approach with defineField and type: 'image', directly violating the

Configure an editorial image with crop control and caption (gold) (2)

🔴 0 · Code Correctness · Configure an editorial image with crop control and caption (gold) · openai:responses:gpt-5.6-terra

The rubric explicitly requires that coverImage be produced by a defineImage function obtained from createPresetsRegistry(), not a hand-rolled defineField({type:'image', options:{hotspot:true}, fields:[...]}). The candidate uses the hand-rolled approach, so the key criterion

🔴 0 · Code Correctness · Configure an editorial image with crop control and caption (gold) · anthropic:messages:claude-opus-4-6

The rubric explicitly requires that coverImage be produced by a defineImage function obtained from a createPresetsRegistry() call. The candidate uses a hand-rolled defineField({type: 'image', options: {hotspot: true}, fields: [...]}) approach instead. This directly violates

Create Studio with page preset (gold) (12)

🔴 0 · Task Completion · Create Studio with page preset (gold) · openai:chat:gpt-5.2

The candidate response does not use createPresetsRegistry from @sanity/presets, does not call definePage with name: 'page', and uses a non-existent pageTypes export. None of the required rubric criteria are met.

🔴 0 · Doc Coverage · Create Studio with page preset (gold) · openai:chat:gpt-5.2

Candidate uses defineConfig correctly (documented inline). However, the core implementation relies on '@sanity/presets' and 'pageTypes' which are completely absent from the reference documentation. The candidate had to hallucinate/guess the page preset API, import path, and schem

🔴 0 · Task Completion · Create Studio with page preset (gold) · anthropic:messages:claude-sonnet-4-6

Response does not use createPresetsRegistry or definePage at all. Uses non-existent pageType import from @sanity/presets/page. Does not place definePage result in schema.types array. Completely wrong approach that contradicts all rubric requirements.

🔴 0 · Doc Coverage · Create Studio with page preset (gold) · anthropic:messages:claude-sonnet-4-6

Candidate references defineConfig and plugins correctly from docs, but @sanity/presets and pageType() are completely hallucinated - not mentioned anywhere in reference. The core implementation details (package name, import path, preset function) have no documentation support. Onl

🔴 0 · Task Completion · Create Studio with page preset (gold) · openai:responses:gpt-5.6-terra

Response does not use createPresetsRegistry from @sanity/presets, does not call definePage with name: 'page', and does not place the result in schema.types. Instead invents a non-existent pageType() plugin API. Fundamentally wrong approach that contradicts all rubric requirements

🔴 0 · Doc Coverage · Create Studio with page preset (gold) · openai:responses:gpt-5.6-terra

The candidate response fabricates '@sanity/presets' package and 'pageType' function which are not documented in the reference. Only 'defineConfig' and 'plugins' array usage align with documented concepts. The core implementation details (preset package, pageType function, its beh

🔴 0 · Task Completion · Create Studio with page preset (gold) · anthropic:messages:claude-opus-4-8

Response does not use createPresetsRegistry from @sanity/presets or call definePage with name: 'page'. Instead invents pageType export which is not in the reference. None of the rubric criteria are met.

🔴 0 · Task Completion · Create Studio with page preset (gold) · anthropic:messages:claude-fable-5

The candidate response does not use createPresetsRegistry from @sanity/presets, does not call definePage with name: 'page', and does not place the result in schema.types. Instead, it invents a pageType plugin pattern that is not documented in the reference. None of th

🔴 0 · Doc Coverage · Create Studio with page preset (gold) · anthropic:messages:claude-fable-5

Response uses documented concepts (defineConfig, definePlugin, plugins) correctly but fabricates @sanity/presets package, pageType() function, and all its features (page builder, SEO metadata, etc.). The reference contains no information about presets or page types. Core task imp

🔴 0 · Task Completion · Create Studio with page preset (gold) · openai:responses:gpt-5.4

The response does not use createPresetsRegistry from @sanity/presets, does not call definePage with name: 'page', and does not place the result in schema.types. Instead, it imports a non-existent pageType and adds it to plugins. None of the rubric criteria are met.

🔴 0 · Doc Coverage · Create Studio with page preset (gold) · openai:responses:gpt-5.4

Candidate uses defineConfig and plugins correctly per docs. However, @sanity/presets and pageType() are completely hallucinated - not mentioned anywhere in reference. The core task (page preset) required inventing an undocumented package and API. Only basic Studio config structur

🔴 0 · Task Completion · Create Studio with page preset (gold) · anthropic:messages:claude-opus-5

The response does not use createPresetsRegistry from @sanity/presets, does not call definePage with name: 'page', and does not place the result in schema.types. Instead, it guesses at exports like pageType and uses plugins array. None of the rubric criteria are met.

💡 Recommendations

🔴 studio-presets (score: 43) — needs significant doc improvements.

Generated by AI Literacy Framework · Aug 18, 2026, 10:12 AM UTC · view detailed results

Previous runs

2026-08-18 10:12 UTC - ?/100

AI Literacy Evaluation - `plugins/@sanity/presets`

No report generated. Check the workflow logs.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Script path omitted from CI triggers
- Added .github/scripts/post-ailf-comment.cjs to the workflow's pull_request.paths filter so changes to the extracted comment script trigger the AI Literacy Eval workflow.

Or push these changes by commenting:

@cursor push c07541e310

Preview (c07541e310)

diff --git a/.github/workflows/ailf-eval.yml b/.github/workflows/ailf-eval.yml
--- a/.github/workflows/ailf-eval.yml
+++ b/.github/workflows/ailf-eval.yml
@@ -5,6 +5,7 @@
     branches: [main]
     paths:
       - 'plugins/**/.ailf/**'
+      - '.github/scripts/post-ailf-comment.cjs'
       - '.github/workflows/ailf-eval.yml'
 
   workflow_dispatch:

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 230a649. Configure here.}

cursor · 2026-08-18T10:09:31Z

        env:
          PLUGIN_KEY: ${{ steps.key.outputs.key }}
          PLUGIN_PATH: ${{ matrix.plugin }}
+          WORKFLOW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}


Script path omitted from CI triggers

Low Severity

The PR comment logic moved into .github/scripts/post-ailf-comment.cjs, but the workflow paths filter still only lists plugins/**/.ailf/** and ailf-eval.yml. PRs that change only the extracted script will not run AI Literacy Eval, so the Post PR comment step this migration relies on will not be exercised.

^{Reviewed by Cursor Bugbot for commit 230a649. Configure here.}

github-actions · 2026-08-18T10:11:47Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	32.01%	5511 / 17213
🔵	Statements	31.78%	5815 / 18294
🔵	Functions	27.79%	1428 / 5137
🔵	Branches	23.35%	2831 / 12123

File Coverage

No changed files found.

Generated in workflow #9214 for commit 230a649 by the Vitest Coverage Report Action

fix(ci): make ailf-eval compatible with github-script v9

b177fe1

Move the AILF PR comment helper into a CommonJS module so the v9-injected getOctokit parameter cannot clash with inline script declarations, and pass the workflow URL through env instead of interpolating expressions into JS.

stipsan added the 🤖 bot label Aug 18, 2026 — with Cursor

vercel Bot deployed to Preview – plugins-studio August 18, 2026 10:05 View deployment

chore(lint): allow CommonJS github-script helpers

230a649

github-script loads `.github/scripts/*.cjs` via require(), so those files cannot be ESM. Turn off import/no-commonjs and no-console for them.

vercel Bot deployed to Preview – plugins-studio August 18, 2026 10:07 View deployment

cursor Bot reviewed Aug 18, 2026

View reviewed changes

Navigation Menu

Uh oh!

fix(ci): make ailf-eval compatible with github-script v9 - #1915

fix(ci): make ailf-eval compatible with github-script v9#1915
stipsan wants to merge 2 commits into
mainfrom
cursor/fix-ailf-github-script-v9-5201

stipsan commented Aug 18, 2026 •

edited by cursor Bot

Loading

Uh oh!

changeset-bot Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 18, 2026 •

edited

Loading

Add CTA buttons to a hero section (baseline) (8)

Add CTA buttons to a hero section (gold) (7)

Add SEO metadata to a page (baseline) (3)

Add SEO metadata to a page (gold) (1)

Compose a full landing page schema (baseline) (8)

Compose a full landing page schema (gold) (8)

Configure an editorial image with crop control and caption (baseline) (1)

Configure an editorial image with crop control and caption (gold) (2)

Create Studio with page preset (gold) (12)

AI Literacy Evaluation - `plugins/@sanity/presets`

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot Aug 18, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

stipsan commented Aug 18, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What broke

Fix

Verification

Uh oh!

changeset-bot Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

vercel Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ E2E Tests

Uh oh!

github-actions Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 AI Literacy Score Report

Scores by Feature Area

🔬 Three-Layer Decomposition

📊 Score Comparison

Add CTA buttons to a hero section (baseline) (8)

Add CTA buttons to a hero section (gold) (7)

Add SEO metadata to a page (baseline) (3)

Add SEO metadata to a page (gold) (1)

Compose a full landing page schema (baseline) (8)

Compose a full landing page schema (gold) (8)

Configure an editorial image with crop control and caption (baseline) (1)

Configure an editorial image with crop control and caption (gold) (2)

Create Studio with page preset (gold) (12)

💡 Recommendations

Previous runs

AI Literacy Evaluation - plugins/@sanity/presets

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Aug 18, 2026

Choose a reason for hiding this comment

Script path omitted from CI triggers

Uh oh!

github-actions Bot commented Aug 18, 2026

Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stipsan commented Aug 18, 2026 •

edited by cursor Bot

Loading

changeset-bot Bot commented Aug 18, 2026 •

edited

Loading

vercel Bot commented Aug 18, 2026 •

edited

Loading

github-actions Bot commented Aug 18, 2026 •

edited

Loading

github-actions Bot commented Aug 18, 2026 •

edited

Loading

AI Literacy Evaluation - `plugins/@sanity/presets`

cursor Bot left a comment •

edited

Loading