Skip to content

add /stats-info/{sport}/{division} endpoint#34

Merged
henrygd merged 2 commits into
henrygd:masterfrom
kkolodziej39:claude/clever-rhodes
Apr 16, 2026
Merged

add /stats-info/{sport}/{division} endpoint#34
henrygd merged 2 commits into
henrygd:masterfrom
kkolodziej39:claude/clever-rhodes

Conversation

@kkolodziej39
Copy link
Copy Markdown
Contributor

@kkolodziej39 kkolodziej39 commented Apr 14, 2026

Summary

Adds /stats-info/{sport}/{division} — a discovery endpoint that returns the available stat categories (individual + team) for a given sport and division. Lets API consumers find valid {path} values for the existing /stats endpoint without having to click around on ncaa.com.

Response shape

{
  "individual": [
    { "id": "5",  "label": "Goals Per Game",  "path": "individual/5" },
    { "id": "6",  "label": "Assists Per Game", "path": "individual/6" }
  ],
  "team": [
    { "id": "30", "label": "Scoring Offense", "path": "team/30" }
  ]
}

How it works

  • Fetches https://www.ncaa.com/stats/{sport}/{division} on cache miss
  • Parses the two <select id=\"select-container-{individual|team}\"> dropdowns with linkedom
  • One upstream request per sport/division
  • Caches with a new cache_24h bucket (Cache-Control: public, max-age=86400) since stat categories rarely change
  • Returns 404 if NCAA.com returns 404, or if no stat categories are found in the page

Sport support

Works for any sport/division NCAA.com supports — the parser doesn't special-case soccer. Tested against soccer (men/women, D1/D2/D3), icehockey-men/d1, and football/fbs.

Files

File Change
src/stats/parser.ts New — reusable dropdown parser
src/types/stats.ts New — shared types
src/index.ts Endpoint handler + new cache_24h + onBeforeHandle updated to set max-age=86400 for 24h cache
src/openapi.ts Spec entry for /stats-info; description added to /stats {path} parameter pointing users at /stats-info
test/index.test.ts 5 integration tests (live NCAA.com)
test/parser.test.ts New — 6 HTML fixture tests for the parser

Hardening separated

Per #34 review feedback, the unrelated security/code-quality fixes from the original PR have been moved to a separate draft: #35. This PR is now feature-only.

Test plan

  • All 35 tests pass (29 existing + 5 new integration tests + 6 parser fixture tests, with previous Stats info block removed and replaced)
  • Verify dropdown parsing on additional sports: icehockey-men/d1 and football/fbs + screenshots added below

Additional Test Items from PR Reviewer

  • Verify dropdown parsing on additional sports beyond what I tested
  • Confirm 24h cache TTL is acceptable (or adjust)

Screenshots

Screenshot 2026-04-14 at 4 55 04 PM Screenshot 2026-04-14 at 4 54 37 PM

🤖 Generated with Claude Code

@kkolodziej39 kkolodziej39 changed the title Add stat path discovery endpoint and security hardening Add stat path discovery endpoint (currently just soccer support) Apr 14, 2026
@kkolodziej39
Copy link
Copy Markdown
Contributor Author

Screenshots attached of the new endpoint
Screenshot 2026-04-14 at 12 13 52 PM
Screenshot 2026-04-14 at 12 14 02 PM

@kkolodziej39 kkolodziej39 marked this pull request as ready for review April 14, 2026 19:16
Copy link
Copy Markdown
Owner

@henrygd henrygd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I'm really swamped and don't have time to go through line by line at the moment.

If possible, it would be better for me if this could be split this into two PRs. One for the new feature and one for changes to existing code that aren't directly related to the new feature.

Regarding the new endpoint: Is it incompatible currently with other sports, or is soccer just the only one that you've tested with?

How many requests do we need to make upstream in order to gather the data for a specific sport and division? Is it just the one request to ncaa.com/stats/sport/division and then gathering the values from the select fields?

Is there a reason for using a generator script rather than fetching the values when the first user request comes in and saving it in a cache like the other endpoints? It doesn't seem like that expensive of an operation. I know these values rarely change, so we can add a new longer term cache like 24 hours.

Returns the list of available stat categories (individual + team) for
a given sport and division, so API consumers can discover valid
{path} values for the existing /stats endpoint without having to
click around on ncaa.com.

Response shape:
  {
    "individual": [{ "id": "5", "label": "Goals Per Game", "path": "individual/5" }, ...],
    "team":       [{ "id": "30", "label": "Scoring Offense",  "path": "team/30"       }, ...]
  }

Implementation notes:
- Fetches https://www.ncaa.com/stats/{sport}/{division} on cache miss,
  parses the two <select id="select-container-{individual|team}">
  dropdowns with linkedom. One upstream request per sport/division.
- 24h cache (new cache_24h) since stat categories rarely change. Returns
  Cache-Control: public, max-age=86400.
- Works for any sport/division NCAA.com supports — the parser doesn't
  special-case soccer. Tested against soccer (men/women, D1/D2/D3)
  and basketball-men/d1.

Files:
- src/stats/parser.ts — reusable dropdown parser
- src/types/stats.ts — shared types
- src/index.ts — endpoint handler + cache_24h + onBeforeHandle update
- src/openapi.ts — spec entry for /stats-info, plus description update
  on /stats {path} param pointing users at /stats-info
- test/index.test.ts — integration tests (live NCAA.com)
- test/parser.test.ts — HTML fixture tests for the parser

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kkolodziej39 kkolodziej39 force-pushed the claude/clever-rhodes branch from 69bef35 to 3d0729a Compare April 14, 2026 23:47
@kkolodziej39
Copy link
Copy Markdown
Contributor Author

If possible, it would be better for me if this could be split this into two PRs. One for the new feature and one for changes to existing code that aren't directly related to the new feature.

Yes 👍 I can get a new PR up that splits the two to make it easier for review

Regarding the new endpoint: Is it incompatible currently with other sports, or is soccer just the only one that you've tested with?

Not incompatible, but I only fully tested it out with soccer. The src/stats/parser.ts should be agnostic to any sport with the dropdowns appearing on the NCAA stats landing pages I checked. However, I only tested it fully for soccer but happy to extend it to any sport + fetch at runtime (to your last point too of updating the PR to fetch at runtime + 24h cache added in)

How many requests do we need to make upstream in order to gather the data for a specific sport and division? Is it just the one request to ncaa.com/stats/sport/division and then gathering the values from the select fields?

Yes just that one request and then parse the two <select> dropdowns from that HTML

Is there a reason for using a generator script rather than fetching the values when the first user request comes in and saving it in a cache like the other endpoints? It doesn't seem like that expensive of an operation. I know these values rarely change, so we can add a new longer term cache like 24 hours.

Good point. I'll update the PR to follow those other endpoints pattern of the save in cache delete the script + generated json. Also, will add in a 24 hour cache to go with cache_30m and cache_45s. This will also help remove the full soccer scoping so it works on any sport/division

@kkolodziej39 kkolodziej39 changed the title Add stat path discovery endpoint (currently just soccer support) add /stats-info/{sport}/{division} endpoint Apr 14, 2026
@kkolodziej39
Copy link
Copy Markdown
Contributor Author

@henrygd Updated the PR following the comments above as well as splitting the unrelated cleanup to a draft PR here #35

Lmk if you have any other items to address and/or if you want me to go ahead with publishing that cleanup PR for review

Copy link
Copy Markdown
Owner

@henrygd henrygd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, this should be a useful feature.

I made a few small updates. The most significant is changing the endpoint to be /stats/{sport}/{division}. So it's the same as the existing /stats route, but without the year and path added on.

The public API and docker image will be updated in a few minutes.

@henrygd henrygd merged commit 621d615 into henrygd:master Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants