Corpus refresh 2026-10-10: User Guide f5676ad..ac82c47, web 0 added, 10 changed, 1 removed, 0 failed

[github-actions]

Corpus refresh, 2026-10-10

Left open for review: pytest -q did not pass, so this was not merged and Deploy was not run.

User Guide	f5676ad..ac82c47, 33 upstream commits: rcc-uchicago/user-guide@f5676ad...ac82c47
docs/	51 added, 33 changed, 0 removed
web/	55 fetched, 0 added, 10 changed, 1 removed, 0 failed

Guards

guard	result
ruff check .	passed
pytest -q	FAILED
tools/assembly_check.py	passed
tools/metrics.py --against (golden-set recall must not drop)	passed
tools/corpus_check.py	passed

Website pages

Added (0)

Changed (10):

• https://beag3.rcc.uchicago.edu
• https://cloud-skyway.rcc.uchicago.edu/faqs
• https://cloud-skyway.rcc.uchicago.edu/news-and-events
• https://cloud-skyway.rcc.uchicago.edu/user-guide
• https://rcc.uchicago.edu/about-rcc/our-team
• https://rcc.uchicago.edu/data-visualization-initiative-advisory-committee
• https://rcc.uchicago.edu/grants-publications/list-of-publications
• https://rcc.uchicago.edu/support-and-services/workshops-and-training
• https://vislab.rcc.uchicago.edu/tools-resources-color
• https://vislab.rcc.uchicago.edu/workshops-events

Removed (1):

• midway2.txt: lands on the same page as https://rcc.uchicago.edu/support-and-services/midway2

Failed (0)

Redirected (1):

• https://rcc.uchicago.edu/midway2 -> https://rcc.uchicago.edu/support-and-services/midway2

Blocked by robots.txt (5):

• https://learn-radiology.rcc.uchicago.edu/incidence
• https://learn-radiology.rcc.uchicago.edu/mpMRI
• https://learn-radiology.rcc.uchicago.edu/pirads
• https://learn-radiology.rcc.uchicago.edu/takecourse
• https://learn-radiology.rcc.uchicago.edu/team

Available, not included (65): in the sitemap and not in web_pages.toml. Add a page under [allow] to start fetching it, or under [deny] to stop it being listed here.

• https://rcc.uchicago.edu/
• https://rcc.uchicago.edu/about-rcc/brain-awareness-day
• https://rcc.uchicago.edu/about-rcc/committees
• https://rcc.uchicago.edu/about-rcc/faculty-education-advisory-committee
• https://rcc.uchicago.edu/about-rcc/job-opportunities
• https://rcc.uchicago.edu/about-rcc/location-contact
• https://rcc.uchicago.edu/about-rcc/mind-bytes-research-computing-expo-and-symposium
• https://rcc.uchicago.edu/about-rcc/news-features
• https://rcc.uchicago.edu/about-rcc/news-features/research-computing-center-roll-out-new-hpc-cluster-midway2
• https://rcc.uchicago.edu/about-rcc/resource-allocation-advisory-committee
• https://rcc.uchicago.edu/about-rcc/speaker-series-0
• https://rcc.uchicago.edu/accounts-allocations/account-extension-request
• https://rcc.uchicago.edu/accounts-allocations/accounting-service-units-0
• https://rcc.uchicago.edu/accounts-allocations/allocation-service-units
• https://rcc.uchicago.edu/accounts-allocations/calculations-service-units
• https://rcc.uchicago.edu/accounts-allocations/education-allocation-request
• https://rcc.uchicago.edu/accounts-allocations/general-user-account-request
• https://rcc.uchicago.edu/accounts-allocations/how-apply
• https://rcc.uchicago.edu/accounts-allocations/join-different-pi-account
• https://rcc.uchicago.edu/accounts-allocations/midwayr-pi-account-request
• https://rcc.uchicago.edu/accounts-allocations/midwayr-user-account-request
• https://rcc.uchicago.edu/accounts-allocations/msca-graham-school-capstones-project-request
• https://rcc.uchicago.edu/accounts-allocations/pi-account-request
• https://rcc.uchicago.edu/accounts-allocations/request-account
• https://rcc.uchicago.edu/accounts-allocations/request-allocation
• https://rcc.uchicago.edu/accounts-allocations/research-i-allocation-request
• https://rcc.uchicago.edu/accounts-allocations/research-ii-allocation-request
• https://rcc.uchicago.edu/accounts-allocations/sde2-project-workspace-request
• https://rcc.uchicago.edu/accounts-allocations/sde2-user-account-request
• https://rcc.uchicago.edu/accounts-allocations/sde3-project-workspace-request
• https://rcc.uchicago.edu/accounts-allocations/sde3-user-account-request
• https://rcc.uchicago.edu/accounts-allocations/special-allocation-request
• https://rcc.uchicago.edu/accounts-allocations/user-guidelines
• https://rcc.uchicago.edu/ancient-civilizations-modern-computation
• https://rcc.uchicago.edu/bottom
• https://rcc.uchicago.edu/computing-cereals-parallel
• https://rcc.uchicago.edu/cosmos-box
• https://rcc.uchicago.edu/decisions-decisions
• https://rcc.uchicago.edu/getting-started/supplemental-allocation-request
• https://rcc.uchicago.edu/grants-publications/grant-support-request
• https://rcc.uchicago.edu/happy-pi-day
• https://rcc.uchicago.edu/hi-neighbors
• https://rcc.uchicago.edu/hiv-epidemic-versus-social-network
• https://rcc.uchicago.edu/midway3
• https://rcc.uchicago.edu/modular-universe
• https://rcc.uchicago.edu/mood-market
• https://rcc.uchicago.edu/one-step-ahead-flu
• https://rcc.uchicago.edu/planetary-puzzle
• https://rcc.uchicago.edu/protein-mystery-unfolds
• https://rcc.uchicago.edu/rcc-support-digital-humanities
• https://rcc.uchicago.edu/research/funding-opportunities-and-calls-proposals
• https://rcc.uchicago.edu/research/supported-projects
• https://rcc.uchicago.edu/resources/hadooprcc
• https://rcc.uchicago.edu/rolling-punches
• https://rcc.uchicago.edu/signs-things-come
• https://rcc.uchicago.edu/simulation-you-never-have-run
• https://rcc.uchicago.edu/soft-materials-hard-problems
• https://rcc.uchicago.edu/splashes-space
• https://rcc.uchicago.edu/statistical-cell
• https://rcc.uchicago.edu/sweet-spot
• https://rcc.uchicago.edu/tiny-anvils
• https://rcc.uchicago.edu/uchicago-partners-british-universities-research-computing-and-big-data-cyber-infrastructure
• https://rcc.uchicago.edu/whats-your-scene
• https://rcc.uchicago.edu/woods
• https://rcc.uchicago.edu/words-wise

User Guide files

docs/: 51 added, 33 changed, 0 removed

• added: docs/software/apps-and-envs/binsanity.md
• added: docs/software/apps-and-envs/cesm.md
• added: docs/software/apps-and-envs/meshlab.md
• added: docs/software/apps-and-envs/wrf.md
• added: docs/tutorials/sde2/compute.md
• added: docs/tutorials/sde2/connection.md
• added: docs/tutorials/sde2/data-transfer.md
• added: docs/tutorials/sde2/images/avd1.png
• added: docs/tutorials/sde2/images/avd2.png
• added: docs/tutorials/sde2/images/avd3.png
• added: docs/tutorials/sde2/images/avd4.png
• added: docs/tutorials/sde2/images/avd5.png
• added: docs/tutorials/sde2/images/avd6.png
• added: docs/tutorials/sde2/images/avd7.png
• added: docs/tutorials/sde2/images/avd8.png
• added: docs/tutorials/sde2/images/avd9.png
• added: docs/tutorials/sde2/images/avd_desktop.png
• added: docs/tutorials/sde2/images/avd_desktop_add.png
• added: docs/tutorials/sde2/images/avd_desktop_workspace.png
• added: docs/tutorials/sde2/images/avd_login.png
• added: docs/tutorials/sde2/images/avd_webpage.png
• added: docs/tutorials/sde2/images/avd_winscp.png
• added: docs/tutorials/sde2/images/box_download.jpg
• added: docs/tutorials/sde2/images/box_login.jpg
• added: docs/tutorials/sde2/images/connection.png
• added: docs/tutorials/sde2/images/connection_sde2.png
• added: docs/tutorials/sde2/images/downloads.jpg
• added: docs/tutorials/sde2/images/putty.jpg
• added: docs/tutorials/sde2/images/putty_confirm.jpg
• added: docs/tutorials/sde2/images/sde2_tmp.png
• added: docs/tutorials/sde2/images/tl_2fa.jpg
• added: docs/tutorials/sde2/images/tl_login.jpg
• added: docs/tutorials/sde2/images/tl_logout.jpg
• added: docs/tutorials/sde2/images/tl_terminal.jpg
• added: docs/tutorials/sde2/images/tl_terminal2.jpg
• added: docs/tutorials/sde2/images/vdi_desktopicon.jpg
• added: docs/tutorials/sde2/images/vdi_login.jpg
• added: docs/tutorials/sde2/images/vdi_logout.jpg
• added: docs/tutorials/sde2/images/vdi_putty.jpg
• added: docs/tutorials/sde2/images/vdi_puttyserver.jpg
• added: docs/tutorials/sde2/images/vdi_puttywarning.jpg
• added: docs/tutorials/sde2/images/vdi_servername.jpg
• added: docs/tutorials/sde2/images/winscp.jpg
• added: docs/tutorials/sde2/images/winscp_hostkey.jpg
• added: docs/tutorials/sde2/images/winscp_hostkey_sde2.png
• added: docs/tutorials/sde2/images/winscp_upload1.jpg
• added: docs/tutorials/sde2/images/winscp_upload1_sde2.png
• added: docs/tutorials/sde2/images/winscp_upload2.jpg
• added: docs/tutorials/sde2/images/winscp_upload2_sde2.png
• added: docs/tutorials/sde2/overview.md
• added: docs/tutorials/sde2/software.md
• changed: docs/connection/ssh/main.md
• changed: docs/connection/thinlinc/troubleshooting.md
• changed: docs/data_sharing/access_control.md
• changed: docs/data_sharing/globus_sharing.md
• changed: docs/data_transfer/cloud/cloud-sync.md
• changed: docs/ecosystems.md
• changed: docs/glossary.md
• changed: docs/slurm/faq.md
• changed: docs/slurm/main.md
• changed: docs/slurm/sbatch.md
• changed: docs/software/apps-and-envs/gaussian.md
• changed: docs/software/apps-and-envs/gromacs.md
• changed: docs/software/apps-and-envs/mathematica.md
• changed: docs/software/apps-and-envs/matlab.md
• changed: docs/software/apps-and-envs/namd.md
• changed: docs/software/apps-and-envs/nwchem.md
• changed: docs/software/apps-and-envs/openmm.md
• changed: docs/software/apps-and-envs/r.md
• changed: docs/software/apps-and-envs/scode/environments.md
• changed: docs/software/apps-and-envs/scode/main.md
• changed: docs/software/apps-and-envs/scode/python.md
• changed: docs/software/apps-and-envs/singularity.md
• changed: docs/software/apps-and-envs/spaceranger.md
• changed: docs/software/apps-and-envs/tf-and-torch.md
• changed: docs/software/compilers.md
• changed: docs/software/faq.md
• changed: docs/software/index.md
• changed: docs/storage/main.md
• changed: docs/stylesheets/extra.css
• changed: docs/tutorials/kicp.md
• changed: docs/tutorials/sde3/compute.md
• changed: docs/tutorials/sde3/data-transfer.md
• changed: docs/tutorials/sde3/overview.md

Retrieval metrics, before and after

retrieval: golden set (n=36)
   recall@5  100.0%  (+0.0%)
   recall@3  100.0%  (+0.0%)
   p@1        88.9%  (+0.0%)
   MRR        0.944  (+0.005)
   depth      2.278  (+0.000)
   better: 'what clusters does RCC run' 3 -> 2
retrieval: + known gaps (n=38)
   recall@5  100.0%
   recall@3  100.0%
   p@1        84.2%
   MRR        0.921
   depth      2.237

corpus_check

What the refreshed corpus can and cannot answer

corpus     636 chunks
   docs     80 pages   545 chunks     432.1k chars
   kb        1 pages     9 chunks       4.7k chars
   web      42 pages    82 chunks     131.1k chars

empty documents (2): topics nothing can answer
       0 bytes  docs/data_transfer/cloud/rclone.md
     188 bytes  web/takecourse.txt

indexed nothing (1): read, and no section came out
   docs/data_transfer/cloud/rclone.md
   -> asked by outcome rather than by file size, so a reader that stops recognising a heading style shows up here rather than nowhere.

duplicate sections   one page indexed twice 0   shared boilerplate 5   near-duplicate pairs 0 (0 across sources)
   shared boilerplate (keep both, they cite different pages): ['docs/tutorials/sde2/compute.md#slurm-partitions', 'docs/tutorials/sde3/compute.md#slurm-partitions']
   shared boilerplate (keep both, they cite different pages): ['docs/tutorials/sde2/data-transfer.md#method-1-download-data-from-a-website', 'docs/tutorials/sde3/data-transfer.md#method-1-download-data-from-a-website']
   shared boilerplate (keep both, they cite different pages): ['docs/tutorials/sde2/data-transfer.md#method-2-download-data-from-a-server', 'docs/tutorials/sde3/data-transfer.md#method-2-download-data-from-a-server']
   shared boilerplate (keep both, they cite different pages): ['docs/data_sharing/databases/bfi.md#method-2-globus', 'docs/data_sharing/databases/booth.md#method-2-globus']
   shared boilerplate (keep both, they cite different pages): ['docs/data_sharing/databases/bfi.md#method-3-ssh-scp-sftp-etc', 'docs/data_sharing/databases/booth.md#method-3-ssh-scp-sftp-etc']

reachability   204 of 636 chunks surfaced by 79 questions
   NOT MEASURABLE: 79 questions x 6 results is a ceiling of 474 chunks, below the 636 indexed. A percentage here would describe the question set, not the index.
   asked for by its own title: 98.4% of 123 pages are retrievable (2 are not)
      unreachable: docs/software/apps-and-envs/singularity.md  (titled 'Modules')
      unreachable: docs/tutorials/sde3/software.md  (titled 'Software')

topics the profile advertises
   ok         32.1  accounts       -> accounts-allocations.txt
   ok         25.8  connecting     -> connection/ssh/main.md
   CAVEATED   11.5  Slurm          -> slurm/main.md
   CAVEATED   16.5  storage        -> storage/main.md
   ok         26.9  software       -> tutorials/sde2/software.md
   ok         36.0  GPUs           -> software/compilers.md
   ok         25.9  policy         -> about-rcc_rcc-user-policy.txt
   -> a one-word query scores low because the floor is an unnormalised sum, not because the topic is missing. Worth knowing: a reader who types one word gets the caveat.

integrity   ids that do not resolve 0   chunks with no URL 9   unusable URLs 0
   no url: kb/thinlinc-login-problems.md#thinlinc-will-not-connect
   no url: kb/thinlinc-login-problems.md#login-fails-or-logs-you-straight-back-out
   no url: kb/thinlinc-login-problems.md#server-disconnected-after-logging-in
   no url: kb/thinlinc-login-problems.md#the-desktop-opens-but-the-display-is-wrong
   no url: kb/thinlinc-login-problems.md#no-agent-available

freshness  {"refreshed_at": "2026-10-10 18:00 UTC", "user_guide_commit": "ac82c47", "docs_files": 81, "web_files": 54, "web_refreshed_at": "2026-10-10 18:00 UTC", "sitemap_urls": 101, "web_pages_fetched": 55, "web_pages_changed": 10, "web_pages_removed": 1, "web_pages_failed": 0}

Opened by .github/workflows/refresh-corpus.yml; CI does not run on pull requests a workflow opens, so the guards above ran in that workflow.

[github-actions]

Not merged: a guard failed, or main moved while the refresh ran. The run: https://github.com/PursuitOfDataScience/sage/actions/runs/38073448878