[NielsRogge]

Why

• Legacy imports bound every arXiv byline of a common name to the first HF account that claimed it. One example is hao-li, with 306 papers here against 19 on HF.
• paper_authors.claimed was only backfilled from HF's daily-papers list. In a 300-link sample, a third of "unclaimed" links were in fact claimed by that account on HF.
• Ingestion no longer matches HF-linked rows by name. As a result, the same person can end up on two rows, for example /authors/181986 and /authors/251936, which are both Vik Paruchuri.

What

backend/scripts/backfill_hf_author_claims.py is a one-off job with two steps.

1. fetch caches the HF author list of every latest paper that has an unclaimed link on an HF-linked row (about 92.6k papers) into a JSON-lines file. It is read-only on the database, rate-limited, backs off on 429, and resumes on rerun.
2. run replays the cache in phases. It rolls back unless --apply is passed, and every change goes to a --report JSON-lines file:
   • claims: replays each list through AuthorHfProfileSyncService.sync_paper, the same logic as the daily sync.
   • doubles: keeps one link per byline slot, preferring the claimed link, else the name-only one. Same-name links at different positions may be two real coauthors, so they are only reported.
   • split: for accounts with claims, an unclaimed link that shares no coauthor with a claimed paper moves to the name-only author of the paper's byline name. If a row was renamed by a claim and the byline name is unsure, the link is skipped and reported.
   • merge: a name-only link moves to the only HF account with that name, and only on hard evidence:
     • a shared official repository with one of the account's anchor papers;
     • a coauthor whose identity is certain on both papers (an HF-linked row with claimed or curated links);
     • or at least two distinctive name-only coauthors: no two accounts use the name, and the row has at most 100 papers.
   • empty: deletes name-only rows left without papers and with nothing else pointing at them. Their URL becomes an author_aliases entry pointing to the row that took their papers, else to a same-name author.

Status

• Do not merge yet. The split rule still needs tuning on the full cache. A partial dry run would strip prolific authors with few claims, such as cavanloy and liuziwei7. The full fetch is running.
• Nothing has been applied to production. A dry run against production on 15% of the cache rolls back cleanly in about 3.5 minutes. It moves /authors/251936's paper to vikp (evidence: official repo) and aliases 251936 to 181986.

Validation

• uv run pytest backend/tests/test_author_claim_backfill_service.py passes 12 tests. test_author_hf_profile_sync_service.py passes 11.
• ruff check and ruff format are clean.

🤖 Generated with Claude Code

---

Note

High Risk
Bulk mutations to authors, paper links, and deletions with heuristic split/merge rules; PR notes split tuning still needed before production apply.

Overview
Adds a one-off migration to fix legacy Hugging Face author linkage: papers tied to the wrong HF account, missing paper_authors.claimed flags, and duplicate name-only vs HF rows.

backfill_hf_author_claims exposes two steps: fetch (DB read-only) appends HF author lists for papers with unclaimed HF-linked links into a resumable JSON-lines cache, with pacing and 429/5xx backoff; run replays the cache through ordered phases—claims via existing AuthorHfProfileSyncService.sync_paper, doubles (dedupe same-name links), split (move unclaimed namesake papers off HF rows), merge (move name-only links when repo/coauthor evidence is strong), empty (delete orphan name-only authors and wire author_aliases). Default is rollback (dry run); --apply commits per phase; every action is logged to --report.

author_claim_backfill_service holds the HF client, SQL-heavy merge logic, and AuthorClaimCleanup. 12 unit tests cover claims, split/merge edge cases, doubles, empty cleanup, and fetch resume behavior.

^{Reviewed by Cursor Bugbot for commit 8391823. Bugbot is set up for automated code reviews on this repo. Configure here.}

[NielsRogge]

Applied to production on 2026-11-03 at 10:10 UTC, from commit 6c2d1f1, with phases claims,doubles,merge,empty and a cache covering 16,875 of the ~92.6k papers. split was not applied.

• claims: 18,616 links marked claimed, 495 links moved to accounts that HF shows claimed them, 99 rows renamed
• doubles: 34 duplicate links removed; 267 same-name pairs reported and left untouched
• merge: 303 name-only links merged into 201 accounts, on evidence of a shared official repository (10), a verified coauthor (84) or shared coauthors (233)
• empty: 1,421 paper-less name-only rows deleted, 609 of them aliased to another author

Verified live: /authors/251936 resolves to /authors/181986 (Vik Paruchuri), and /authors/266 (Yann LeCun) gained 2 papers from /authors/240744.

Backup taken before the apply: reports/authors-backup-20261103T100343Z.dump on the VPS (pg_dump data-only of authors, paper_authors and author_aliases). The change report is at reports/author-claims-apply1-20261103.jsonl.

Still open: finish the HF fetch, re-run the same four phases on the full cache, tune split (it currently strips prolific authors who have few claims), then merge this PR.

[NielsRogge]

Second production apply, full HF cache (2026-11-03 17:30 UTC, same commit 6c2d1f1, phases claims,doubles,merge,empty; backup reports/authors-backup-20261103T170848Z.dump)

The cache now holds 92,656 papers, all of them fetched.

• claims: 85,426 links marked claimed, 1,437 accounts linked, 155 rows renamed, 31 conflicts left alone
• doubles: 85 removed; 244 skipped (different positions or no single keeper)
• merge: 920 links moved (+22 duplicates removed) into 629 accounts; evidence: repo 31, verified coauthor 551, shared coauthors 545
• empty: 790 rows deleted, 593 aliased (the old URLs 301 to the alias target)

The apply matched the dry run exactly. Smoke checks pass.

Split not applied. The dry run would move 101,398 links off 12,177 rows:

• 35,346 links on 1,269 rows whose name at least two HF accounts publish under, e.g. Yang Liu (8 accounts), Hao Li (10), Lei Zhang (5);
• 66,052 links on rows with the only HF account of that name. These mix conflated common names (Jing Zhang) with distinctive prolific authors whose papers would be wrongly stripped (Philip Torr 118, Shuicheng Yan 129, Yann LeCun 18).

Awaiting a decision on the split scope.

[NielsRogge]

Split applied in production (2026-11-03 18:21 UTC, commit 8391823, phases split,empty; backup reports/authors-backup-20261103T181633Z.dump)

The split now moves only bylines that two or more HF accounts publish under. A name with a single account stays, since those may be that person's own unclaimed papers.

• 35,346 links moved off 1,269 shared-name rows onto unattributed name-only rows (+46 duplicates removed)
• 66,053 links on single-account names left alone: Yann LeCun, Philip Torr, Shuicheng Yan, Chen Change Loy and Ziwei Liu are unchanged
• 93,298 links kept because they share a coauthor with the account's claimed papers

Example: Yang Liu (jacklanda) goes from 595 papers to 44 (14 claimed + 30 coauthor-backed). The other 584 are on an unattributed "Yang Liu" author. Smoke checks pass.

Every phase of this PR has now been applied. It is a one-off backfill CLI, so merging it changes no running service.