From c534897d951168f29d0bb8c0ea6bff89998f082b Mon Sep 17 00:00:00 2001 From: Niels Rogge Date: Tue, 3 Nov 2026 13:21:55 +0000 Subject: [PATCH] Make X like velocity a lift-only trending signal Few papers have a linked X post (29 of ~2,200 scored papers on 2026-11-03), so the linear blend capped every paper without one at 2/3 of the scale and let X-tracked papers with 3 likes/hour outrank today's top papers. The score is now max(GitHub/HF score, weight x X rank): papers without X keep their score and a viral post can lift a paper to the top. PWC_TRENDING_X_WEIGHT now scales the lift in [0, 1] and stays 0 (off). Co-Authored-By: Claude Opus 5.5 --- .../pwc-cron-health/references/job-probes.md | 2 +- .github/workflows/scripts/README.md | 7 +++--- .../hourly/calculate_paper_trending_scores.py | 8 +++--- backend/tests/paper/test_trending_scores.py | 13 ++++++---- backend/tests/test_trending_formula.py | 25 +++++++++++++------ backend/utils/trending.py | 23 ++++++++--------- 6 files changed, 44 insertions(+), 34 deletions(-) diff --git a/.agents/skills/pwc-cron-health/references/job-probes.md b/.agents/skills/pwc-cron-health/references/job-probes.md index ffc5c355..75877b42 100644 --- a/.agents/skills/pwc-cron-health/references/job-probes.md +++ b/.agents/skills/pwc-cron-health/references/job-probes.md @@ -225,7 +225,7 @@ For every managed leaderboard sync, fetch the current upstream source read-only - **Effect:** Hourly at :35, adds an `x_post_metrics` row for every `paper_x_threads` post from the last 7 days, then reads each due post (about 1, 2, 4, 8, 12, 24, 48, 72, 120 and 144 h after posting) and updates its likes-per-hour EMA (`like_velocity`). Syndication is tried first; posts it cannot serve use the paid X API (`X_API_BEARER_TOKEN`), capped at `PWC_X_LIKES_API_DAILY_CAP` (default 300) posts per UTC day, tracked in `x_post_likes_api_usage.json` under `PWC_CRON_LOCK_DIR`. Posts the API reports unavailable get `gone_at`. Report-only (reads through syndication and rolls back) unless `PWC_X_LIKES_APPLY=true`. - **Freshness metrics** (dashboard Database Metrics): `last_write` = latest `x_post_metrics.last_read_at`, `tracked_posts` = posts with a pending reading. Report-only runs write nothing, so both stay empty until apply is on. - **Independent proof:** Pick a recently read post (`ORDER BY last_read_at DESC`), open it on X, and confirm its like count is close to `like_count`; then confirm the paper's `x_trending_component` is set after the next `calculate_paper_trending_scores` run. -- **Correctness:** The JSON summary lists `syndication_reads`, `api_reads`, `api_reads_today`, `unread` and `api_error`. `api_error` with HTTP 401/402 means a bad token or exhausted X credits and exits 1; many `unread` with `api_reads_today` at the cap means syndication is failing and the cap is doing its job. The X part only affects rankings when `PWC_TRENDING_X_WEIGHT` in `/etc/paperswithcode/trending-cron.env` is above 0. +- **Correctness:** The JSON summary lists `syndication_reads`, `api_reads`, `api_reads_today`, `unread` and `api_error`. `api_error` with HTTP 401/402 means a bad token or exhausted X credits and exits 1; many `unread` with `api_reads_today` at the cap means syndication is failing and the cap is doing its job. The X part only affects rankings when `PWC_TRENDING_X_WEIGHT` in `/etc/paperswithcode/trending-cron.env` is above 0, and then it can only lift a paper: the score is the higher of the GitHub/HF score and 100 × weight × `x_trending_component`. ## `link_google_scholar_profiles` diff --git a/.github/workflows/scripts/README.md b/.github/workflows/scripts/README.md index e4f8ddd6..4d595fdd 100644 --- a/.github/workflows/scripts/README.md +++ b/.github/workflows/scripts/README.md @@ -193,9 +193,10 @@ fall back to the paid X API (`X_API_BEARER_TOKEN`) for at most reading through syndication and rolling back, unless `PWC_X_LIKES_APPLY=true`. `calculate_paper_trending_scores` ranks each paper's fastest post from the last week against the others (posts under `PWC_TRENDING_X_MIN_LIKES_PER_HOUR`, -default 1, count for nothing) and weighs it by `PWC_TRENDING_X_WEIGHT` from -`/etc/paperswithcode/trending-cron.env` (default 0; GitHub and Hugging Face -share the rest equally). +default 1, count for nothing). The X rank only lifts a paper: the score is the +higher of the GitHub/Hugging Face score and the X rank scaled by +`PWC_TRENDING_X_WEIGHT` from `/etc/paperswithcode/trending-cron.env` (default 0 += off, 1 = full lift). ```bash uv run --env-file backend/keys.env python -m backend.scripts.refresh_x_post_likes --max-posts 20 diff --git a/.github/workflows/scripts/hourly/calculate_paper_trending_scores.py b/.github/workflows/scripts/hourly/calculate_paper_trending_scores.py index e7a16b50..5c8a5fd7 100644 --- a/.github/workflows/scripts/hourly/calculate_paper_trending_scores.py +++ b/.github/workflows/scripts/hourly/calculate_paper_trending_scores.py @@ -3,9 +3,9 @@ Computes a paper-level trending score that blends the existing GitHub star velocity EMA (``repositories.trending_score``, stars/hour) with Hugging Face -artifact Hub trending scores (``hf_artifacts.trending_score``) and, when +artifact Hub trending scores (``hf_artifacts.trending_score``). When ``PWC_TRENDING_X_WEIGHT`` is above 0, the like velocity of the paper's fastest -linked X post (``x_post_metrics.like_velocity``). The formula lives in +linked X post (``x_post_metrics.like_velocity``) can lift that score. The formula lives in ``backend/utils/trending.py`` so it is unit-testable without a DB. Pipeline: @@ -74,8 +74,8 @@ def log(message: str): def x_settings() -> tuple[float, float]: """Return the configured ``(x_weight, x_min_likes_per_hour)``.""" weight = float(os.getenv("PWC_TRENDING_X_WEIGHT") or DEFAULT_X_WEIGHT) - if not 0.0 <= weight < 1.0: - raise ValueError("PWC_TRENDING_X_WEIGHT must be in [0, 1)") + if not 0.0 <= weight <= 1.0: + raise ValueError("PWC_TRENDING_X_WEIGHT must be in [0, 1]") min_likes = float( os.getenv("PWC_TRENDING_X_MIN_LIKES_PER_HOUR") or DEFAULT_X_MIN_LIKES_PER_HOUR ) diff --git a/backend/tests/paper/test_trending_scores.py b/backend/tests/paper/test_trending_scores.py index 3dde8bcb..becc3342 100644 --- a/backend/tests/paper/test_trending_scores.py +++ b/backend/tests/paper/test_trending_scores.py @@ -210,20 +210,23 @@ def test_x_only_paper_stays_unscored_while_x_weight_is_zero(): assert clears == [2] updates, clears = cron.build_score_changes( - population, github_corpus, hf_corpora, x_sorted=[50.0], x_weight=1 / 3 + population, github_corpus, hf_corpora, x_sorted=[50.0], x_weight=1.0 ) by_id = {update["id"]: update for update in updates} assert by_id[2]["x"] == 1.0 - assert by_id[2]["score"] == pytest.approx(100 / 3, abs=1e-3) - assert by_id[1]["score"] == pytest.approx(100 / 3, abs=1e-3) + assert by_id[2]["score"] == pytest.approx(100, abs=1e-3) + # The GitHub-only paper keeps its score: X only lifts. + assert by_id[1]["score"] == pytest.approx(50, abs=1e-3) assert clears == [] -def test_x_settings_reject_a_weight_that_would_silence_github_and_hf(monkeypatch): +def test_x_settings_accept_a_full_lift_and_reject_out_of_range(monkeypatch): cron = _load_module() - monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "1") + monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "1.5") with pytest.raises(ValueError, match="PWC_TRENDING_X_WEIGHT"): cron.x_settings() + monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "1") + assert cron.x_settings() == (1.0, 1.0) monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "0.3333") monkeypatch.setenv("PWC_TRENDING_X_MIN_LIKES_PER_HOUR", "2") assert cron.x_settings() == (0.3333, 2.0) diff --git a/backend/tests/test_trending_formula.py b/backend/tests/test_trending_formula.py index c3830d03..34b524ec 100644 --- a/backend/tests/test_trending_formula.py +++ b/backend/tests/test_trending_formula.py @@ -172,14 +172,23 @@ def test_hf_aggregators_are_excluded_by_fan_out_and_denylist(): assert not is_hf_aggregator("model", "tardellirs/paper-pulse", 1) -def test_x_weight_takes_its_share_equally_from_github_and_hf(): - assert composite_score(1.0, 1.0, 1.0, x_weight=1 / 3) == pytest.approx(SCORE_SCALE) - assert composite_score(1.0, 0.0, 0.0, x_weight=1 / 3) == pytest.approx( - SCORE_SCALE / 3 +def test_x_only_lifts_the_github_hf_score(): + # A paper without X data keeps its GitHub/HF score. + assert composite_score(1.0, 0.0, 0.0, x_weight=1.0) == pytest.approx( + SCORE_SCALE * GITHUB_WEIGHT ) - assert composite_score(0.0, 0.0, 1.0, x_weight=1 / 3) == pytest.approx( - SCORE_SCALE / 3 + # A faster X rank lifts the paper, a slower one never lowers it. + assert composite_score(0.2, 0.2, 0.9, x_weight=1.0) == pytest.approx( + SCORE_SCALE * 0.9 ) + assert composite_score(0.8, 0.8, 0.5, x_weight=1.0) == pytest.approx( + SCORE_SCALE * 0.8 + ) + # The weight scales the lift; a viral post alone can reach the top. + assert composite_score(0.0, 0.0, 1.0, x_weight=0.5) == pytest.approx( + SCORE_SCALE / 2 + ) + assert composite_score(0.0, 0.0, 1.0, x_weight=1.0) == pytest.approx(SCORE_SCALE) # Weighted at 0, X never changes a score. assert composite_score(0.4, 0.6, 1.0) == composite_score(0.4, 0.6) @@ -192,7 +201,7 @@ def test_slow_x_posts_count_for_nothing_and_stay_out_of_the_corpus(): assert x_component(40.0, corpus, 1.0) == 1.0 score, g, h, x = compute_paper_score( - 0.0, [], [], {}, x_raw=40.0, x_sorted=corpus, x_weight=1 / 3 + 0.0, [], [], {}, x_raw=40.0, x_sorted=corpus, x_weight=1.0 ) assert (g, h, x) == (0.0, 0.0, 1.0) - assert score == pytest.approx(SCORE_SCALE / 3) + assert score == pytest.approx(SCORE_SCALE) diff --git a/backend/utils/trending.py b/backend/utils/trending.py index 6f6517b1..76aa7d50 100644 --- a/backend/utils/trending.py +++ b/backend/utils/trending.py @@ -19,11 +19,13 @@ X = quantile_rank(fastest linked X post likes/hour, all live post velocities) - paper_trending_score = SCORE_SCALE * (w_g * G + w_h * H + w_x * X) + paper_trending_score = SCORE_SCALE * max(w_g * G + w_h * H, w_x * X) -The X weight ``w_x`` is configurable (``PWC_TRENDING_X_WEIGHT``, default 0) and -takes its share equally from GitHub and HF, so ``w_g = w_h = (1 - w_x) / 2`` and -the weights always sum to 1. Posts slower than +X only lifts: few papers have a linked post, so blending it in linearly would +cap every paper without one below the X-tracked papers. As a lift, a paper +without X data keeps its GitHub/HF score, and a viral post can carry a paper to +the top on its own. The X weight ``w_x`` scales that lift +(``PWC_TRENDING_X_WEIGHT`` in [0, 1], default 0 = off, 1 = full). Posts slower than ``PWC_TRENDING_X_MIN_LIKES_PER_HOUR`` (default 1) count for nothing and stay out of the X corpus: the corpus is small and pre-selected for notable papers, so a slow post would otherwise rank mid-table. @@ -51,13 +53,12 @@ import bisect from typing import Dict, Iterable, List, Sequence, Tuple -# Blend weights between the GitHub and Hugging Face components, both scaled by -# (1 - X weight). Equal weights so the top of each signal reaches the same +# Blend weights between the GitHub and Hugging Face components. Equal weights so the top of each signal reaches the same # single-signal ceiling. GITHUB_WEIGHT = 0.5 HF_WEIGHT = 0.5 -# Default X weight (off) and the slowest X post that still counts, in likes/hour. +# Default X lift weight (off) and the slowest X post that still counts, in likes/hour. DEFAULT_X_WEIGHT = 0.0 DEFAULT_X_MIN_LIKES_PER_HOUR = 1.0 # Only posts from the last week count; refresh_x_post_likes stops reading them. @@ -237,12 +238,8 @@ def composite_score( x_weight: float = DEFAULT_X_WEIGHT, ) -> float: """Combine the normalized components into the final paper score.""" - rest = 1.0 - x_weight - return SCORE_SCALE * ( - GITHUB_WEIGHT * rest * g_component - + HF_WEIGHT * rest * h_component - + x_weight * x_component - ) + github_hf = GITHUB_WEIGHT * g_component + HF_WEIGHT * h_component + return SCORE_SCALE * max(github_hf, x_weight * x_component) def compute_paper_score(