Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/pwc-cron-health/references/job-probes.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ For every managed leaderboard sync, fetch the current upstream source read-only
- **Effect:** Hourly at :35, adds an `x_post_metrics` row for every `paper_x_threads` post from the last 7 days, then reads each due post (about 1, 2, 4, 8, 12, 24, 48, 72, 120 and 144 h after posting) and updates its likes-per-hour EMA (`like_velocity`). Syndication is tried first; posts it cannot serve use the paid X API (`X_API_BEARER_TOKEN`), capped at `PWC_X_LIKES_API_DAILY_CAP` (default 300) posts per UTC day, tracked in `x_post_likes_api_usage.json` under `PWC_CRON_LOCK_DIR`. Posts the API reports unavailable get `gone_at`. Report-only (reads through syndication and rolls back) unless `PWC_X_LIKES_APPLY=true`.
- **Freshness metrics** (dashboard Database Metrics): `last_write` = latest `x_post_metrics.last_read_at`, `tracked_posts` = posts with a pending reading. Report-only runs write nothing, so both stay empty until apply is on.
- **Independent proof:** Pick a recently read post (`ORDER BY last_read_at DESC`), open it on X, and confirm its like count is close to `like_count`; then confirm the paper's `x_trending_component` is set after the next `calculate_paper_trending_scores` run.
- **Correctness:** The JSON summary lists `syndication_reads`, `api_reads`, `api_reads_today`, `unread` and `api_error`. `api_error` with HTTP 401/402 means a bad token or exhausted X credits and exits 1; many `unread` with `api_reads_today` at the cap means syndication is failing and the cap is doing its job. The X part only affects rankings when `PWC_TRENDING_X_WEIGHT` in `/etc/paperswithcode/trending-cron.env` is above 0.
- **Correctness:** The JSON summary lists `syndication_reads`, `api_reads`, `api_reads_today`, `unread` and `api_error`. `api_error` with HTTP 401/402 means a bad token or exhausted X credits and exits 1; many `unread` with `api_reads_today` at the cap means syndication is failing and the cap is doing its job. The X part only affects rankings when `PWC_TRENDING_X_WEIGHT` in `/etc/paperswithcode/trending-cron.env` is above 0, and then it can only lift a paper: the score is the higher of the GitHub/HF score and 100 × weight × `x_trending_component`.

## `link_google_scholar_profiles`

Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,9 +193,10 @@ fall back to the paid X API (`X_API_BEARER_TOKEN`) for at most
reading through syndication and rolling back, unless `PWC_X_LIKES_APPLY=true`.
`calculate_paper_trending_scores` ranks each paper's fastest post from the last
week against the others (posts under `PWC_TRENDING_X_MIN_LIKES_PER_HOUR`,
default 1, count for nothing) and weighs it by `PWC_TRENDING_X_WEIGHT` from
`/etc/paperswithcode/trending-cron.env` (default 0; GitHub and Hugging Face
share the rest equally).
default 1, count for nothing). The X rank only lifts a paper: the score is the
higher of the GitHub/Hugging Face score and the X rank scaled by
`PWC_TRENDING_X_WEIGHT` from `/etc/paperswithcode/trending-cron.env` (default 0
= off, 1 = full lift).

```bash
uv run --env-file backend/keys.env python -m backend.scripts.refresh_x_post_likes --max-posts 20
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

Computes a paper-level trending score that blends the existing GitHub star
velocity EMA (``repositories.trending_score``, stars/hour) with Hugging Face
artifact Hub trending scores (``hf_artifacts.trending_score``) and, when
artifact Hub trending scores (``hf_artifacts.trending_score``). When
``PWC_TRENDING_X_WEIGHT`` is above 0, the like velocity of the paper's fastest
linked X post (``x_post_metrics.like_velocity``). The formula lives in
linked X post (``x_post_metrics.like_velocity``) can lift that score. The formula lives in
``backend/utils/trending.py`` so it is unit-testable without a DB.

Pipeline:
Expand Down Expand Up @@ -74,8 +74,8 @@ def log(message: str):
def x_settings() -> tuple[float, float]:
"""Return the configured ``(x_weight, x_min_likes_per_hour)``."""
weight = float(os.getenv("PWC_TRENDING_X_WEIGHT") or DEFAULT_X_WEIGHT)
if not 0.0 <= weight < 1.0:
raise ValueError("PWC_TRENDING_X_WEIGHT must be in [0, 1)")
if not 0.0 <= weight <= 1.0:
raise ValueError("PWC_TRENDING_X_WEIGHT must be in [0, 1]")
min_likes = float(
os.getenv("PWC_TRENDING_X_MIN_LIKES_PER_HOUR") or DEFAULT_X_MIN_LIKES_PER_HOUR
)
Expand Down
13 changes: 8 additions & 5 deletions backend/tests/paper/test_trending_scores.py
Original file line number Diff line number Diff line change
Expand Up @@ -210,20 +210,23 @@ def test_x_only_paper_stays_unscored_while_x_weight_is_zero():
assert clears == [2]

updates, clears = cron.build_score_changes(
population, github_corpus, hf_corpora, x_sorted=[50.0], x_weight=1 / 3
population, github_corpus, hf_corpora, x_sorted=[50.0], x_weight=1.0
)
by_id = {update["id"]: update for update in updates}
assert by_id[2]["x"] == 1.0
assert by_id[2]["score"] == pytest.approx(100 / 3, abs=1e-3)
assert by_id[1]["score"] == pytest.approx(100 / 3, abs=1e-3)
assert by_id[2]["score"] == pytest.approx(100, abs=1e-3)
# The GitHub-only paper keeps its score: X only lifts.
assert by_id[1]["score"] == pytest.approx(50, abs=1e-3)
assert clears == []


def test_x_settings_reject_a_weight_that_would_silence_github_and_hf(monkeypatch):
def test_x_settings_accept_a_full_lift_and_reject_out_of_range(monkeypatch):
cron = _load_module()
monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "1")
monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "1.5")
with pytest.raises(ValueError, match="PWC_TRENDING_X_WEIGHT"):
cron.x_settings()
monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "1")
assert cron.x_settings() == (1.0, 1.0)
monkeypatch.setenv("PWC_TRENDING_X_WEIGHT", "0.3333")
monkeypatch.setenv("PWC_TRENDING_X_MIN_LIKES_PER_HOUR", "2")
assert cron.x_settings() == (0.3333, 2.0)
25 changes: 17 additions & 8 deletions backend/tests/test_trending_formula.py
Original file line number Diff line number Diff line change
Expand Up @@ -172,14 +172,23 @@ def test_hf_aggregators_are_excluded_by_fan_out_and_denylist():
assert not is_hf_aggregator("model", "tardellirs/paper-pulse", 1)


def test_x_weight_takes_its_share_equally_from_github_and_hf():
assert composite_score(1.0, 1.0, 1.0, x_weight=1 / 3) == pytest.approx(SCORE_SCALE)
assert composite_score(1.0, 0.0, 0.0, x_weight=1 / 3) == pytest.approx(
SCORE_SCALE / 3
def test_x_only_lifts_the_github_hf_score():
# A paper without X data keeps its GitHub/HF score.
assert composite_score(1.0, 0.0, 0.0, x_weight=1.0) == pytest.approx(
SCORE_SCALE * GITHUB_WEIGHT
)
assert composite_score(0.0, 0.0, 1.0, x_weight=1 / 3) == pytest.approx(
SCORE_SCALE / 3
# A faster X rank lifts the paper, a slower one never lowers it.
assert composite_score(0.2, 0.2, 0.9, x_weight=1.0) == pytest.approx(
SCORE_SCALE * 0.9
)
assert composite_score(0.8, 0.8, 0.5, x_weight=1.0) == pytest.approx(
SCORE_SCALE * 0.8
)
# The weight scales the lift; a viral post alone can reach the top.
assert composite_score(0.0, 0.0, 1.0, x_weight=0.5) == pytest.approx(
SCORE_SCALE / 2
)
assert composite_score(0.0, 0.0, 1.0, x_weight=1.0) == pytest.approx(SCORE_SCALE)
# Weighted at 0, X never changes a score.
assert composite_score(0.4, 0.6, 1.0) == composite_score(0.4, 0.6)

Expand All @@ -192,7 +201,7 @@ def test_slow_x_posts_count_for_nothing_and_stay_out_of_the_corpus():
assert x_component(40.0, corpus, 1.0) == 1.0

score, g, h, x = compute_paper_score(
0.0, [], [], {}, x_raw=40.0, x_sorted=corpus, x_weight=1 / 3
0.0, [], [], {}, x_raw=40.0, x_sorted=corpus, x_weight=1.0
)
assert (g, h, x) == (0.0, 0.0, 1.0)
assert score == pytest.approx(SCORE_SCALE / 3)
assert score == pytest.approx(SCORE_SCALE)
23 changes: 10 additions & 13 deletions backend/utils/trending.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,13 @@

X = quantile_rank(fastest linked X post likes/hour, all live post velocities)

paper_trending_score = SCORE_SCALE * (w_g * G + w_h * H + w_x * X)
paper_trending_score = SCORE_SCALE * max(w_g * G + w_h * H, w_x * X)

The X weight ``w_x`` is configurable (``PWC_TRENDING_X_WEIGHT``, default 0) and
takes its share equally from GitHub and HF, so ``w_g = w_h = (1 - w_x) / 2`` and
the weights always sum to 1. Posts slower than
X only lifts: few papers have a linked post, so blending it in linearly would
cap every paper without one below the X-tracked papers. As a lift, a paper
without X data keeps its GitHub/HF score, and a viral post can carry a paper to
the top on its own. The X weight ``w_x`` scales that lift
(``PWC_TRENDING_X_WEIGHT`` in [0, 1], default 0 = off, 1 = full). Posts slower than
``PWC_TRENDING_X_MIN_LIKES_PER_HOUR`` (default 1) count for nothing and stay out
of the X corpus: the corpus is small and pre-selected for notable papers, so a
slow post would otherwise rank mid-table.
Expand Down Expand Up @@ -51,13 +53,12 @@
import bisect
from typing import Dict, Iterable, List, Sequence, Tuple

# Blend weights between the GitHub and Hugging Face components, both scaled by
# (1 - X weight). Equal weights so the top of each signal reaches the same
# Blend weights between the GitHub and Hugging Face components. Equal weights so the top of each signal reaches the same
# single-signal ceiling.
GITHUB_WEIGHT = 0.5
HF_WEIGHT = 0.5

# Default X weight (off) and the slowest X post that still counts, in likes/hour.
# Default X lift weight (off) and the slowest X post that still counts, in likes/hour.
DEFAULT_X_WEIGHT = 0.0
DEFAULT_X_MIN_LIKES_PER_HOUR = 1.0
# Only posts from the last week count; refresh_x_post_likes stops reading them.
Expand Down Expand Up @@ -237,12 +238,8 @@ def composite_score(
x_weight: float = DEFAULT_X_WEIGHT,
) -> float:
"""Combine the normalized components into the final paper score."""
rest = 1.0 - x_weight
return SCORE_SCALE * (
GITHUB_WEIGHT * rest * g_component
+ HF_WEIGHT * rest * h_component
+ x_weight * x_component
)
github_hf = GITHUB_WEIGHT * g_component + HF_WEIGHT * h_component
return SCORE_SCALE * max(github_hf, x_weight * x_component)


def compute_paper_score(
Expand Down
Loading