Skip to content

amarkosmarkos/media_objectivity_analysis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Media Bias Analyzer — Prensa Española

Full-stack application that scrapes the 10 major Spanish newspapers, groups headlines covering the same event via semantic clustering, and quantifies media bias using three academically validated NLP techniques. Results are displayed through an interactive dark-themed visualizer.

Zero local dependencies. Everything runs inside Docker.


Quick start

# 1. Clone and configure
cp .env.example .env          # fill in your Azure OpenAI credentials

# 2. Start the stack (PostgreSQL + backend + frontend)
docker compose up --build -d

# 3. Populate the database (first time — takes ~5 min)
docker compose exec backend python -m app.cli populate

# 4. Open the app
#    Frontend  → http://localhost
#    API       → http://localhost:8000/api/health

How it works

RSS feeds (24 feeds, 10 newspapers)
        │
        ▼
   ┌─────────┐     ┌────────────────────────┐     ┌───────────────┐
   │ Scraper  │────▶│ sentence-transformers  │────▶│   HDBSCAN     │
   │ (httpx + │     │ paraphrase-multilingual │     │  clustering   │
   │ feedpar) │     │ -mpnet-base-v2          │     │  + coherence  │
   └─────────┘     └────────────────────────┘     └───────┬───────┘
                                                          │
                    ┌─────────────────────────────────────┘
                    ▼
   ┌──────────────────────────────────────────────────────────┐
   │              Bias analysis (per cluster)                  │
   │                                                          │
   │  1. Target-dependent Sentiment (pysentimiento + spaCy)   │
   │  2. Word Choice & Labeling (charged lexicon + Jaccard)   │
   │  3. Coverage Gap (which papers ignored the story)        │
   │                                                          │
   │  4. Political beneficiary → Azure OpenAI GPT-4o          │
   └──────────────────────────────────────────────────────────┘
        │
        ▼
   PostgreSQL (persistent, all history preserved)
        │
        ▼
   FastAPI  ──▶  React frontend

Step 1 — Scraping

24 RSS feeds across 10 newspapers, scraped concurrently with httpx + feedparser. Headlines are deduplicated by URL both in-memory and in PostgreSQL (ON CONFLICT DO NOTHING).

Newspaper Feeds Typical volume
El País 1 (portada) ~140
El Mundo 4 (portada, españa, economía, internacional) ~160
ABC 5 (portada, última hora, economía, internacional, opinión) ~80
La Vanguardia 1 (home) ~140
El Confidencial 4 (españa, economía, mundo, sociedad) ~57
La Razón 5 (españa, economía, internacional, sociedad, opinión) ~75
elDiario.es 1 ~100
InfoLibre 1 ~70
OK Diario 1 ~50
20 Minutos 1 ~210

Step 2 — Embeddings

Each headline is embedded using paraphrase-multilingual-mpnet-base-v2 (768-dim, L2-normalized). Embeddings are persisted in the headlines table as JSONB — only new headlines without an existing embedding are processed.

Step 3 — Clustering

HDBSCAN groups headlines from the last 48 hours into clusters representing the same news event. Three filters are applied:

  1. min_cluster_size=3, min_samples=2 (conservative to avoid loose groupings)
  2. Clusters with a single newspaper are discarded (nothing to compare)
  3. Intra-cluster coherence (average pairwise cosine similarity) must exceed 0.45

Step 4 — Bias analysis

Three techniques applied per cluster:

Target-dependent Sentiment Analysis — spaCy (es_core_news_lg) detects political entities (NER + curated list of 25 Spanish political actors). pysentimiento (Spanish-native model) measures sentiment in [-1, +1] toward each entity. Result: {newspaper: {party: score}}.

Word Choice & Labeling (WCL) — Compares vocabulary across newspapers for the same event. A lexicon of ~60 emotionally charged Spanish words detects loaded language. Framing diversity is quantified via average pairwise Jaccard distance between headline token sets.

Coverage Gap — Identifies which of the 10 newspapers did NOT cover the event. Systematic omission patterns are tracked across clusters to detect bias by omission.

Step 5 — Political beneficiary (GPT-4o)

Azure OpenAI GPT-4o receives the headlines and sentiment scores for each cluster and determines:

  1. Which political actor benefits most from the aggregate coverage
  2. Which newspaper shows the strongest favorable bias
  3. Step-by-step reasoning (always exposed in the UI for transparency)

Tech stack

Layer Technologies
Frontend React 19, Vite 6, TailwindCSS 4, Framer Motion, Recharts, React Query
Backend Python 3.11, FastAPI, Pydantic v2
Database PostgreSQL 16, SQLAlchemy 2.0 (async), asyncpg
NLP spaCy es_core_news_lg, sentence-transformers, pysentimiento, HDBSCAN
LLM Azure OpenAI GPT-4o
Infra Docker Compose (local), Railway (production)

CLI reference

All commands run inside the backend container:

docker compose exec backend python -m app.cli <command>
Command Description
populate Full pipeline: scrape → embed → cluster → analyze → store
recluster Re-cluster and re-analyze existing headlines (no scraping)
recluster --hours 72 Same, but use headlines from a custom time window
create-tables Create database tables (idempotent)
backup -o file.sql pg_dump the entire database to a file
backup Dump to stdout (pipe wherever you want)

Experimenting with parameters

Headlines and embeddings persist across runs. To test different clustering settings:

# 1. Edit clustering.py (min_cluster_size, coherence_threshold, etc.)
# 2. Rebuild and re-cluster — no re-scraping, no re-embedding
docker compose up backend --build -d
docker compose exec backend python -m app.cli recluster

Each execution creates a new pipeline_run. The API always serves the latest completed run; previous runs remain in the DB for direct SQL analysis.


API endpoints

Method Route Description
GET /api/clusters All clusters from the latest pipeline run
GET /api/clusters/{id} Single cluster with full bias analysis
GET /api/newspapers Aggregated stats per newspaper
GET /api/bias-map Scatter plot data (ideology x bias intensity)
GET /api/party-media-matrix Sentiment heatmap: parties x newspapers
POST /api/refresh Trigger pipeline (requires X-API-Key header)
GET /api/health Service status, cluster count, last update timestamp

Database schema

pipeline_runs
  id              SERIAL PRIMARY KEY
  started_at      TIMESTAMPTZ
  finished_at     TIMESTAMPTZ
  headlines_scraped  INT
  headlines_new      INT
  clusters_created   INT
  status          VARCHAR(20)      -- running | completed | failed | empty

headlines
  id              VARCHAR(12) PRIMARY KEY
  newspaper       VARCHAR(50) INDEXED
  title           TEXT
  url             TEXT UNIQUE      -- deduplication key
  published_at    TIMESTAMPTZ INDEXED
  embedding       JSONB            -- 768-dim float vector
  scraped_at      TIMESTAMPTZ

clusters
  id              VARCHAR(12) PRIMARY KEY
  topic_summary   TEXT             -- GPT-4o generated
  pipeline_run_id INT → pipeline_runs(id)
  created_at      TIMESTAMPTZ

cluster_headlines                  -- M2M join table
  cluster_id      → clusters(id)
  headline_id     → headlines(id)

bias_analyses
  id              SERIAL PRIMARY KEY
  cluster_id      → clusters(id) UNIQUE
  entity_sentiment         JSONB  -- {newspaper: {party: score}}
  beneficiary_entity       VARCHAR(100)
  beneficiary_confidence   FLOAT
  beneficiary_reasoning    TEXT   -- LLM chain-of-thought
  beneficiary_sentiment_scores  JSONB
  coverage_gap             JSONB  -- [newspaper names]
  framing_diversity        FLOAT  -- 0..1
  wcl_highlights           JSONB  -- {newspaper: [charged_words]}

Nightly job

The pipeline runs as a cron job, not as an in-process scheduler.

Railway

  1. Deploy the backend as the web service
  2. Create a second service from the same repo as a Cron Job
  3. Configure:
    • Root directory: backend
    • Start command: python -m app.cli populate
    • Schedule: 0 3 * * * (3:00 AM UTC daily)
  4. Share the same environment variables (Railway injects DATABASE_URL from the Postgres addon automatically via service references)

Local

docker compose exec backend python -m app.cli populate

Backups

Railway PostgreSQL does not provide automatic backups on free/starter plans.

# Create a backup
docker compose exec backend python -m app.cli backup -o /app/backup.sql
docker compose cp backend:/app/backup.sql ./backup.sql

# Restore a backup
docker compose exec -T db psql -U mediabias mediabias < backup.sql

What survives without a backup: A fresh populate captures whatever is currently in the RSS feeds (~last 24h). What doesn't: all historical headlines, embeddings, clusters, and analyses older than the RSS window. Backups are critical.


Production deployment (Railway)

Services

# Service Type Root dir Start command
1 PostgreSQL Addon
2 Backend (API) Web backend uvicorn app.main:app --host 0.0.0.0 --port $PORT
3 Pipeline (nightly) Cron backend python -m app.cli populate
4 Frontend Web frontend Nginx (via Dockerfile)

Environment variables

DATABASE_URL              ← auto-injected by Railway Postgres addon
AZURE_OPENAI_ENDPOINT     ← base URL only: https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY      ← from Azure Portal → Keys and Endpoint
AZURE_OPENAI_DEPLOYMENT   ← gpt-4o
AZURE_OPENAI_API_VERSION  ← 2025-01-01-preview
REFRESH_API_KEY           ← your own secret for POST /api/refresh

Project structure

.
├── backend/
│   ├── app/
│   │   ├── main.py              # FastAPI application + endpoints
│   │   ├── cli.py               # CLI: populate, recluster, backup, create-tables
│   │   ├── pipeline.py          # Orchestration: scrape → embed → cluster → analyze
│   │   ├── scraper.py           # RSS feed scraping (httpx + feedparser)
│   │   ├── embeddings.py        # sentence-transformers embedding generation
│   │   ├── clustering.py        # HDBSCAN clustering + coherence filtering
│   │   ├── bias_analyzer.py     # 3 bias techniques (sentiment, WCL, coverage gap)
│   │   ├── llm_client.py        # Azure OpenAI GPT-4o (summarize + beneficiary)
│   │   ├── models.py            # Pydantic schemas, RSS config, political entities
│   │   ├── database.py          # SQLAlchemy async engine + session factory
│   │   ├── db_models.py         # ORM table definitions
│   │   └── store.py             # Data access layer (all DB reads/writes)
│   ├── Dockerfile
│   ├── Procfile                 # Railway web process
│   ├── railway.toml             # Railway build/deploy config
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── pages/
│   │   │   ├── Home.jsx         # Cluster feed (main view)
│   │   │   ├── Cluster.jsx      # Cluster detail with bias breakdown
│   │   │   └── Newspapers.jsx   # Newspaper stats + bias map
│   │   ├── components/
│   │   │   ├── ClusterCard.jsx  # Event card for the feed
│   │   │   ├── ClusterDetail.jsx # Headlines with WCL highlights
│   │   │   ├── BiasMap.jsx      # Scatter plot (ideology × bias intensity)
│   │   │   ├── SentimentBar.jsx # Horizontal bar chart (sentiment × entity)
│   │   │   ├── FramingScore.jsx # Gauge visualization (0–100)
│   │   │   ├── PartyMediaMatrix.jsx # Heatmap (parties × newspapers)
│   │   │   └── MethodologyNote.jsx  # Expandable methodology explainers
│   │   ├── lib/
│   │   │   ├── api.js           # API client (React Query)
│   │   │   └── constants.js     # Colors, newspaper palette
│   │   ├── App.jsx              # Router + layout
│   │   └── main.jsx             # Entry point
│   ├── Dockerfile               # Multi-stage: build + Nginx
│   ├── nginx.conf
│   └── package.json
├── docker-compose.yml           # PostgreSQL + backend + frontend
├── .env.example
└── README.md

Design decisions

  • Ideological position is computed, never assigned. Each newspaper's position on the left-right spectrum is derived from its aggregate sentiment toward political parties, weighted by party ideology scores. No manual labeling.
  • Headlines accumulate forever. The headlines table is append-only (upsert by URL). Clustering operates on a sliding 48h window, but the full corpus remains available for analysis.
  • Pipeline runs are immutable. Each populate or recluster creates a new pipeline_run. The API serves the latest; old runs stay in the DB for comparison via SQL.
  • Spanish-native NLP. pysentimiento is trained on Spanish text — no translation step. spaCy es_core_news_lg provides entity recognition tuned for Spanish political discourse.
  • LLM transparency. GPT-4o's reasoning is stored verbatim and always shown in the UI. The model never assigns scores — it interprets the NLP-generated scores.
  • HDBSCAN over K-means. No need to predefine K. Handles variable-size clusters and naturally produces a noise label for headlines that don't belong to any event.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors