Politics in Social Media Activism: A Case Study of Netizens' Reactions to Gibran's Demographic Bonus Video
Moh. Maskurudin Hafid & Odri Prince Agustinus D. Sembiring
Universitas Gadjah Mada, Indonesia
Published in Journal of Local Government and Administration Research (JLARG)
DOI: 10.57185/jlarg.v3i7.124
The published paper is attached in /paper. It is the version of record — peer-reviewed and fully citable.
At the time of publication, the quantitative side of this study was produced under significant methodological constraints. The API-based LLM analysis was our best attempt given the tools and statistical skills available then.
Since then, through the mentorship of Arya Budi and hands-on work with R and Claude Code, we have been able to revisit the same dataset and produce substantially better inferential statistics and data visualisations. The scripts and figures in this repository represent that revamped analysis — same data, sharper lens.
This study examines the dynamics of digital politics through a case study of netizens' reactions to Gibran Rakabuming Raka's "Demographic Bonus" YouTube speech. The comment section is treated as a discursive arena where netizens express symbolic resistance to political elite narratives — a form of digital civil society that creates affective solidarity and micro-political participation within Indonesia's democratic landscape.
Using a mixed methods approach, we combine:
- Qualitative critical discourse analysis (substantive criticism, sarcasm, moral delegitimisation)
- Quantitative LLM-based analysis via three APIs — OpenAI, Grok, and Gemini — on 44,249 comments collected through Python web scraping
Dimensions analysed: sentiment, sentiment objects, emotion categories, writing styles, cognitive engagement levels, and potential bot engagement.
All three APIs significantly underperform against a manually validated gold standard of 1,992 hand-labelled comments — with Cohen's Kappa scores of only 0.07–0.11 (near-chance agreement). The central reason: LLMs currently struggle to capture sarcasm and irony, which are the dominant modes of political resistance in Indonesian digital discourse.
cyberactivism · political legitimacy · digital civil society · elite-people polarization · Large Language Models (LLMs)
api-nlp-gibran-study/
├── paper/ # Published PDF (version of record)
├── R/ # Revamped analysis scripts
│ ├── analysis_01_stylistic_mode.R — Writing style × sentiment breakdown
│ ├── analysis_02_emotion.R — Emotion categories (Disgust, Anger, Joy...)
│ ├── analysis_03_position.R — Position toward Gibran (Favorable/Unfavorable)
│ ├── analysis_04_cognitive.R — Cognitive engagement levels
│ ├── audit_analysis.R — Accuracy, Cohen's Kappa, confusion matrices
│ └── audit_visualise.R — Audit visualisations
├── python/
│ └── audit_analysis.py — Python version of the audit pipeline
└── visuals/
├── dashboards/ — Full combined dashboard PNGs per analysis
└── individual/ — Individual highlight plots
| API | N Evaluated | Accuracy | Cohen's Kappa |
|---|---|---|---|
| OpenAI | 1,992 | 48.9% | 0.075 |
| Grok | 1,975 | 64.1% | 0.111 |
| Gemini | 1,940 | 51.0% | 0.075 |
Ground truth class distribution: 95.7% Negative · 2.8% Positive · 1.5% Neutral
A naive classifier that always guesses "Negative" would score ~95% accuracy — which is precisely why raw accuracy is meaningless here and Kappa is the right metric. All three APIs sit at slight agreement, well below the 0.41 threshold for moderate reliability.
- Open any script in RStudio
Session → Set Working Directory → To Source File Location- Update the
BASEpath at the top of each script to point to your local data folder Cmd+Shift+Enterto run
Required packages (auto-installed by each script): ggplot2, dplyr, tidyr, patchwork, scales, forcats, caret
Hafid, M. M., & Sembiring, O. P. A. D. (2024). Politics in Social Media Activism:
A Case Study of Netizens' Reactions to Gibran's Demographic Bonus Video.
Journal of Local Government and Administration Research, 3(7).
https://doi.org/10.57185/jlarg.v3i7.124
The revamped statistical analysis in this repository was produced using R and Claude Code, building on skills developed under the guidance of Arya Budi.