Reduce Apify to Facebook trail status only#559
Conversation
Remove Apify from the news collection pipeline (renderPage.js) and strip Instagram support. Only Reagan-Huffman MTB trail status still uses Apify's Facebook scraper via trailStatusService. Saves ~$25/month by eliminating Facebook Posts Scraper and Instagram Scraper usage from the daily 19-POI news collection job. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request removes Instagram scraping support and simplifies the Apify service to focus solely on Facebook trail status scraping. This involves removing Instagram-related constants, helper functions, tests, and the associated admin setting, as well as removing the Apify integration from the page rendering service. The review feedback recommends adding defensive checks in fetchFacebookPosts to prevent potential TypeError crashes: specifically, verifying that statusUrl is a string before processing, and ensuring that the result from runActorSync is an array before calling .map() on it.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| const target = extractFacebookPageUrl(statusUrl); | ||
| if (!target) { | ||
| console.log(`[Apify] Could not extract Facebook page from: ${statusUrl}`); | ||
| return { markdown: null, reachable: false, reason: 'invalid Facebook URL' }; | ||
| } |
There was a problem hiding this comment.
Defensive check: statusUrl is passed to extractFacebookPageUrl(statusUrl) which calls url.match(...). If statusUrl is null, undefined, or not a string, this will throw an unhandled TypeError because it is outside the try-catch block. Adding a type check ensures the function fails gracefully.
if (typeof statusUrl !== 'string') {
console.log(`[Apify] Invalid statusUrl: ${statusUrl}`);
return { markdown: null, reachable: false, reason: 'invalid Facebook URL' };
}
const target = extractFacebookPageUrl(statusUrl);
if (!target) {
console.log(`[Apify] Could not extract Facebook page from: ${statusUrl}`);
return { markdown: null, reachable: false, reason: 'invalid Facebook URL' };
}| const items = await runActorSync(FACEBOOK_ACTOR_ID, { startUrls: [{ url: target }], maxPosts: maxItems }, token); | ||
| if (!items || items.length === 0) { | ||
| console.log(`[Apify] No posts found for ${target}`); | ||
| return { markdown: null, rawText: null, ogDates: {}, ogImage: null, links: [], reachable: true, reason: 'no posts found' }; | ||
| return { markdown: null, reachable: true, reason: 'no posts found' }; | ||
| } |
There was a problem hiding this comment.
Defensive check: runActorSync returns the parsed JSON response. If the API returns a non-array JSON object (e.g., an error payload or metadata), calling items.map will throw a TypeError. Checking Array.isArray(items) prevents this potential crash.
| const items = await runActorSync(FACEBOOK_ACTOR_ID, { startUrls: [{ url: target }], maxPosts: maxItems }, token); | |
| if (!items || items.length === 0) { | |
| console.log(`[Apify] No posts found for ${target}`); | |
| return { markdown: null, rawText: null, ogDates: {}, ogImage: null, links: [], reachable: true, reason: 'no posts found' }; | |
| return { markdown: null, reachable: true, reason: 'no posts found' }; | |
| } | |
| const items = await runActorSync(FACEBOOK_ACTOR_ID, { startUrls: [{ url: target }], maxPosts: maxItems }, token); | |
| if (!Array.isArray(items) || items.length === 0) { | |
| console.log(`[Apify] No posts found or invalid response format for ${target}`); | |
| return { markdown: null, reachable: true, reason: 'no posts found' }; | |
| } |
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
renderPage.js) — Facebook/Instagram URLs from Serper results now fall through to Playwright/Readability or snippet recovery like any other URLisSocialUrl,fetchSocialPostsfromapifyService.jssocial_apify_collection_enabledadmin settingWhat stays
fetchFacebookPosts+isFacebookUrl— used bytrailStatusService.jsfor Reagan-Huffman MTB (POI 5680, medinaTRAILS Facebook page)apify_api_tokenadmin setting + test endpointImpact
Closes #551 (partially — Apify retained for trail status)
🤖 Generated with Claude Code