Skip to content

fix: don't mark headline backfill done when no sources resolve#3991

Closed
capJavert wants to merge 1 commit into
mainfrom
fix/headline-backfill-empty-no-mark
Closed

fix: don't mark headline backfill done when no sources resolve#3991
capJavert wants to merge 1 commit into
mainfrom
fix/headline-backfill-empty-no-mark

Conversation

@capJavert

@capJavert capJavert commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Problem

dailyHeadlines can race onboarding. We observed a production case where the query resolved before the user's onboarding tag preferences were persisted (the backfill mark landed under a second before the first tag write).

So resolveHeadlineChannelSourcesForUser saw zero tags, returned [], and seedHeadlineChannelsForUser hit the !sourceIds.length branch which called markBackfilled. Every subsequent call short-circuits on alreadyBackfilled, so the user never gets their channel digest follows — even though their tags resolve to matching digest sources afterwards.

Fix

Don't mark the user as backfilled when the resolved source list is empty (no tags yet, or no channel clears the min-posts threshold). The resolve query is cheap, so retrying on later calls is fine. The user still gets marked once seeding actually happens (or when they already have a digest preference).

Tests

  • Updated the two existing "mark backfilled without seeding" tests to assert no user_action row is written.
  • Added a regression test for the race: first call with no tags doesn't lock the user; a second call after tags exist seeds the follows and marks backfilled.

A companion apps PR gates the dailyHeadlines fetch behind isOnboardingComplete so the query doesn't fire mid-onboarding in the first place.

@pulumi

pulumi Bot commented Jul 8, 2026

Copy link
Copy Markdown

🍹 The Update (preview) for dailydotdev/api/prod (at ac511e4) was successful.

Resource Changes

    Name                                                       Type                           Operation
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-materialize-yearly-best-post-archives-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-subscription-anniversary-achievements-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-achievements-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-highlights-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                                      kubernetes:apps/v1:Deployment  update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob    update
+   vpc-native-api-clickhouse-migration-f5683cd4               kubernetes:batch/v1:Job        create
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-user-posts-analytics-refresh-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-gifted-plus-cron                          kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-generate-search-invites-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-public-threshold-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-check-analytics-report-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tag-materialized-views-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-history-day-clickhouse-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment  update
~   vpc-native-rotate-daily-quests-cron                        kubernetes:batch/v1:CronJob    update
-   vpc-native-api-db-migration-fd115e90                       kubernetes:batch/v1:Job        delete
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment  update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-old-notifications-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-bg-deployment                                   kubernetes:apps/v1:Deployment  update
... and 15 other changes

@capJavert capJavert force-pushed the fix/headline-backfill-empty-no-mark branch from 978533f to 143c8b8 Compare July 8, 2026 10:20
dailyHeadlines can race onboarding: when it runs before the user's
onboarding tags are saved, resolveHeadlineChannelSourcesForUser returns
an empty list and the user was permanently marked as backfilled without
ever getting their channel digest follows.

Since the resolve query is cheap, skip markBackfilled on an empty
result so a later call can retry once tags exist (or once channels
clear the minimum post threshold).
@capJavert capJavert force-pushed the fix/headline-backfill-empty-no-mark branch from 143c8b8 to ac511e4 Compare July 8, 2026 10:23
@capJavert

Copy link
Copy Markdown
Contributor Author

Not needed for now

@capJavert capJavert closed this Jul 8, 2026
@capJavert capJavert deleted the fix/headline-backfill-empty-no-mark branch July 8, 2026 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant