fix(cdc): re-enrich scheduled freeform posts when they become visible#3989
Closed
idoshamun wants to merge 1 commit into
Closed
fix(cdc): re-enrich scheduled freeform posts when they become visible#3989idoshamun wants to merge 1 commit into
idoshamun wants to merge 1 commit into
Conversation
A scheduled freeform post's tags/contentQuality are captured once, when it's created/scheduled, and are never refreshed at publish time — publishScheduledPost only flips visibility. So a post enriched during a broken enrichment window (e.g. the empty-tags bug) goes live broken, and correcting the source afterwards has no effect because nothing re-sends. On the visible false->true transition for a freeform post, emit the same content-requested event used at creation. yggdrasil resolves it to the existing entry (deterministic id), flips it back to processing, and re-enriches against current models, so the post publishes with correct tags/contentQuality. The existing op=u re-request is gated on a content delta, which a scheduled publish never has, so this path was uncovered.
|
🍹 The Update (preview) for dailydotdev/api/prod (at a62a2de) was successful. Resource Changes Name Type Operation
~ vpc-native-temporal-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-update-current-streak-cron kubernetes:batch/v1:CronJob update
+ vpc-native-api-clickhouse-migration-6f93ebd7 kubernetes:batch/v1:Job create
~ vpc-native-generic-referral-reminder-cron kubernetes:batch/v1:CronJob update
~ vpc-native-rotate-weekly-quests-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-posts-analytics-refresh-cron kubernetes:batch/v1:CronJob update
~ vpc-native-daily-digest-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-views-cron kubernetes:batch/v1:CronJob update
~ vpc-native-bg-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-clean-zombie-users-cron kubernetes:batch/v1:CronJob update
~ vpc-native-materialize-monthly-best-post-archives-cron kubernetes:batch/v1:CronJob update
~ vpc-native-ws-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-worker-job-deployment kubernetes:apps/v1:Deployment update
~ vpc-native-hourly-notification-cron kubernetes:batch/v1:CronJob update
- vpc-native-api-db-migration-8bcbf299 kubernetes:batch/v1:Job delete
- vpc-native-api-clickhouse-migration-8bcbf299 kubernetes:batch/v1:Job delete
~ vpc-native-generate-search-invites-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-tag-materialized-views-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-channel-highlights-cron kubernetes:batch/v1:CronJob update
~ vpc-native-sync-subscription-with-cio-cron kubernetes:batch/v1:CronJob update
~ vpc-native-channel-highlights-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-stale-user-transactions-cron kubernetes:batch/v1:CronJob update
+ vpc-native-api-db-migration-6f93ebd7 kubernetes:batch/v1:Job create
~ vpc-native-post-analytics-history-day-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-post-lifecycle-state-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-gifted-plus-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-images-cron kubernetes:batch/v1:CronJob update
~ vpc-native-materialize-yearly-best-post-archives-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-profile-analytics-history-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-post-analytics-achievements-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-trending-cron kubernetes:batch/v1:CronJob update
~ vpc-native-calculate-top-readers-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-old-notifications-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-profile-analytics-clickhouse-cron kubernetes:batch/v1:CronJob update
~ vpc-native-clean-zombie-user-companies-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-highlighted-views-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-tags-str-cron kubernetes:batch/v1:CronJob update
~ vpc-native-user-profile-updated-sync-cron kubernetes:batch/v1:CronJob update
~ vpc-native-validate-active-users-cron kubernetes:batch/v1:CronJob update
~ vpc-native-check-analytics-report-cron kubernetes:batch/v1:CronJob update
~ vpc-native-update-achievement-rarity-cron kubernetes:batch/v1:CronJob update
... and 15 other changes |
Member
Author
|
Closing — this was defense-in-depth, not a root-cause fix. The two actual root causes are already deployed: bragi #257 (tags were optional in the enrich schema → empty) and daily-api #3982 (contentQuality was missing from freeformAllowedFields → stripped on the freeform update path). With both fixed, new scheduled freeform posts enrich correctly at creation on both dimensions, so re-enriching at publish only adds a redundant enrichment. The affected already-scheduled posts were remediated separately. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Scheduled freeform posts publish with stale or empty enrichment. A post's
tagsStr/contentQualityare populated exactly once — when it's created/scheduled, via the creation-timecontent-requested→ yggdrasil round-trip. AtscheduledAt,publishScheduledPostonly flipsvisible/visibleAt/createdAt/flags; nothing re-enriches. So a post that was enriched during a broken window (e.g. the recent bragi empty-tags bug, or the freeform contentQuality allowlist bug) goes live broken and stays broken.Correcting the source in yggdrasil afterwards doesn't help either: yggdrasil sends its
content-publishedmessage only once (onprocessing → completed), and a later correction is acompleted → completedCDC event that's dropped unlesspublish_on_updateis set. There is no re-send at publish.This was surfaced by real scheduled posts (an "Agentic Tips" series) going live with no tags and empty content quality.
Fix
On the
visible: false → truetransition for a Freeform post (op=u), emit the samenotifyFreeformContentRequestedevent used at creation. yggdrasil resolves it to the existing entry (id is a deterministicNewID(source, url)hash), flips it back toprocessing, and re-enriches against current models (canUpdateFreeformContentEntry→updateFreeformContentEntry). The post then publishes with correcttags/contentQualityvia the normal path.Why the existing op=
ure-request didn't already cover thisThe existing freeform re-request in
onPostChangeis gated onisFreeformPostChangeLongEnough(a content-length delta). A scheduled publish changes visibility, not content, so that gate is never satisfied — the publish transition was uncovered.Notes
content-requestedfor a scheduled post → entry wentprocessing → completed→ post got correct tags/CQ while remaining invisible, then is set to publish normally).!before.visibleis false so this path stays quiet.updatePostruns while the post is already visible (before.visible = true), so the transition guard prevents re-triggering.Test plan
tsc --noEmitclean, eslint clean.