Skip to content

fix(cdc): re-enrich scheduled freeform posts when they become visible#3989

Closed
idoshamun wants to merge 1 commit into
mainfrom
fix/reenrich-scheduled-freeform-on-publish
Closed

fix(cdc): re-enrich scheduled freeform posts when they become visible#3989
idoshamun wants to merge 1 commit into
mainfrom
fix/reenrich-scheduled-freeform-on-publish

Conversation

@idoshamun

Copy link
Copy Markdown
Member

Problem

Scheduled freeform posts publish with stale or empty enrichment. A post's tagsStr/contentQuality are populated exactly once — when it's created/scheduled, via the creation-time content-requested → yggdrasil round-trip. At scheduledAt, publishScheduledPost only flips visible/visibleAt/createdAt/flags; nothing re-enriches. So a post that was enriched during a broken window (e.g. the recent bragi empty-tags bug, or the freeform contentQuality allowlist bug) goes live broken and stays broken.

Correcting the source in yggdrasil afterwards doesn't help either: yggdrasil sends its content-published message only once (on processing → completed), and a later correction is a completed → completed CDC event that's dropped unless publish_on_update is set. There is no re-send at publish.

This was surfaced by real scheduled posts (an "Agentic Tips" series) going live with no tags and empty content quality.

Fix

On the visible: false → true transition for a Freeform post (op=u), emit the same notifyFreeformContentRequested event used at creation. yggdrasil resolves it to the existing entry (id is a deterministic NewID(source, url) hash), flips it back to processing, and re-enriches against current models (canUpdateFreeformContentEntryupdateFreeformContentEntry). The post then publishes with correct tags/contentQuality via the normal path.

Why the existing op=u re-request didn't already cover this

The existing freeform re-request in onPostChange is gated on isFreeformPostChangeLongEnough (a content-length delta). A scheduled publish changes visibility, not content, so that gate is never satisfied — the publish transition was uncovered.

Notes

  • Idempotent / no duplicate entry: the deterministic entry id means a re-request updates the existing entry; verified live end-to-end (published a content-requested for a scheduled post → entry went processing → completed → post got correct tags/CQ while remaining invisible, then is set to publish normally).
  • No double-fire: at publish the content is unchanged (existing content-delta path stays quiet); on a normal edit-while-visible, !before.visible is false so this path stays quiet.
  • No loop: the re-enrichment's updatePost runs while the post is already visible (before.visible = true), so the transition guard prevents re-triggering.
  • Bonus: posts scheduled up to 14 days out get current-model enrichment at go-live.

Test plan

  • New unit tests: scheduled freeform becomes visible → re-requests enrichment once; too-short freeform becomes visible → does not.
  • All existing freeform CDC tests pass (10/10), tsc --noEmit clean, eslint clean.

A scheduled freeform post's tags/contentQuality are captured once, when
it's created/scheduled, and are never refreshed at publish time —
publishScheduledPost only flips visibility. So a post enriched during a
broken enrichment window (e.g. the empty-tags bug) goes live broken, and
correcting the source afterwards has no effect because nothing re-sends.

On the visible false->true transition for a freeform post, emit the same
content-requested event used at creation. yggdrasil resolves it to the
existing entry (deterministic id), flips it back to processing, and
re-enriches against current models, so the post publishes with correct
tags/contentQuality. The existing op=u re-request is gated on a content
delta, which a scheduled publish never has, so this path was uncovered.
@pulumi

pulumi Bot commented Jul 8, 2026

Copy link
Copy Markdown

🍹 The Update (preview) for dailydotdev/api/prod (at a62a2de) was successful.

Resource Changes

    Name                                                       Type                           Operation
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment  update
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob    update
+   vpc-native-api-clickhouse-migration-6f93ebd7               kubernetes:batch/v1:Job        create
~   vpc-native-generic-referral-reminder-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob    update
~   vpc-native-user-posts-analytics-refresh-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-daily-digest-cron                               kubernetes:batch/v1:CronJob    update
~   vpc-native-update-views-cron                               kubernetes:batch/v1:CronJob    update
~   vpc-native-bg-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-materialize-monthly-best-post-archives-cron     kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob    update
-   vpc-native-api-db-migration-8bcbf299                       kubernetes:batch/v1:Job        delete
-   vpc-native-api-clickhouse-migration-8bcbf299               kubernetes:batch/v1:Job        delete
~   vpc-native-generate-search-invites-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tag-materialized-views-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-highlights-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob    update
+   vpc-native-api-db-migration-6f93ebd7                       kubernetes:batch/v1:Job        create
~   vpc-native-post-analytics-history-day-clickhouse-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-post-lifecycle-state-clickhouse-cron            kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-gifted-plus-cron                          kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-materialize-yearly-best-post-archives-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-achievements-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-old-notifications-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tags-str-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-check-analytics-report-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob    update
... and 15 other changes

@idoshamun

Copy link
Copy Markdown
Member Author

Closing — this was defense-in-depth, not a root-cause fix. The two actual root causes are already deployed: bragi #257 (tags were optional in the enrich schema → empty) and daily-api #3982 (contentQuality was missing from freeformAllowedFields → stripped on the freeform update path). With both fixed, new scheduled freeform posts enrich correctly at creation on both dimensions, so re-enriching at publish only adds a redundant enrichment. The affected already-scheduled posts were remediated separately.

@idoshamun idoshamun closed this Jul 8, 2026
@idoshamun idoshamun deleted the fix/reenrich-scheduled-freeform-on-publish branch July 8, 2026 07:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant