Skip to content

Improve the order of message operation at _release_batch()#47

Open
kawasin73 wants to merge 1 commit into
Nextdoor:masterfrom
aces-inc:avoid_message_lost
Open

Improve the order of message operation at _release_batch()#47
kawasin73 wants to merge 1 commit into
Nextdoor:masterfrom
aces-inc:avoid_message_lost

Conversation

@kawasin73

Copy link
Copy Markdown

In order to avoid message lost, you should enqueue messages to Dead Letter Queue before deleting messages in case of sudden death of worker process.

I think it is better to release_messages() soon after checking RESET_TIMEOUT_THRESHOLD because delete_messages() may take some time.

@timnd timnd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks like a good improvement.

Comment thread kale/worker.py
Comment on lines +302 to +306
if messages_to_be_deleted:
# Note: This includes failed tasks.
self._consumer.delete_messages(messages_to_be_deleted,
self._batch_queue.name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this block before the comment block above.

@MasashiFukuzawa MasashiFukuzawa deleted the avoid_message_lost branch October 16, 2025 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants