Add job cleaner excluded queues that can be configured through pilot#1004
Merged
Conversation
brandur
force-pushed
the
brandur-ommitted-queues
branch
from
16c4468 to
8cd9ee1
Compare
brandur
force-pushed
the
brandur-ommitted-queues
branch
7 times, most recently
from
a977e1c to
c31a4cb
Compare
bgentry
approved these changes
Aug 14, 2025
Comment on lines
+1044
to
+1057
| // I ran into yet another huge sqlc SQLite bug in that when mixing | ||
| // normal parameters with a `sqlc.slice` the latter must appear at | ||
| // the very end because it'll produce unnamed placeholders (?) | ||
| // instead of positional placeholders (?1) like most parameters. The | ||
| // trick of putting it at the end works, but only if you have | ||
| // exactly one `sqlc.slice` needed. If you need multiple and they | ||
| // need to be interspersed with other parameters (like in the case | ||
| // of `queues_excluded` and `queues_included`), everything stops | ||
| // working real fast. I could have worked around this by breaking | ||
| // the SQLite version of this operation into two sqlc queries, but | ||
| // since we only expect to need `queues_excluded` on SQLite (and not | ||
| // `queues_included` for the foreseeable future), I've just set | ||
| // SQLite to not support `queues_included` for the time being. |
Contributor
Author
There was a problem hiding this comment.
Yeah, ugh :( We should chat about this, but using Sqlc for SQLite might've been a mistake. I was somewhat optimistic that I'd be able to start landing some fixes over there, but the one fix I got in is still unreleased months later, and a few other high quality patches from others are ignored. I'm honestly not 100% sure what to do because I don't really see any good alternatives. Options are kind of "bad" or "worse".
Add a new pilot function that lets it dictate that certain queues should be excluded from the standard cleaning process that manages retention. Along with the `JobDeleteBefore` driver function picking up a new `QueuesExcluded` parameter, we also add its inverse of `QueuesIncluded` which lets us delete jobs from a specific set of queues.
brandur
force-pushed
the
brandur-ommitted-queues
branch
from
c31a4cb to
a32caa8
Compare
Contributor
Author
|
Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a new pilot function that lets it dictate that certain queues should
be excluded from the standard cleaning process that manages retention.
Along with the
JobDeleteBeforedriver function picking up a newQueuesExcludedparameter, we also add its inverse ofQueuesIncludedwhich lets us delete jobs from a specific set of queues.