feat(learning): add the workers and redis job queue roadmap#82
Open
OthmaneZ05 wants to merge 1 commit into
Open
feat(learning): add the workers and redis job queue roadmap#82OthmaneZ05 wants to merge 1 commit into
OthmaneZ05 wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
Adds the third catalog roadmap, “Workers & the Redis job queue” (
redis-queue-workers, EN + FR, 8 steps, ~40 min, intermediate) in the Nimbus Books universe. The learner feels a 2-second synchronous checkout, decouples it with a producer pushing orders onto a real Redis list, writes a blockingBRPOPworker, archives receipts into PostgreSQL (stateless workers / shared state), floods the queue with a 300-order flash sale to watch backpressure live, scales consumption with an auto-scaling group cloned from the worker (a security-group rule with the ASG node as Source covers every replica), and finishes with a poison message handled by retry counters shared in Redis plus a dead-letter queue. Step 7 is an explicit "validate before fixing" beat, and every validator was chosen to remain stable when earlier steps are re-validated after completion.Uses existing validators only:
container_running,http_get_contains,edge_exists,redis_key_exists,table_exists,port_denied,asg_replicas.Types of Changes
Verification & Testing
Automated Checks
npm run lintsuccessfully with no errorsnpm run buildsuccessfully with no compilation errorsnpm testsuccessfully (all tests pass) — backend 400/400Manual Verification
npm run roadmap:validateOK for both language files; a script asserts EN/FR ids and validators are strictly identical.Played the full roadmap against real Docker with an API driver that mirrors the canvas exactly (network config with default subnet routes, per-node security groups, ASG deploy via the same endpoints as the modal) and executes in the containers the exact code blocks extracted from the roadmap JSON. Result: 34/34 checks green —
SELECT worker, COUNT(*) FROM receipts GROUP BY workershows 4 distinct container hostnames (67/99/67/68);queue:orders:dead, checkout still accepting;webthenworker-1stopped/restarted one at a time (canvas-style) — servers revive via the baked boot line and all 8 steps re-validate green;Adjacent findings (not addressed here, per scope rules)
derssa/backend-lab-mongo:v1image is currently unusable on a Fedora 44 / kernel 7.0.9 host: mongod silently segfaults (exit 139) 30–90 s after boot, including a baredocker runoutside Torollo (reproduced 3×). This is why the receipt archive uses PostgreSQL (table_exists) instead of the originally considered Mongo (mongo_collection_existsstays unexercised by the catalog). The image likely needs a rebuild on a newer mongod.nodeIpMapwins over live state), silently breaking cross-node traffic. A single-container stop/start (the canvas gesture) keeps its IP and re-wires correctly — verified. Worth a backlog entry.Checklist