commit abe83af5aba2e7ebc7bfa08869a0c644e9e780ee Author: ruinivist Date: Tue Apr 14 19:58:37 2026 +0000 init diff --git a/README.md b/README.md new file mode 100644 index 0000000..e69de29 diff --git a/TASK.md b/TASK.md new file mode 100644 index 0000000..7f634be --- /dev/null +++ b/TASK.md @@ -0,0 +1,172 @@ +# RabbitMQ Hands-On Lab: Build One End-to-End System + +This lab gives you one working async system on your laptop. + +## What You Will Build + +A mini **order processing pipeline**: + +1. `producer` sends `order.created` messages. +2. `worker` consumes orders and processes them. +3. `worker` publishes final status (`order.processed` or `order.failed`). +4. `notifier` consumes status messages and prints final result. +5. Failed messages go to a DLQ after retries. + +By the end, you will have a real event flow running through RabbitMQ. + +## Definition of Done + +- [ ] You can run all services. +- [ ] You can submit 5 orders and see processed/failure outcomes. +- [ ] You can kill the worker and observe redelivery. +- [ ] You can force failures and see DLQ behavior. + +## Lab Stack + +- Language: **Python** +- Broker: RabbitMQ (Docker) +- Library: `pika` + +## Step 1: Project Setup (10 min) + +- [ ] Create files: +- [ ] `docker-compose.yml` +- [ ] `requirements.txt` +- [ ] `producer.py` +- [ ] `worker.py` +- [ ] `notifier.py` + +- [ ] Put this in `requirements.txt`: + +```txt +pika==1.3.2 +``` + +- [ ] Put this in `docker-compose.yml`: + +```yaml +version: "3.9" +services: + rabbitmq: + image: rabbitmq:3-management + ports: + - "5672:5672" + - "15672:15672" +``` + +- [ ] Start RabbitMQ: + +```bash +docker compose up -d +``` + +- [ ] Open UI: `http://localhost:15672` (`guest` / `guest`). + +## Step 2: RabbitMQ Topology (15 min) + +Create these resources from code (preferred) or UI: + +- [ ] Exchange `orders.x` (direct) +- [ ] Exchange `orders.dlx` (direct) +- [ ] Queue `orders.q` +- [ ] Queue `orders.processed.q` +- [ ] Queue `orders.failed.q` +- [ ] Queue `orders.dlq` + +Bindings: + +- [ ] `orders.q` <- `orders.x` with `order.created` +- [ ] `orders.processed.q` <- `orders.x` with `order.processed` +- [ ] `orders.failed.q` <- `orders.x` with `order.failed` +- [ ] `orders.dlq` <- `orders.dlx` with `order.dead` + +Queue args for `orders.q`: + +- [ ] `x-dead-letter-exchange=orders.dlx` +- [ ] `x-dead-letter-routing-key=order.dead` + +## Step 3: Producer (20 min) + +- [ ] `producer.py` should publish 5 JSON orders to routing key `order.created`. +- [ ] Each message should include: +- [ ] `order_id` +- [ ] `user_id` +- [ ] `amount` +- [ ] `should_fail` (set true for at least 1 order) +- [ ] `retry_count` (start at 0) +- [ ] Mark messages persistent. + +Success check: + +- [ ] Running `python producer.py` prints “Published order …” 5 times. + +## Step 4: Worker (35 min) + +- [ ] `worker.py` consumes from `orders.q` with manual ACK. +- [ ] For each message: +- [ ] Parse JSON +- [ ] Simulate processing (`sleep(1)`) +- [ ] If `should_fail` and `retry_count < 2`, republish same order with `retry_count + 1` to `order.created`, ACK original. +- [ ] If `should_fail` and `retry_count >= 2`, publish to `order.failed`, ACK original. +- [ ] Else publish to `order.processed`, ACK original. + +Important: + +- [ ] Set `prefetch_count=1`. +- [ ] Use persistent publish for all outgoing messages. + +Success check: + +- [ ] You see logs for retries and final outcomes. + +## Step 5: Notifier (15 min) + +- [ ] `notifier.py` consumes both `orders.processed.q` and `orders.failed.q`. +- [ ] Print concise final messages, e.g.: +- [ ] `ORDER 1001 PROCESSED` +- [ ] `ORDER 1003 FAILED AFTER RETRIES` +- [ ] ACK all notifier messages. + +Success check: + +- [ ] After running producer, notifier shows both success and failure events. + +## Step 6: End-to-End Run (10 min) + +Run in 3 terminals: + +- [ ] Terminal A: `python worker.py` +- [ ] Terminal B: `python notifier.py` +- [ ] Terminal C: `python producer.py` + +Expected behavior: + +- [ ] Normal orders finish in `orders.processed.q` path. +- [ ] Failing orders retry 2 times then go `orders.failed.q`. + +## Step 7: Failure Drill (10 min) + +- [ ] Start worker and producer. +- [ ] Kill worker while processing a message. +- [ ] Restart worker. +- [ ] Verify unacked message is redelivered and processed. + +## Step 8: DLQ Drill (10 min) + +- [ ] In worker, temporarily `basic_nack(requeue=False)` for one special order. +- [ ] Verify message appears in `orders.dlq`. +- [ ] Inspect message payload from RabbitMQ UI. + +## What You Learn (Concrete) + +- [ ] RabbitMQ core flow (producer -> exchange -> queue -> consumer) +- [ ] Routing keys and direct exchange +- [ ] Manual ACK and redelivery +- [ ] Retry strategy and failure handling +- [ ] DLQ basics used in real systems + +## Optional Next Lab + +- [ ] Add HTTP API (`FastAPI`) that accepts `/orders` and publishes to RabbitMQ. +- [ ] Add publisher confirms. +- [ ] Replace classic queue with quorum queue and compare behavior.