diff --git a/README.md b/README.md index e69de29..4645728 100644 --- a/README.md +++ b/README.md @@ -0,0 +1,168 @@ +# RabbitMQ Hands-On Lab: Build One End-to-End System + +This lab gives you one working async system on your laptop. + +## What You Will Build + +A mini **order processing pipeline**: + +1. `producer` sends `order.created` messages. +2. `worker` consumes orders and processes them. +3. `worker` publishes final status (`order.processed` or `order.failed`). +4. `notifier` consumes status messages and prints final result. +5. Failed messages go to a DLQ after retries. + +By the end, you will have a real event flow running through RabbitMQ. + +## Definition of Done + +- [ ] You can run all services. +- [ ] You can submit 5 orders and see processed/failure outcomes. +- [ ] You can kill the worker and observe redelivery. +- [ ] You can force failures and see DLQ behavior. + +## Lab Stack + +- Language: **Python** +- Tooling: `uv` +- Broker: RabbitMQ **4+** (Docker) +- Library: `pika` + +## Step 1: Project Setup (10 min) + +- [x] Create files: +- [x] `docker-compose.yml` +- [x] `pyproject.toml` (created by `uv init`) +- [x] `producer.py` +- [x] `worker.py` +- [x] `notifier.py` + +- [x] Initialize project and dependency with `uv`: + +```bash +uv init --name rabbitmq-lab +uv add pika==1.3.2 +``` + +- [x] Put this in `docker-compose.yml`: + +```yaml +version: "3.9" +services: + rabbitmq: + image: rabbitmq:4-management + ports: + - "5672:5672" + - "15672:15672" +``` + +- [x] Start RabbitMQ: + +```bash +docker compose up -d +``` + +- [x] Open UI: `http://localhost:15672` (`guest` / `guest`). + +## Step 2: RabbitMQ Topology (15 min) + +Create these resources from code (preferred) or UI: + +- [x] Exchange `orders.x` (direct) +- [x] Exchange `orders.dlx` (direct) +- [x] Queue `orders.q` +- [x] Queue `orders.processed.q` +- [x] Queue `orders.failed.q` +- [x] Queue `orders.dlq` + +Bindings: + +- [x] `orders.q` <- `orders.x` with `order.created` +- [x] `orders.processed.q` <- `orders.x` with `order.processed` +- [x] `orders.failed.q` <- `orders.x` with `order.failed` +- [x] `orders.dlq` <- `orders.dlx` with `order.dead` + +Queue args for `orders.q`: + +- [x] `x-dead-letter-exchange=orders.dlx` +- [x] `x-dead-letter-routing-key=order.dead` + +## Step 3: Producer (20 min) + +- [x] `producer.py` should publish 5 JSON orders to routing key `order.created`. +- [x] Each message should include: +- [x] `order_id` +- [x] `user_id` +- [x] `amount` +- [x] `should_fail` (set true for at least 1 order) +- [x] `retry_count` (start at 0) +- [x] Mark messages persistent. + +Success check: + +- [x] Running `uv run producer.py` prints “Published order …” 5 times. + +## Step 4: Worker (35 min) + +- [x] `worker.py` consumes from `orders.q` with manual ACK. +- [x] For each message: +- [x] Parse JSON +- [x] Simulate processing (`sleep(1)`) +- [x] If `should_fail` and `retry_count < 2`, republish same order with `retry_count + 1` to `order.created`, ACK original. +- [x] If `should_fail` and `retry_count >= 2`, publish to `order.failed`, ACK original. +- [x] Else publish to `order.processed`, ACK original. + +Important: + +- [x] Set `prefetch_count=1`. +- [x] Use persistent publish for all outgoing messages. + +Success check: + +- [x] You see logs for retries and final outcomes. + +## Step 5: Notifier (15 min) + +- [x] `notifier.py` consumes both `orders.processed.q` and `orders.failed.q`. +- [x] Print concise final messages, e.g.: +- [x] `ORDER 1001 PROCESSED` +- [x] `ORDER 1003 FAILED AFTER RETRIES` +- [x] ACK all notifier messages. + +Success check: + +- [x] After running producer, notifier shows both success and failure events. + +## Step 6: End-to-End Run (10 min) + +Run in 3 terminals: + +- [x] Terminal A: `uv run worker.py` +- [x] Terminal B: `uv run notifier.py` +- [x] Terminal C: `uv run producer.py` + +Expected behavior: + +- [x] Normal orders finish in `orders.processed.q` path. +- [x] Failing orders retry 2 times then go `orders.failed.q`. + +## Step 7: Failure Drill (10 min) + +- [x] Start worker and producer. +- [x] Kill worker while processing a message. +- [x] Restart worker. +- [x] Verify unacked message is redelivered and processed. + +## Step 8: DLQ Drill (10 min) + +- [x] In worker, temporarily `basic_nack(requeue=False)` for one special order. +- [x] Verify message appears in `orders.dlq`. +- [x] Inspect message payload from RabbitMQ UI. + +## What You Learn (Concrete) + +- [ ] RabbitMQ core flow (producer -> exchange -> queue -> consumer) +- [ ] Routing keys and direct exchange +- [ ] Manual ACK and redelivery +- [ ] Retry strategy and failure handling +- [ ] DLQ basics used in real systems diff --git a/TASK.md b/TASK.md deleted file mode 100644 index 4645728..0000000 --- a/TASK.md +++ /dev/null @@ -1,168 +0,0 @@ -# RabbitMQ Hands-On Lab: Build One End-to-End System - -This lab gives you one working async system on your laptop. - -## What You Will Build - -A mini **order processing pipeline**: - -1. `producer` sends `order.created` messages. -2. `worker` consumes orders and processes them. -3. `worker` publishes final status (`order.processed` or `order.failed`). -4. `notifier` consumes status messages and prints final result. -5. Failed messages go to a DLQ after retries. - -By the end, you will have a real event flow running through RabbitMQ. - -## Definition of Done - -- [ ] You can run all services. -- [ ] You can submit 5 orders and see processed/failure outcomes. -- [ ] You can kill the worker and observe redelivery. -- [ ] You can force failures and see DLQ behavior. - -## Lab Stack - -- Language: **Python** -- Tooling: `uv` -- Broker: RabbitMQ **4+** (Docker) -- Library: `pika` - -## Step 1: Project Setup (10 min) - -- [x] Create files: -- [x] `docker-compose.yml` -- [x] `pyproject.toml` (created by `uv init`) -- [x] `producer.py` -- [x] `worker.py` -- [x] `notifier.py` - -- [x] Initialize project and dependency with `uv`: - -```bash -uv init --name rabbitmq-lab -uv add pika==1.3.2 -``` - -- [x] Put this in `docker-compose.yml`: - -```yaml -version: "3.9" -services: - rabbitmq: - image: rabbitmq:4-management - ports: - - "5672:5672" - - "15672:15672" -``` - -- [x] Start RabbitMQ: - -```bash -docker compose up -d -``` - -- [x] Open UI: `http://localhost:15672` (`guest` / `guest`). - -## Step 2: RabbitMQ Topology (15 min) - -Create these resources from code (preferred) or UI: - -- [x] Exchange `orders.x` (direct) -- [x] Exchange `orders.dlx` (direct) -- [x] Queue `orders.q` -- [x] Queue `orders.processed.q` -- [x] Queue `orders.failed.q` -- [x] Queue `orders.dlq` - -Bindings: - -- [x] `orders.q` <- `orders.x` with `order.created` -- [x] `orders.processed.q` <- `orders.x` with `order.processed` -- [x] `orders.failed.q` <- `orders.x` with `order.failed` -- [x] `orders.dlq` <- `orders.dlx` with `order.dead` - -Queue args for `orders.q`: - -- [x] `x-dead-letter-exchange=orders.dlx` -- [x] `x-dead-letter-routing-key=order.dead` - -## Step 3: Producer (20 min) - -- [x] `producer.py` should publish 5 JSON orders to routing key `order.created`. -- [x] Each message should include: -- [x] `order_id` -- [x] `user_id` -- [x] `amount` -- [x] `should_fail` (set true for at least 1 order) -- [x] `retry_count` (start at 0) -- [x] Mark messages persistent. - -Success check: - -- [x] Running `uv run producer.py` prints “Published order …” 5 times. - -## Step 4: Worker (35 min) - -- [x] `worker.py` consumes from `orders.q` with manual ACK. -- [x] For each message: -- [x] Parse JSON -- [x] Simulate processing (`sleep(1)`) -- [x] If `should_fail` and `retry_count < 2`, republish same order with `retry_count + 1` to `order.created`, ACK original. -- [x] If `should_fail` and `retry_count >= 2`, publish to `order.failed`, ACK original. -- [x] Else publish to `order.processed`, ACK original. - -Important: - -- [x] Set `prefetch_count=1`. -- [x] Use persistent publish for all outgoing messages. - -Success check: - -- [x] You see logs for retries and final outcomes. - -## Step 5: Notifier (15 min) - -- [x] `notifier.py` consumes both `orders.processed.q` and `orders.failed.q`. -- [x] Print concise final messages, e.g.: -- [x] `ORDER 1001 PROCESSED` -- [x] `ORDER 1003 FAILED AFTER RETRIES` -- [x] ACK all notifier messages. - -Success check: - -- [x] After running producer, notifier shows both success and failure events. - -## Step 6: End-to-End Run (10 min) - -Run in 3 terminals: - -- [x] Terminal A: `uv run worker.py` -- [x] Terminal B: `uv run notifier.py` -- [x] Terminal C: `uv run producer.py` - -Expected behavior: - -- [x] Normal orders finish in `orders.processed.q` path. -- [x] Failing orders retry 2 times then go `orders.failed.q`. - -## Step 7: Failure Drill (10 min) - -- [x] Start worker and producer. -- [x] Kill worker while processing a message. -- [x] Restart worker. -- [x] Verify unacked message is redelivered and processed. - -## Step 8: DLQ Drill (10 min) - -- [x] In worker, temporarily `basic_nack(requeue=False)` for one special order. -- [x] Verify message appears in `orders.dlq`. -- [x] Inspect message payload from RabbitMQ UI. - -## What You Learn (Concrete) - -- [ ] RabbitMQ core flow (producer -> exchange -> queue -> consumer) -- [ ] Routing keys and direct exchange -- [ ] Manual ACK and redelivery -- [ ] Retry strategy and failure handling -- [ ] DLQ basics used in real systems