Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
[flake8]
max-line-length = 115
max-line-length = 115
ignore = F403, F401, F405, W503, E302
224 changes: 224 additions & 0 deletions example/DEMO_WORKFLOWS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
# Durable Execution Engine SDK - Demo Workflows

This document provides a comprehensive overview of all workflows in the demo application, including their failure characteristics, retry configurations, and expected behavior.

## Overview

The demo showcases three main services with multiple workflows that demonstrate durable execution patterns, retry mechanisms, and error handling:

- **Orders Service**: Complex multi-step workflows with dependent actions
- **Users Service**: User management with notifications
- **Payments Service**: Payment processing and refund handling

## Services and Workflows

### 🛒 Orders Service (`orders`)

#### 1. Process Order (`process_order`)
**Endpoint**: `POST /execute/orders/process_order`

A complex multi-step workflow demonstrating sequential action execution with different retry strategies.

**Input**: `OrderInput`
```json
{
"order_id": "string",
"customer_email": "string",
"items": [
{
"id": "string",
"quantity": number,
"price": number
}
],
"total_amount": number
}
```

**Workflow Steps**:
1. **Payment Validation** (8s processing time)
- Action: `validate_payment`
- Failure Rate: **50%**
- Max Retries: 3
- Retry Mechanism: Exponential
- Error: "Payment validation failed"

2. **Inventory Reservation** (10s processing time per item)
- Action: `reserve_inventory` (executed for each item)
- Failure Rate: **50%**
- Max Retries: 2
- Retry Mechanism: Linear
- Error: "Insufficient inventory for {item_id}"
- Custom Action Names: `reserve_inventory_0`, `reserve_inventory_1`, etc.

3. **Order Confirmation Notification** (6s processing time)
- Action: `send_notification`
- Failure Rate: **0%** (Always succeeds)
- Max Retries: 2
- Retry Mechanism: Constant
- Recipient: Customer email

**Expected Behavior**: Due to high failure rates in payment and inventory steps, this workflow frequently requires retries and may fail entirely if retries are exhausted.

#### 2. Get Order Status (`get_order_status`)
**Endpoint**: `POST /execute/orders/get_order_status`

A simple status check workflow that always succeeds.

**Input**: `OrderStatusInput`
```json
{
"order_id": "string"
}
```

**Workflow Steps**:
1. **Status Check** (5s processing time)
- Action: `check_order_status`
- Failure Rate: **0%** (Always succeeds)
- Max Retries: 1
- Retry Mechanism: Constant
- Returns random status: "pending", "processing", "shipped", or "delivered"

**Expected Behavior**: Reliable workflow that demonstrates successful action execution.

### 👤 Users Service (`users`)

#### 1. Register User (`register_user`)
**Endpoint**: `POST /execute/users/register_user`

User registration workflow with welcome notification.

**Input**: `UserInput`
```json
{
"email": "string",
"username": "string",
"password": "string"
}
```

**Workflow Steps**:
1. **User Creation** (7s processing time)
- Action: `create_user`
- Failure Rate: **0%** (Always succeeds)
- Max Retries: 2
- Retry Mechanism: Exponential

2. **Welcome Notification** (6s processing time)
- Action: `send_notification`
- Failure Rate: **0%** (Always succeeds)
- Max Retries: 1
- Retry Mechanism: Constant
- Message: "Welcome {username}!"

**Expected Behavior**: Highly reliable workflow that consistently succeeds.

### 💳 Payments Service (`payments`)

#### 1. Process Refund (`process_refund`)
**Endpoint**: `POST /execute/payments/process_refund`

Complex refund processing with order verification and notifications.

**Input**: `RefundInput`
```json
{
"order_id": "string",
"amount": number,
"reason": "string"
}
```

**Workflow Steps**:
1. **Pre-Refund Order Check** (5s processing time)
- Action: `check_order_status`
- Failure Rate: **0%** (Always succeeds)
- Max Retries: 2
- Retry Mechanism: Constant
- Custom Action Name: `pre_refund_order_check`

2. **Refund Processing** (9s processing time)
- Action: `process_refund`
- Failure Rate: **90%** (Very high failure rate)
- Max Retries: 3
- Retry Mechanism: Exponential
- Error: "Refund processing failed"

3. **Finance Notification** (6s processing time)
- Action: `send_notification`
- Failure Rate: **0%** (Always succeeds)
- Max Retries: 2
- Retry Mechanism: Linear
- Recipient: "finance@company.com"

**Expected Behavior**: This workflow has the highest failure rate and will frequently exhaust retries due to the 90% failure rate in refund processing.

#### 2. Verify Payment and Notify (`verify_payment_and_notify`)
**Endpoint**: `POST /execute/payments/verify_payment_and_notify`

Payment verification with admin notification.

**Input**: `PaymentInput`
```json
{
"amount": number,
"payment_method": "string"
}
```

**Workflow Steps**:
1. **Payment Validation** (8s processing time)
- Action: `validate_payment`
- Failure Rate: **50%**
- Max Retries: 3
- Retry Mechanism: Exponential
- Custom Action Name: `primary_payment_validation`

2. **Admin Notification** (6s processing time)
- Action: `send_notification`
- Failure Rate: **0%** (Always succeeds)
- Max Retries: 2
- Retry Mechanism: Linear
- Recipient: "admin@company.com"

**Expected Behavior**: Moderate failure rate due to payment validation step.

## Action Failure Summary

| Action | Processing Time | Failure Rate | Typical Error |
|--------|----------------|--------------|---------------|
| `validate_payment` | 8 seconds | **50%** | "Payment validation failed" |
| `reserve_inventory` | 10 seconds | **50%** | "Insufficient inventory for {item_id}" |
| `process_refund` | 9 seconds | **90%** | "Refund processing failed" |
| `send_notification` | 6 seconds | **0%** | Never fails |
| `create_user` | 7 seconds | **0%** | Never fails |
| `check_order_status` | 5 seconds | **0%** | Never fails |

## Retry Mechanisms

The demo showcases three different retry mechanisms:

1. **Exponential**: Exponentially increasing delays between retries
2. **Linear**: Fixed incremental delays between retries
3. **Constant**: Fixed delay between retries

## Testing the Demo

### High Success Rate Workflows
- `users/register_user` - Should consistently succeed
- `orders/get_order_status` - Always succeeds

### Moderate Failure Rate Workflows
- `orders/process_order` - 50% failure rate on payment and inventory
- `payments/verify_payment_and_notify` - 50% failure rate on payment

### High Failure Rate Workflows
- `payments/process_refund` - 90% failure rate on refund processing

### Recommended Test Scenarios

1. **Success Path**: Call `register_user` or `get_order_status`
2. **Retry Demonstration**: Call `process_order` or `verify_payment_and_notify` multiple times
3. **Failure Demonstration**: Call `process_refund` to see retry exhaustion
4. **Complex Workflow**: Call `process_order` with multiple items to see parallel inventory reservations
42 changes: 20 additions & 22 deletions example/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,37 +1,35 @@
# Stage 1: Build stage
FROM python:3.10-slim AS builder

# Set the working directory
WORKDIR /app

# Copy the requirements file into the container
COPY requirements.txt .

# Create a virtual environment and install dependencies
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

# Copy the application code
COPY . .
COPY ../src ./src
COPY ./ ./example

# Stage 2: Final stage
FROM python:3.10-slim

# Set the working directory to /app/Example
WORKDIR /app/Example
WORKDIR /app

COPY --from=builder /usr/local/lib/python3.10/site-packages /usr/local/lib/python3.10/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin

# Copy only the virtual environment from the builder stage
COPY --from=builder /opt/venv /opt/venv

# Copy the application code into the Example directory
COPY . .
COPY src/ ./src/
COPY example/ ./example/

# Set the environment variables
ENV PATH="/opt/venv/bin:$PATH"
ENV DURABLE_ENGINE_BASE_URL=http://host.docker.internal:8080/api/v1
ENV PYTHONPATH=/app/src:/app
ENV PYTHONUNBUFFERED=1

ENV LOG_LEVEL=DEBUG

# Expose the port the app runs on
EXPOSE 8000

# Command to run the application using Uvicorn
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8000/discover || exit 1

WORKDIR /app/example

CMD ["python", "main.py"]
1 change: 0 additions & 1 deletion example/app/__init__.py

This file was deleted.

7 changes: 0 additions & 7 deletions example/app/main.py

This file was deleted.

Empty file added example/demo/__init__.py
Empty file.
79 changes: 79 additions & 0 deletions example/demo/actions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
import asyncio
import random
import time

from .models import *


def validate_payment_action(input_data: PaymentInput) -> PaymentResult:
time.sleep(8)

if random.random() < 0.5:
raise Exception("Payment validation failed")

return PaymentResult(
payment_id=f"pay_{random.randint(1000, 9999)}",
amount=input_data.amount,
status="validated",
)


def reserve_inventory_action(input_data: InventoryInput) -> InventoryResult:
time.sleep(10)

if random.random() < 0.50:
raise Exception(f"Insufficient inventory for {input_data.item_id}")

return InventoryResult(
reservation_id=f"res_{random.randint(1000, 9999)}",
item_id=input_data.item_id,
quantity=input_data.quantity,
status="reserved",
)


async def send_notification_action(
input_data: NotificationInput,
) -> NotificationResult:
await asyncio.sleep(6)

return NotificationResult(
notification_id=f"notif_{random.randint(1000, 9999)}",
recipient=input_data.recipient,
status="sent",
)


def create_user_action(input_data: UserInput) -> UserResult:
time.sleep(7)

return UserResult(
user_id=f"user_{random.randint(1000, 9999)}",
email=input_data.email,
status="active",
)


def process_refund_action(input_data: RefundInput) -> RefundResult:
time.sleep(9)

if random.random() < 0.9:
raise Exception("Refund processing failed")

return RefundResult(
refund_id=f"ref_{random.randint(1000, 9999)}",
amount=input_data.amount,
status="processed",
)


def check_order_status_action(order_id: str) -> dict:
time.sleep(5)

return {
"order_id": order_id,
"status": random.choice(
["pending", "processing", "shipped", "delivered"]
),
"tracking_number": f"TRK{random.randint(100000, 999999)}",
}
Loading