Jobber - Microservices Platform

Microservices platform built with NestJS, featuring event-driven architecture, gRPC communication, and GraphQL API

---

📋 Table of Contents

• Jobber - Microservices Platform
  • 📋 Table of Contents
  • 🎯 Overview
  • 🏗️ Architecture
    • System Architecture
    • Key Architectural Patterns
    • Microservices
      • 1. Auth Service (apps/auth)
      • 2. Jobs Service (apps/jobs)
      • 3. Products Service (apps/products)
      • 4. Executor Service (apps/executor)
  • 🚀 Getting Started
    • Prerequisites
    • Installation
  • 💻 Development
    • Running Services
    • Building Services
    • Testing
  • 🐳 Deployment
    • Docker Deployment
    • Kubernetes Deployment

---

🎯 Overview

Jobber is a modern, scalable microservices platform built with NestJS and Nx monorepo. It demonstrates enterprise-grade architecture patterns including:

• Event-Driven Architecture using Apache Pulsar for asynchronous communication
• gRPC for high-performance inter-service communication
• GraphQL API for flexible client interactions
• Multi-database strategy with Prisma and Drizzle ORM
• Background job processing with a distributed task execution system
• Containerized deployment with Docker and Kubernetes (Helm)

---

🏗️ Architecture

System Architecture

The Jobber platform follows a microservices architecture with three primary communication patterns: GraphQL for client APIs, gRPC for inter-service communication, and Apache Pulsar for asynchronous job processing.

┌─────────────────────────────────────────────────────────────────────────┐
│                         CLIENT APPLICATIONS                             │
│                   (Web Browser, Mobile App, Postman)                    │
└────────────────────────────┬────────────────────────────────────────────┘
                             │
                             │ (GraphQL)
                             │ Mutations & Queries
                             ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        GRAPHQL API LAYER                                │
│                                                                         │
│  ┌──────────────────┐   ┌──────────────────┐   ┌──────────────────┐     │
│  │   Auth Service   │   │   Jobs Service   │   │ Products Service │     │
│  │   (Port 3000)    │   │   (Port 3001)    │   │   (Port 3002)    │     │
│  ├──────────────────┤   ├──────────────────┤   ├──────────────────┤     │
│  │ • register()     │   │ • createJob()    │   │ • getProducts()  │     │
│  │ • login()        │   │ • uploadFile()   │   │ • addProduct()   │     │
│  │ • getUser()      │   │ • getJobs()      │   │ • updateStock()  │     │
│  └────────┬─────────┘   └────────┬─────────┘   └────────┬─────────┘     │
└───────────┼──────────────────────┼──────────────────────┼───────────────┘
            │                      │                      │
            │                      │                      │
            │ ┌────────────────────┘                      │
            │ │ gRPC: Authenticate(token)                 │
            │ │ Returns: User{id, email}                  │
            ▼ ▼                                           ▼
┌────────────────────────────────────────────────────────────────────────┐
│                      BUSINESS LOGIC LAYER                              │
│                                                                        │
│  ┌──────────────────┐   ┌──────────────────┐   ┌──────────────────┐    │
│  │ Auth Microservice│   │ Jobs Microservice│   │Products Micro-   │    │
│  │ (gRPC: 50051)    │   │ (gRPC: 50052)    │   │service (50053)   │    │
│  ├──────────────────┤   ├──────────────────┤   ├──────────────────┤    │
│  │ • Hash passwords │   │ • Create tasks   │   │ • Manage catalog │    │
│  │ • Verify tokens  │   │ • Store uploads  │   │ • Track stock    │    │
│  │ • Manage users   │   │ • Schedule jobs  │   │ • Price updates  │    │
│  │                  │   │                  │   │                  │    │
│  │ Publishes:       │   │ Publishes:       │   │ Subscribes:      │    │
│  │ - UserEvents     │   │ - JobCreated ────┼───┼──► ProductLoad   │    │
│  └────────┬─────────┘   └────────┬─────────┘   └────────┬─────────┘    │
└───────────┼──────────────────────┼──────────────────────┼──────────────┘
            │                      │                      │
            │                      │ Pulsar Topic         │
            │                      │                      │
            │                      ▼                      │
            │           ┌─────────────────────┐           │
            │           │ Executor Service    │           │
            │           │                     │           │
            │           │ • Consumes jobs     │           │
            │           │ • Executes:         │           │
            │           │   - FibonacciJob    │           │
            │           │   - LoadProductsJob │           │
            │           │   - ...             │           │
            │           │                     │           │
            │           │ • Reports results   │           │
            │           └──────────┬──────────┘           │
            │                      │                      │
            │                      │ Publish results      │
            │                      │                      │
            │                      ▼                      │
            │           ┌─────────────────────┐           │
            │           │  Apache Pulsar      │           │
            │           │  Message Broker     │           │
            │           │  (Port 6650)        │           │
            │           │                     │           │
            │           │  Topics:            │           │
            │           │  • job-events       │           │
            │           │  • job-results      │           │
            │           │  • user-events      │           │
            │           └─────────────────────┘           │
            │                                             │
            ▼                                             ▼
┌─────────────────────────┐                 ┌─────────────────────────┐
│   PostgreSQL            │                 │   PostgreSQL            │
│   Auth Database         │                 │   Other Databases       │
└─────────────────────────┘                 └─────────────────────────┘

Key Architectural Patterns

✅ Separation of Concerns: Each service has a single responsibility
✅ Event-Driven: Asynchronous job processing decouples services
✅ API Gateway Pattern: GraphQL endpoints act as unified client interface
✅ Service Discovery: gRPC enables typed inter-service communication
✅ Database Per Service: Each microservice owns its data
✅ Message Queue: Pulsar ensures reliable, ordered job processing
✅ Scalability: Executor services can be horizontally scaled

---

Microservices

1. Auth Service (apps/auth)

Responsibilities:

• User authentication and authorization
• JWT token management
• User registration and login
• Session management

Technology:

• Database: PostgreSQL with Prisma ORM
• API: GraphQL (Apollo Server)
• Communication: gRPC server for internal auth verification
• Port: HTTP/GraphQL endpoint + gRPC endpoint

---

2. Jobs Service (apps/jobs)

Responsibilities:

• Job/task definition and management
• File upload handling
• Task scheduling and coordination
• Integration with executor service

Technology:

• Database: PostgreSQL with Prisma ORM
• API: GraphQL
• Communication: gRPC client (Auth), gRPC server, Pulsar producer
• Features: File upload system, job orchestration

Available Job Types:

The system currently implements 2 demonstration jobs to showcase the architecture:

1. Fibonacci Job - Computes Fibonacci sequences asynchronously
2. Load Products Job - Imports product data from external sources

Extensibility: The architecture is designed to be highly scalable and easily adaptable for new job types. Simply create a new job class implementing the job interface, and the system will automatically discover and execute it through the executor service.

---

3. Products Service (apps/products)

Responsibilities:

• Product catalog management
• Inventory tracking
• Product CRUD operations

Technology:

• Database: PostgreSQL with Drizzle ORM
• API: GraphQL
• Communication: gRPC server
• Data: Seed data from data/products.json

---

4. Executor Service (apps/executor)

Responsibilities:

• Background job execution
• Distributed task processing
• Job result reporting
• Processes jobs published to Apache Pulsar

Technology:

• Communication: Pulsar consumer
• Features: Worker pool, job execution engine

Job Processing Flow:

1. Jobs service publishes job events to Pulsar
2. Executor consumes events from job queue
3. Jobs are executed in isolated worker processes
4. Results are published back through Pulsar
5. Jobs service updates status in the database

Current Job Implementations:

• Fibonacci Calculator - Demonstrates compute-intensive operations
• Product Loader - Demonstrates data import operations

💡 Note: The system is architected to seamlessly support additional job types. New jobs can be added without modifying the executor core - simply implement the job interface and the system will handle discovery and execution automatically.

---

🚀 Getting Started

Prerequisites

• Node.js (v18 or higher)
• npm or yarn
• Docker & Docker Compose
• Protocol Buffers Compiler (protoc)

Installation

1. Clone the repository

git clone https://github.com/MarwanRadwan7/node-microservices.git
cd jobber

2. Install dependencies

npm install

3. Start infrastructure services

docker-compose up -d

This starts:

• PostgreSQL (port 5432)
• Apache Pulsar (port 6650)

4. Set up environment variables

Create .env files for each service:

# apps/auth/.env
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/auth"
AUTH_GRPC_SERVICE_URL="0.0.0.0:50051"

# apps/jobs/.env
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/jobs"
JOBS_GRPC_SERVICE_URL="0.0.0.0:50052"
AUTH_GRPC_SERVICE_URL="localhost:50051"

# apps/products/.env
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/products"
PRODUCTS_GRPC_SERVICE_URL="0.0.0.0:50053"

5. Run database migrations

# Auth service
npx nx run auth:prisma-migrate

# Jobs service
npx nx run jobs:prisma-migrate

# Products service (Drizzle)
npx nx run products:drizzle-push

6. Generate gRPC types

npx nx run grpc:generate-proto

---

💻 Development

Running Services

Start all services in development mode:

npx nx run-many --target=serve --all --parallel

Start individual services:

# Auth service
npx nx serve auth

# Jobs service
npx nx serve jobs

# Products service
npx nx serve products

# Executor service
npx nx serve executor

Building Services

Build all services:

npx nx run-many --target=build --all

Build individual service:

npx nx build auth

Testing

Run all tests:

npx nx run-many --target=test --all

Run tests for a specific service:

npx nx test auth

---

🐳 Deployment

Docker Deployment

Build Docker images:

# Build all services
docker-compose build

# Build specific service
docker build -f apps/auth/Dockerfile -t jobber-auth .

Run with Docker Compose:

docker-compose up

Kubernetes Deployment

Using Helm:

# Install the Helm chart
helm install jobber ./charts/jobber

# Upgrade existing deployment
helm upgrade jobber ./charts/jobber

# Uninstall
helm uninstall jobber

Chart Configuration:

The Helm chart (charts/jobber/) includes:

• Apache Pulsar cluster configuration
• PostgreSQL database
• Service deployments for all microservices
• Ingress configuration
• ConfigMaps and Secrets

Edit charts/jobber/values.yaml to customize:

• Replica counts
• Resource limits
• Image tags
• Environment variables

---

Built with ❤️ using modern technologies and best practices