Skip to content

README.md

Latest commit

 

History

History
373 lines (288 loc) · 9.74 KB

README.md

File metadata and controls

373 lines (288 loc) · 9.74 KB
Raw

CreditAI - Documentation

Complete index of CreditAI system technical documentation.

Also see: CONTRIBUTING.md · License & support · LLM OpenAI-compatible · llms.txt · version version.txt

General Documentation

ARCHITECTURE.md ⭐ Start here!

System overview in 3 levels:

  • Application layers
  • Simplified pipeline
  • Main components (11 modules)
  • Complete tech stack
  • Design patterns
  • Quality characteristics

Ideal for: Developers new to the project, architects, tech leads

architecture-pipeline.md 🔧 Complete technical documentation

Detailed decision pipeline phase by phase:

  1. INTAKE - Request reception
  2. REGULATORY - CRC/CIRBE
  3. EXTERNAL DATA - 11 data providers
  4. AI AGENTS - 6 specialized agents
  5. FEATURE ENGINEERING - Vector construction
  6. ML SCORING - Probability of default
  7. SOLVENCY - F1-F5 rules
  8. NARRATIVE AI - Local LLM
  9. DECISION - Consolidation and audit

Also includes:

  • Manual Data Entry Pipeline
  • Document Processing Pipeline (OCR + AI)
  • Early Warning System
  • Model Risk Management
  • State diagram
  • Folder structure

Ideal for: Developers implementing features, QA, DevOps

pipeline-diagram.mmd 📊 Visual diagram

Mermaid diagram of complete pipeline flow.

How to visualize:

  • Copy content to mermaid.live
  • Or use Mermaid extension in VS Code
  • Or GitHub/GitLab render automatically

Ideal for: Presentations, onboarding, executive documentation

🔌 Integrations and APIs

integrations.md

Configuration of all external integrations:

Regulatory:

  • CRC (Banco de Portugal)
  • CIRBE (Banco de España)

Data Providers:

  • Yapily (Open Banking)
  • Orbis / SABI (Financial)
  • IES (Fiscal)
  • Creditsafe (Credit)
  • Moody's EDF (Risk)
  • ESG (Sustainability)
  • World-Check (AML)
  • Citius (Judicial - manual only)

Includes:

  • Modes: Stub vs Production
  • Secrets configuration (Key Vault)
  • Health checks
  • Fallback policies

Ideal for: DevOps, environment configuration

📄 Document Processing

document-processing.md 🆕 OCR + AI System

Complete guide for upload and automatic extraction system:

Features:

  • PDF, PNG, JPG, TIFF upload
  • OCR with Tesseract
  • AI Parsing via OpenAI-compatible LLM
  • Structured field extraction
  • Integration with manual entry

Content:

  • Architecture (TesseractOcrService + OpenAI-compatible LLM parsing)
  • API endpoints (/api/documents)
  • Supported formats
  • Data sources (11 types)
  • Configuration (Tesseract + Llm:* / ContextMemory)
  • Best practices
  • Troubleshooting

Ideal for: Analysts (usage), developers (implementation)

📋 Processes and Workflows

Workflow: Credit Request Creation

  1. Analyst accesses Web UI → Login
  2. Menu "Requests" → "New Request"
  3. Fill form (applicant, amount, term)
  4. Submit → Automatic pipeline starts
  5. If manual data needed → notification
  6. Analyst fills form OR uploads document
  7. Pipeline resumes → decision generated
  8. Supervisor reviews → approves/rejects
  9. Audit log recorded → continuous monitoring

Workflow: Document Upload (OCR)

  1. Analyst has PDF/image document
  2. API POST /api/documents/{requestId}/process
  3. System executes OCR (Tesseract)
  4. LLM extracts fields (POST /v1/chat/completions)
  5. Returns JSON with structured fields
  6. Analyst reviews extracted fields
  7. Submits to manual entry
  8. Pipeline resumes automatically

Workflow: Early Warning

  1. Credit approved and disbursed
  2. Background job executes every 60 min
  3. Re-queries CRC/CIRBE
  4. Re-calculates current score
  5. Detects degradation (PSI, PD, incidents)
  6. Creates alert if necessary
  7. Analyst reviews alert
  8. Decides action (continuous monitoring)

🧪 Testing

Integration Tests

  • Location: tests/CreditAI.IntegrationTests/
  • Total: 95 tests (6 new document processing)
  • Coverage: >80% of critical pipeline

Main suites:

  • PipelineIntegrationTests - End-to-end pipeline
  • IberianExpansionIntegrationTests - CRC/CIRBE routing
  • CitiusManualDataIntegrationTests - Judicial data
  • DocumentProcessingIntegrationTests - OCR + AI parsing
  • OperationalHealthTests - Health checks
  • ManualDataTests - Robust fallback

Execute:

dotnet test tests/CreditAI.IntegrationTests/

Architecture Tests

  • Location: tests/CreditAI.ArchitectureTests/
  • Validates dependency rules between layers
  • Ensures Clean Architecture

🔐 Security and Compliance

Authentication

  • Development: ASP.NET Identity (local)
  • Production: Entra ID (Azure AD)
  • Roles: Analyst, Supervisor, Admin
  • Policies: CreditRead, CreditWrite

F1-F5 Compliance

  • F1: High risk blocks (PD, incidents, sanctions)
  • F2: Income/capacity validation
  • F3: Structural checks
  • F4: Market/sector risks
  • F5: Model governance (PSI, approval)

Audit

  • Immutable audit log (SHA-256 hash chain)
  • Each action recorded with timestamp + actor
  • Export: CSV, JSON
  • Verifiable integrity

Privacy

  • On-premise LLM (Llm:* or ContextMemory) + Tesseract
  • Zero data leakage to cloud
  • GDPR compliant
  • Tenant isolation (PostgreSQL RLS)

🛠️ Operations

Deployment

Development:

dotnet run --project src/CreditAI.Web
dotnet run --project src/CreditAI.API

Production (Docker):

docker build -t creditai-api -f Dockerfile.api .
docker build -t creditai-web -f Dockerfile.web .
docker compose up -d

Migrations

# Create new migration
dotnet ef migrations add MigrationName \
  -p src/CreditAI.Infrastructure \
  -s src/CreditAI.API

# Apply to database
dotnet ef database update \
  -p src/CreditAI.Infrastructure \
  -s src/CreditAI.API

Health Checks

# API
curl https://localhost:7257/health

# Integrations
curl https://localhost:7257/api/ops/health

Logs

  • Structured logging (JSON)
  • Application Insights (optional)
  • Correlation IDs for tracing

📊 Dashboards and Metrics

Web UI - Dashboards

  • Home: General KPIs, alerts, recent requests
  • Requests: Request list, filters, status
  • Manual Data: Pending data queue
  • Model Registry: Models by segment, backtesting
  • Risk Dashboard: PSI, drift, early warnings
  • Audit Log: Search, export

Key Metrics

  • Pipeline: Average latency, throughput, success rate
  • ML Models: AUC, Gini, Brier, PSI
  • Data Quality: Completeness, freshness
  • Early Warning: Alerts by severity
  • Manual Data: Fallback rate, resolution time

🆘 Troubleshooting

Problem: Pipeline blocks on manual data

Cause: API integration failed
Solution: Analyst fills manual form or uploads document

Problem: OCR does not extract text

Cause: Encrypted document or low quality
Solution: Convert to high-resolution PNG, verify quality

Problem: LLM does not recognize fields

Cause: Unknown document format
Solution: Direct manual entry, adjust prompt if necessary

Problem: Inconsistent score

Cause: Outdated model or drift
Solution: Check Model Registry, backtesting, PSI

Problem: Early Warning does not trigger

Cause: Background job not running
Solution: Check EarlyWarningBatchHostedService logs

📚 Technical References

Clean Architecture

Machine Learning

Compliance

  • Banco de Portugal - Instrução 32/2018
  • EBA Guidelines on ML/AI
  • Basel III - Credit Risk

Frameworks

AI/ML Stack

🗺️ Roadmap

In Production

  • ✅ Complete decision pipeline
  • ✅ 11 integrations (Stub + Production modes)
  • ✅ 6 AI Agents
  • ✅ ML Scoring + SHAP
  • ✅ Manual data entry
  • ✅ OCR + AI document parsing
  • ✅ Early Warning System
  • ✅ Model Risk Management
  • ✅ Immutable audit log

Next Releases

  • 🔄 Azure Document Intelligence (advanced OCR)
  • 🔄 Structured table extraction
  • 🔄 Automatic document classification
  • 🔄 Blazor interface for drag-and-drop upload
  • 🔄 Multi-language (ES, EN, FR)
  • 🔄 Signature recognition
  • 🔄 Executive dashboard (BI)
  • 🔄 API webhooks for external integrations

💬 Contributing

Before contributing, read:

  1. ARCHITECTURE.md - Understand the system
  2. architecture-pipeline.md - Technical details
  3. Existing code - Follow established patterns
  4. Tests - Create tests for new features

Pull Requests:

  • Follow Clean Architecture
  • Add integration tests
  • Update relevant documentation
  • Follow .NET conventions (EditorConfig)

📞 Support

  • Issues: Create issue in repository
  • Documentation: Consult docs/ folder
  • Logs: Check Application Insights or local logs
  • Health: Endpoint /health for status

CreditAI v1.0 - AI-powered Credit Analysis System
© 2026 - Developed with .NET 9, Blazor, PostgreSQL, OpenAI-compatible LLM and ML.NET