test(ocr): add degraded regression dataset + improve rotation/low-contrast OCR

.github/workflows/ocr-regression-degraded.yml

@@ -0,0 +1,82 @@
+name: OCR Regression Test (Degraded)
+
+on:
+  push:
+    paths:
+      - 'app/ai-service/services/ocr.py'
+      - 'app/ai-service/services/preprocessing.py'
+      - 'app/ai-service/regression_harness/dataset/degraded/**'
+      - 'app/ai-service/regression_harness/**'
+    branches: [ main, develop ]
+  pull_request:
+    paths:
+      - 'app/ai-service/services/ocr.py'
+      - 'app/ai-service/services/preprocessing.py'
+      - 'app/ai-service/regression_harness/dataset/degraded/**'
+      - 'app/ai-service/regression_harness/**'
+    branches: [ main ]
+  workflow_dispatch:
+
+jobs:
+  regression-degraded:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+          cache: 'pip'
+
+      - name: Install System Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y tesseract-ocr libtesseract-dev
+
+      - name: Install Python Dependencies
+        working-directory: ./app/ai-service
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install Pillow pytesseract
+
+      - name: Run OCR Regression Harness (degraded)
+        working-directory: ./app/ai-service
+        run: |
+          set -euo pipefail
+          export PYTHONPATH=$PYTHONPATH:.
+          python regression_harness/cli.py \
+            --dataset regression_harness/dataset/degraded/ground_truth.json \
+            --output ocr_degraded_report.json \
+            --threshold 0.8
+          python - <<'PYTHON_SCRIPT'
+          import json
+          with open('ocr_degraded_report.json', 'r') as f:
+              report = json.load(f)
+          summary = report.get('summary', {})
+          total = summary.get('total', 0)
+          passed = summary.get('passed', 0)
+          accuracy = float(summary.get('accuracy', 0.0))
+          pass_ratio = (passed / total) if total else 0.0
+          print('Degraded regression summary:', {
+              'total': total,
+              'passed': passed,
+              'pass_ratio': pass_ratio,
+              'accuracy': accuracy
+          })
+          if pass_ratio < 0.9:
+              raise SystemExit('FAILED: pass_ratio {:.3f} < 0.9'.format(pass_ratio))
+          if accuracy < 60.0:
+              raise SystemExit('FAILED: accuracy {:.3f}% < 60.0%'.format(accuracy))
+          PYTHON_SCRIPT
+
+      - name: Upload Regression Report
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ocr-regression-degraded-report
+          path: app/ai-service/ocr_degraded_report.json
+          retention-days: 14

app/ai-service/conftest.py

@@ -39,7 +39,7 @@ def _make_pkg(name: str):
         has_pkg = spec is not None
     except Exception:
         has_pkg = False
-       
+
     if not has_pkg:
         if _mod not in sys.modules:
             sys.modules[_mod] = _make_pkg(_mod)
@@ -52,9 +52,32 @@ def _make_pkg(name: str):
 
 # Patch metrics.check_system_resources so the monitor_requests middleware
 # doesn't crash when torch (vram) is a MagicMock.
-import metrics
+import metrics  # type: ignore
 metrics.check_system_resources = lambda **kwargs: True
 
+# Ensure cv2 mocks return realistic numpy arrays for the preprocessing pipeline.
+import numpy as np
+import cv2 as _cv2  # type: ignore
+if isinstance(_cv2, MagicMock):
+    # CLAHE mock
+    _clahe_mock = MagicMock()
+    _clahe_mock.apply = MagicMock(side_effect=lambda arr: arr.astype(np.uint8) if hasattr(arr, 'astype') else np.zeros((100, 100), dtype=np.uint8))
+    _cv2.createCLAHE = MagicMock(return_value=_clahe_mock)
+
+    # Threshold mocks
+    _dummy_thresh = np.zeros((100, 100), dtype=np.uint8)
+    _cv2.threshold = MagicMock(return_value=(127.0, _dummy_thresh))
+    _cv2.adaptiveThreshold = MagicMock(return_value=_dummy_thresh)
+
+    # Morphology mock
+    _cv2.MORPH_CLOSE = 2
+    _cv2.morphologyEx = MagicMock(return_value=_dummy_thresh)
+
+    # Denoising mock
+    _cv2.fastNlMeansDenoisingColored = MagicMock(return_value=_dummy_thresh)
+    _cv2.cvtColor = MagicMock(return_value=_dummy_thresh)
+    _cv2.COLOR_GRAY2BGR = 0
+    _cv2.COLOR_BGR2GRAY = 1
 
 def pytest_terminal_summary(terminalreporter):
     try:

app/ai-service/regression_harness/cli.py

@@ -1,7 +1,15 @@
 import os
+import sys
 import json
 import argparse
 from typing import List
+# Ensure this script can be executed directly regardless of CWD / PYTHONPATH.
+# When running as: python app/ai-service/regression_harness/cli.py
+# we want to treat `app/ai-service` as the import root.
+import_path_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+if import_path_root not in sys.path:
+    sys.path.insert(0, import_path_root)
+
 from regression_harness.models import EvaluationSample, BoundingBox
 from regression_harness.evaluator import OCREvaluator
 
@@ -54,11 +62,13 @@ def main():
     parser.add_argument("--dataset", default="regression_harness/dataset/ground_truth.json", help="Path to ground truth JSON")
     parser.add_argument("--output", help="Path to save JSON report")
     parser.add_argument("--threshold", type=float, default=0.8, help="Confidence threshold")
-
+    parser.add_argument("--min_pass_ratio", type=float, default=None, help="If set, CI can enforce minimum pass ratio (0-1).")
+
     args = parser.parse_args()
 
     base_dir = os.path.dirname(os.path.abspath(__file__))
-    # Adjust base_dir if it's currently inside regression_harness
+    # Ensure args.dataset paths work regardless of where this script is run from.
+    # Default expects to be relative to app/ai-service.
     if base_dir.endswith("regression_harness"):
         base_dir = os.path.dirname(base_dir)
         # We want base_dir to be app/ai-service
@@ -81,7 +91,12 @@ def main():
             json.dump(report.to_dict(), f, indent=2)
         print(f"Report saved to {args.output}")
 
-    if report.failed_samples > 0:
+    if args.min_pass_ratio is not None:
+        pass_ratio = (report.passed_samples / report.total_samples) if report.total_samples > 0 else 0
+        print(f"Min pass ratio requirement: {args.min_pass_ratio:.2f}, actual: {pass_ratio:.2f}")
+        if pass_ratio < args.min_pass_ratio:
+            exit(1)
+    elif report.failed_samples > 0:
         exit(1)
 
 if __name__ == "__main__":

app/ai-service/regression_harness/dataset/degraded/README.md

@@ -0,0 +1,12 @@
+# Degraded regression dataset
+
+This dataset contains intentionally degraded versions of the golden `sample_001.png` fixture to validate OCR robustness against:
+- 90° rotations
+- low contrast
+- blur
+- low resolution
+- a faint watermark overlay
+
+Images live in `documents/`.
+Ground truth lives in `ground_truth.json`.
+

app/ai-service/regression_harness/dataset/degraded/TODO.md

@@ -0,0 +1,44 @@
+# Degraded OCR Regression - Fix Summary
+
+## Changes Made
+
+### 1. Degraded Dataset
+- Created `app/ai-service/regression_harness/dataset/degraded/ground_truth.json` - 12 samples
+- Generated 12 degraded variants in `degraded/documents/`:
+  - `sample_001_orig.png` - baseline
+  - `sample_001_rot90/180/270.png` - rotated
+  - `sample_001_lowc1/2.png` - low contrast
+  - `sample_001_blur2/4_lowc.png` - blur + low contrast
+  - `sample_001_lowres/lowres2.png` - low resolution
+  - `sample_001_watermark30/60.png` - watermark overlay
+
+### 2. Preprocessing Improvements (`preprocessing.py`)
+- Added CLAHE contrast normalization before thresholding
+- Added morphological closing (MORPH_CLOSE) for blur robustness
+
+### 3. OCR Rotation Sweep (`ocr.py`)
+- Added `_try_orientation()` method for evaluating OCR at any angle
+- Orientation sweep across [0, 90, 180, 270] degrees
+- Picks best candidate by (field_count, total_confidence)
+
+### 4. CI Workflow
+- Created `.github/workflows/ocr-regression-degraded.yml` - runs on pushes/PRs
+- Enforces pass_ratio >= 0.9 and accuracy >= 60.0%
+
+### 5. Test Fixes
+- Fixed `test_ocr.py` mock assertion to expect >= 5 metric observations
+- Fixed `conftest.py` to properly mock `cv2.createCLAHE` and `cv2.morphologyEx`
+
+### 6. CLI Enhancement (`cli.py`)
+- Added `--min_pass_ratio` flag for CI enforcement
+
+## Remaining Issues to Fix
+
+1. The standard OCR regression workflow (non-degraded) may time out due to 4x Tesseract calls per image - consider adding a cache or reducing sweep size for the default dataset
+2. The `cv2` mock in `conftest.py` needs to return proper numpy arrays so preprocessor tests pass in CI
+
+## CI Checks Status
+- AI Service CI (build, docker-build, lint, security-scan, test) - ✅ all passing
+- CI Python Tests - ❌ need to verify mock fixes work
+- OCR Regression Test - ❌ need to verify standard dataset works with sweep
+- OCR Regression Test (Degraded) - ❌ need to verify thresholds met