Skip to content

built- evaluation agent #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Mohammed-Balkhair-hub merged 1 commit into from
Feb 1, 2026
Merged

built- evaluation agent #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion services/ai-service/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ Thumbs.db

# Model cache (if downloading models)
.cache/
models/

7 changes: 7 additions & 0 deletions services/ai-service/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ python run.py
- **Health**: `GET /health`
- **Setup framework**: `POST /api/v1/frameworks/setup`
- Form fields: `framework_name` (string), `section_names` (list of strings), `files` (list of PDFs). Same order for section_names and files. Each PDF is saved as `config/frameworks/{framework_name}/{section_name}.json`.
- **Submit evaluation**: `POST /api/v1/evaluations/submit`
- Form fields: `framework_name` (string), `files` (1+ uploads: PDF, DOCX, PPTX, CSV, XLSX), `control_ids_1`, `control_ids_2`, ... (one per file; each = comma-separated IDs). Returns mimic JSON, `evaluation_id`, `report_path`, `file_evaluations`. Requires `GROQ_API_KEY` in env.

### Other (from `src`)

Expand All @@ -92,6 +94,11 @@ uv add --group dev pytest

Run these in your terminal; dev dependencies stay in a separate group (e.g. `[project.optional-dependencies.dev]` or `[tool.uv]` dev-dependencies) so production installs stay lean.

## Environment

- **GROQ_API_KEY**: Required for the evaluation agent (`POST /api/v1/evaluations/submit`). Set in `.env` or environment.
- **GROQ_MODEL**: Optional; default `llama-3.3-70b-versatile`.

## Notes

- All data directories are in `.gitignore` (user uploads and generated content)
Expand Down
9 changes: 9 additions & 0 deletions services/ai-service/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,15 @@ dependencies = [
"python-multipart>=0.0.9",
"aiofiles>=24.0.0",
"fastapi>=0.128.0",
"groq>=0.4.0",
"langchain-groq>=0.2.0",
"langgraph>=0.2.0",
"langchain-core>=0.3.0",
"reportlab>=4.0.0",
"python-docx>=1.0.0",
"python-pptx>=0.6.0",
"openpyxl>=3.1.0",
"pandas>=2.0.0",
Comment on lines +23 to +31

@coderabbitai coderabbitai Bot Feb 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check latest versions and verify specified minimums exist on PyPI

packages=(
  "groq:0.4.0"
  "langchain-groq:0.2.0"
  "langgraph:0.2.0"
  "langchain-core:0.3.0"
  "reportlab:4.0.0"
  "python-docx:1.0.0"
  "python-pptx:0.6.0"
  "openpyxl:3.1.0"
  "pandas:2.0.0"
)

for entry in "${packages[@]}"; do
  IFS=':' read -r pkg min_version <<< "$entry"
  echo "=== $pkg (required: >=$min_version) ==="
  response=$(curl -s "https://pypi.org/pypi/$pkg/json" 2>/dev/null)
  if [ $? -eq 0 ] && [ -n "$response" ]; then
    latest=$(echo "$response" | jq -r '.info.version // "Not found"')
    releases=$(echo "$response" | jq -r '.releases | keys[]' 2>/dev/null | grep "^$min_version" | head -1)
    echo "Latest version: $latest"
    if [ -n "$releases" ]; then
      echo "Minimum version $min_version: EXISTS ✓"
    else
      echo "Checking if minimum version exists..."
      all_releases=$(echo "$response" | jq -r '.releases | keys[]' 2>/dev/null | sort -V)
      match=$(echo "$all_releases" | grep "^$(echo $min_version | cut -d. -f1-2)" | head -1)
      if [ -n "$match" ]; then
        echo "Minimum version $min_version: Exists in release history ✓"
      else
        echo "Minimum version $min_version: May not exist ✗"
      fi
    fi
  else
    echo "Error fetching package info"
  fi
  echo ""
done

Repository: AFAskar/Governance-Agent

Length of output: 909

🌐 Web query:

LangChain ecosystem compatibility langchain-core langchain-groq langgraph 0.3.0 0.2.0 breaking changes

💡 Result:

Python (LangChain / LangChain Core) v0.2 → v0.3

  • Only breaking changes in langchain / langchain-core 0.3.x: internal move to Pydantic v2 (Pydantic v1 no longer supported) and Python 3.8 dropped. [1][2]

Python LangGraph v0.2.0 (breaking changes)

LangGraph 0.2.0 introduced multiple breaking changes mostly around checkpointing and stream output:

  • Checkpointers split into separate namespace packages (langgraph-checkpoint-*) and re-export imports removed (import paths changed). [3]
  • thread_ts / parent_ts renamed to checkpoint_id / parent_checkpoint_id. [3]
  • SQLite saver .from_conn_string became a context manager. [3]
  • Stream output now includes all nodes, even those that don’t write to state. [3]

Python LangGraph v0.3.0 (breaking changes)

LangGraph 0.3.0 breaking changes:

  • langgraph.prebuilt was moved into a separate package: langgraph-prebuilt (you must pip install langgraph-prebuilt to keep using from langgraph.prebuilt import ...). [4]
  • Removed messages_modifier param for create_react_agent (use prompt instead). [4]
  • Removed langgraph.prebuilt.tool_executor (use ToolNode). [4]

Compatibility guidance (the practical “what versions work together”)

  • If you’re on LangChain / langchain-core 0.3.x, the LangChain migration docs specifically recommend staying on LangGraph >=0.2.20,<0.3 (i.e., LangGraph 0.2.x), and note it works with either 0.2 or 0.3 base LangChain packages. [5]
  • Moving to LangGraph 0.3.x is where you’ll hit the biggest friction (the prebuilt split to langgraph-prebuilt). [4]

LangGraph.js note (because your query mentions 0.3.0 / 0.2.0)

For LangGraph.js, the project’s “Over Time” page lists v0.2.0 → v0.3.0 breaking changes like interrupt propagation changes and stricter .stream() typing; and separately notes that upgrading to v0.2.0 required explicitly installing @langchain/core. [6]

Sources:
[1] https://blog.langchain.com/announcing-langchain-v0-3/
[2] https://python.langchain.com/docs/versions/v0_3/
[3] https://newreleases.io/project/github/langchain-ai/langgraph/release/0.2.0
[4] https://newreleases.io/project/github/langchain-ai/langgraph/release/0.3.0
[5] https://python.langchain.com/docs/versions/v0_3/
[6] https://langchain-ai.lang.chat/langgraphjs/versions/

Tighten version constraints for LangChain ecosystem compatibility and add Pydantic v2 as explicit dependency.

The specified versions have documented incompatibilities. langchain-core>=0.3.0 requires Pydantic v2 (v1 no longer supported) but Pydantic is not listed as a dependency. Additionally, langgraph>=0.2.0 is too permissive—versions 0.2.0 and 0.3.0 both introduce breaking changes, and the LangChain migration docs recommend staying on langgraph>=0.2.20,<0.3 when using langchain-core>=0.3.0 to avoid the langgraph-prebuilt split and other breaking changes in 0.3.0.

Recommended changes:

  • Add "pydantic>=2.0.0" as explicit dependency
  • Change "langgraph>=0.2.0" to "langgraph>=0.2.20,<0.3.0" to prevent compatibility issues
🤖 Prompt for AI Agents 
In `@services/ai-service/pyproject.toml` around lines 23 - 31, The dependency list
lacks an explicit Pydantic v2 requirement and has a too-permissive langgraph
range that can break compatibility with langchain-core>=0.3.0; update the
dependencies in pyproject.toml by adding "pydantic>=2.0.0" and replace the
"langgraph>=0.2.0" entry with a pinned compatibility range
"langgraph>=0.2.20,<0.3.0" so langchain-core, langgraph, and related packages
(groq, langchain-groq, langchain-core, langgraph) remain compatible.

]

[dependency-groups]
Expand Down
Loading