Feature/ai/rag & prompt extraction - #2
Conversation
chunk all frameworks files to build rag + extract controls from chosen framewirk files.
refine controls extractor
enhance controls extraction + leverage controls in rag chunks to optimize retrieval for evaluation agent for later
📝 WalkthroughWalkthroughThis PR introduces a complete AI service for compliance framework evaluation with PDF extraction, LLM-based control extraction, vector embeddings, and retrieval-augmented generation (RAG). It includes API and CLI entry points, comprehensive documentation, and supporting infrastructure for managing frameworks and evaluating applicants. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant API as FastAPI
participant FrameworkSvc as FrameworkService
participant PDFParser as PDF Parser
participant Extractor as LLM Extractor
participant Storage as File Storage
participant RAG as RAG Indexer
Client->>API: POST /api/v1/frameworks/setup<br/>(framework_name, files)
API->>FrameworkSvc: setup_framework(name, pdf_sections)
FrameworkSvc->>PDFParser: extract_text_from_pdf()
PDFParser-->>FrameworkSvc: pdf_text
FrameworkSvc->>Extractor: extract_controls_from_pdfs()<br/>(with OpenRouter API)
Extractor-->>FrameworkSvc: controls_json
FrameworkSvc->>Storage: save_extraction_json()
Storage-->>FrameworkSvc: json_paths
FrameworkSvc->>RAG: index_framework()<br/>(load JSONs & PDFs)
RAG->>RAG: chunk & embed documents
RAG->>RAG: store in Qdrant
RAG-->>FrameworkSvc: indexing complete
FrameworkSvc-->>API: result summary
API-->>Client: SetupFrameworkResponse
sequenceDiagram
participant Client as Agent/Client
participant API as FastAPI
participant RAG as retrieve_control_details()
participant QdrantDB as Qdrant Vector DB
participant Embedder as GemmaEmbedder
Client->>API: retrieve_control_details(control_id, framework)
API->>RAG: retrieve_control_details()
RAG->>QdrantDB: fetch JSON cards by filter<br/>(control_id, source=json)
QdrantDB-->>RAG: json_cards (metadata & text)
RAG->>Embedder: embed(description) for semantic search
Embedder-->>RAG: query_embedding
RAG->>QdrantDB: search_similar_filtered()<br/>(embedding, source=pdf)
QdrantDB-->>RAG: top_k pdf_chunks<br/>(text, score, metadata)
RAG-->>API: {json_cards, pdf_chunks}
API-->>Client: retrieval result
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Tip 🧪 Unit Test Generation v2 is now available!We have significantly improved our unit test generation capabilities. To enable: Add this to your reviews:
finishing_touches:
unit_tests:
enabled: trueTry it out by using the Have feedback? Share your thoughts on our Discord thread! Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 18
🤖 Fix all issues with AI agents
In `@services/ai-service/main.py`:
- Around line 51-65: The file defines main() but never calls it; instead the if
__name__ == "__main__" block directly invokes setup_framework() with hardcoded
paths. Fix by consolidating startup logic into main() (use get_input_paths(),
print messages and call setup_framework as appropriate) and replace the direct
call in the __main__ block with a single call to main(); alternatively remove
the unused main() if you prefer the current inline behavior—update the __main__
block and/or main() so only one entrypoint (main or the inline setup_framework
call) remains.
- Line 61: The print statement print(f" Framework outputs: config/frameworks/")
uses an unnecessary f-string; replace it with a normal string literal by
removing the leading 'f' so it becomes print(" Framework outputs:
config/frameworks/") — locate the exact print call (print(f" Framework outputs:
config/frameworks/")) in main.py and update it accordingly.
In `@services/ai-service/pyproject.toml`:
- Line 4: Replace the placeholder description value in pyproject.toml (the
description = "Add your description here" entry) with a concise, meaningful
string that describes the AI service's purpose (e.g., what the service does, its
domain or primary feature). Update the description field only, keeping TOML
syntax intact and using double quotes, and ensure the new text succinctly
summarizes the project for package metadata consumers.
In `@services/ai-service/README.md`:
- Around line 9-32: Update the two fenced code blocks showing the directory
trees in README.md so they include a language identifier (`text`) after the
opening triple backticks; specifically add ```text to the block starting with
the `data/` tree and the block starting with the `config/` tree (the blocks that
show the `data/ inputs/ frameworks/ applicants/` tree and the `config/
frameworks/` / `data/ outputs/ evaluations/` tree) to satisfy markdownlint MD040
and improve rendering.
In `@services/ai-service/src/api/README.md`:
- Line 14: Update the README run instruction to use the correct repository path:
replace the "From `ai-service/`" text with "From `services/ai-service/`" in the
Run line in README.md so the example command (`python src/api/app.py` or `python
run.py`) accurately reflects the service location; verify both docs endpoints
`/api/docs` and `/api/redoc` remain unchanged.
In `@services/ai-service/src/api/routers/frameworks.py`:
- Around line 25-36: Before processing, check that section_names_list contains
only unique values and reject duplicates: after computing section_names_list
(used later as JSON filenames by save_extraction_json) validate
len(set(section_names_list)) == len(section_names_list) and if not raise an
HTTPException(400) with a clear message about duplicate section names; do this
check prior to the loop that zips section_names_list with files (and before any
file I/O) so duplicates cannot overwrite prior outputs.
In `@services/ai-service/src/ARCHITECTURE.md`:
- Around line 16-24: The table separator rows directly under the header row "|
Module | Purpose |" are missing spaces around the pipe characters and violate
markdownlint MD060; update those separator lines (the row of hyphens) to include
spaces around each pipe (e.g., "| --- | --- |") so they match the header
spacing, and apply the same fix to the other tables referenced later (the tables
around lines 342–376) to ensure consistent spacing across ARCHITECTURE.md.
- Around line 32-70: The fenced directory-structure code block in
ARCHITECTURE.md triggers markdownlint MD040 because it lacks a language; update
the opening fence from ``` to ```text so the block is explicitly marked as plain
text and keep the closing ``` unchanged, ensuring the directory tree content
(the src/ tree) is unchanged.
In `@services/ai-service/src/core/evaluator.py`:
- Around line 94-110: The JSON parsing in the try/except block of
services/ai-service/src/core/evaluator.py should be hardened so malformed or
unterminated fenced JSON doesn't raise and bypass the fallback; in the except
handler for json.JSONDecodeError when "```json" is present, extract the fenced
block defensively (use json_end != -1 to decide whether to slice to the closing
fence or to the end), then attempt a json.loads inside its own try/except and on
failure return the text fallback with an error message (instead of letting the
inner json.loads propagate); ensure the code paths around result_text,
json_start/json_end and the returned dict for the text fallback are updated
accordingly within the parsing logic in the function handling evaluation_report.
- Around line 61-75: The OpenAI request in evaluator.py uses
client.chat.completions.create (where response is assigned) without a timeout;
update the code to set a timeout either when creating the OpenAI client (pass
timeout=30.0 to the client constructor) or on this call by using
client.with_options(timeout=30.0).chat.completions.create so the chat completion
request for model "openai/gpt-4.1" will time out (e.g., 30s) and avoid hanging
workers.
In `@services/ai-service/src/core/framework_extraction.py`:
- Around line 143-156: The truncated-JSON recovery in framework_extraction.py
(the block using last_brace/last_bracket and json.loads) must validate the
parsed object shape before returning; after parsing into result_json, verify it
is a dict with keys like "framework_name" (string or match variable
framework_name) and "controls" (a list), and that each control entry has the
expected fields/types (e.g., id/name/details) — if validation fails, treat as a
parse failure and return {"framework_name": framework_name, "controls": []};
place this validation immediately after the json.loads call inside the try in
the function handling extraction to avoid accepting semantically incomplete
results.
In `@services/ai-service/src/embeddings/gemma_embedder.py`:
- Line 10: The module-level assignment os.environ["HF_HUB_OFFLINE"] = "1" causes
a global side effect; update GemmaEmbedder to accept an offline flag (e.g.,
offline param on __init__ or a DEFAULT_OFFLINE class attribute) and only
set/restore HF_HUB_OFFLINE around model download/loading in the method that
loads the model (e.g., the constructor or load_model function) so the
environment change is temporary and scoped to GemmaEmbedder; ensure you restore
the previous env value after loading to avoid affecting other code.
In `@services/ai-service/src/embeddings/haystack_retriever.py`:
- Around line 43-48: The scroll call in haystack_retriever.py uses a hardcoded
limit=10000 on qdrant_client.scroll which can truncate results; update the code
in the method that calls qdrant_client.scroll (look for the scroll_result
variable and collection_name usage) to paginate properly by looping and using
the offset/next_offset returned by scroll() (or repeatedly calling scroll with
updated offset until no more results) instead of a single large fixed limit, or
alternatively expose/configure the page size and document this limitation.
- Around line 40-63: The _load_documents_from_qdrant method swallows all
exceptions and populates self.documents which is never used by
retrieve_documents, causing silent failures and confusion; either remove the
unused method or make it meaningful: catch exceptions from qdrant_client.scroll
but log the full exception (use self.logger or a module logger) before
clearing/setting self.documents, and update retrieve_documents to use
self.documents (or conversely remove _load_documents_from_qdrant if you prefer
direct queries). Locate _load_documents_from_qdrant, qdrant_client.scroll, and
retrieve_documents to implement logging (including exception details) or delete
the unused method so behavior is consistent.
In `@services/ai-service/src/embeddings/qdrant_manager.py`:
- Around line 57-66: The current bare except around client.get_collection masks
real errors; update the try/except in qdrant_manager.py so you catch ValueError
(embedded/local client missing collection) and
qdrant_client.http.exceptions.UnexpectedResponse (remote client) specifically,
and only create the collection when ValueError or an UnexpectedResponse with
status_code == 404 is encountered; re-raise any other UnexpectedResponse or
unexpected exceptions so failures aren't hidden. Locate the try around
client.get_collection and the subsequent client.create_collection call
(referencing collection_name, vector_size, VectorParams, Distance) and implement
the two-case exception handling described.
In `@services/ai-service/src/services/framework_service.py`:
- Line 66: The loop using zip(section_names_order, controls_arrays) can silently
truncate if the two iterables differ in length; update that call to
zip(section_names_order, controls_arrays, strict=True) so Python raises a
ValueError on length mismatch. Locate the for loop that currently reads "for
section_name, controls_array in zip(section_names_order, controls_arrays):" in
framework_service.py and change it to include strict=True; if any surrounding
code relies on older Python versions, ensure the runtime supports Pythons with
zip(..., strict=True) or add an explicit length check (len(section_names_order)
!= len(controls_arrays)) before iterating.
In `@services/ai-service/src/utils/framework_utils.py`:
- Around line 96-100: The combined glob calls for "*.pdf" and "*.PDF" can
produce duplicate Path objects on case-insensitive filesystems; update the logic
in the block that builds pdfs (the variables and expressions using sub.is_dir(),
sub.glob("*.pdf"), sub.glob("*.PDF"), and vdb.glob(...)) to deduplicate results
after collecting both patterns—e.g., normalize each Path (resolve() or use
name.lower()) and keep a single entry per real file while preserving sort/order;
apply the same deduping for both the sub directory branch and the vdb fallback
so pdfs contains unique files only.
In `@services/ai-service/test.ipynb`:
- Around line 9-60: The notebook contains committed cell outputs leaking a local
path and extracted control text; clear all outputs before committing by running
a notebook-output clear (e.g., jupyter nbconvert --clear-output --inplace
<notebook> or use the Jupyter UI) and remove the printed results from the cell
that calls retrieve_control_details (the cell that invoked
retrieve_control_details("DG.1", "NDI", top_k_pdf=5) and references
RETRIEVE_CONTROL_DETAILS_TOOL_SCHEMA), then recommit the cleaned notebook.
🧹 Nitpick comments (21)
services/ai-service/pyproject.toml (2)
13-13: Moveipykernelto dev dependencies.
ipykernelis a Jupyter kernel interface used for interactive notebook development. It should not be a runtime dependency as it's not needed in production.♻️ Proposed fix
Remove from runtime dependencies:
"pypdf>=3.0.0", - "ipykernel>=7.1.0", "qdrant-client>=1.8.0",Add to dev group:
[dependency-groups] dev = [ "fastapi[standard]>=0.128.0", + "ipykernel>=7.1.0", ]
22-28: Consolidate FastAPI dependencies to avoid redundancy.
fastapiis in main dependencies andfastapi[standard]is in dev dependencies. The[standard]extra includes uvicorn, httpx, and CLI tooling. Consider either:
- Use
fastapi[standard]in main dependencies if you need uvicorn for production.- Or explicitly add only
uvicornto dev for development server needs.♻️ Option 1: Use fastapi[standard] in main dependencies
- "fastapi>=0.128.0", + "fastapi[standard]>=0.128.0", ] [dependency-groups] dev = [ - "fastapi[standard]>=0.128.0", + "ipykernel>=7.1.0", ]services/ai-service/src/processing/pdf_parser.py (1)
37-38: Silent exception swallowing may hide extraction issues.Catching all exceptions and continuing without any logging makes debugging difficult. Consider logging a warning when a page fails to extract so operators can identify problematic PDFs.
🔧 Proposed fix to add logging
+import logging + +logger = logging.getLogger(__name__) +Then in the extraction loop:
try: page_text = page.extract_text() if page_text: text_content.append(page_text) - except Exception: + except Exception as e: + logger.warning("Failed to extract text from page %d of %s: %s", page.page_number, pdf_path, e) continueservices/ai-service/src/embeddings/gemma_embedder.py (1)
66-90: Add exception chaining for better tracebacks.Use
raise ... from eto preserve the original exception context, which helps with debugging.🔧 Proposed fix
- raise RuntimeError( + raise RuntimeError( f"Failed to load model: {error_msg}\n\n" # ... message continues ... " - Set environment variable: export HF_TOKEN=your_token" - ) + ) from e - raise RuntimeError(f"Failed to load model: {e}") + raise RuntimeError(f"Failed to load model: {e}") from eservices/ai-service/src/utils/framework_utils.py (2)
11-12: Duplicate_project_root()definition.This function is also defined in
services/ai-service/src/services/framework_service.py(lines 16-17). Consider extracting it to a shared location to avoid duplication.
81-82: Silent exception swallowing hides JSON parsing errors.When a JSON file fails to parse, the error is silently ignored. This could hide issues like malformed JSON or encoding problems.
🔧 Proposed fix to add logging
+import logging + +logger = logging.getLogger(__name__)Then:
try: with open(p, "r", encoding="utf-8") as f: data = json.load(f) out.append((p.stem, data)) - except Exception: + except Exception as e: + logger.warning("Failed to load JSON file %s: %s", p, e) continueservices/ai-service/src/processing/text_chunker.py (1)
191-234: Remove or deprecate unused function.The comment indicates
chunk_text_by_sentencesis "not used anymore," yet it's still exported in__init__.pyand documented inREADME.md. Either remove it entirely or add a proper deprecation warning if it needs to be retained for backward compatibility.🔧 Option 1: Remove the function
Delete lines 191-234 and remove
chunk_text_by_sentencesfromprocessing/__init__.pyandREADME.md.🔧 Option 2: Add deprecation warning
+import warnings + # not used anymore def chunk_text_by_sentences( text: str, sentences_per_chunk: int = 5, framework_name: Optional[str] = None ) -> List[Dict]: """ Chunk text by sentences instead of fixed character size. + + .. deprecated:: + This function is deprecated. Use :func:`chunk_text` instead. Args: text: Text to chunk sentences_per_chunk: Number of sentences per chunk framework_name: Optional framework name for metadata Returns: List of chunk dictionaries """ + warnings.warn( + "chunk_text_by_sentences is deprecated, use chunk_text instead", + DeprecationWarning, + stacklevel=2 + ) # Split into sentencesservices/ai-service/src/core/framework_extraction.py (2)
200-223: Unbounded thread pool size can exhaust resources.Setting
max_workers=len(pdf_paths_list)creates one thread per PDF. For large batches, this could exhaust system resources or hit API rate limits simultaneously.Proposed fix to cap thread pool size
- with ThreadPoolExecutor(max_workers=len(pdf_paths_list)) as executor: + max_workers = min(len(pdf_paths_list), 5) # Cap concurrent LLM calls + with ThreadPoolExecutor(max_workers=max_workers) as executor: controls_arrays = list(executor.map(extract_pdf, pdf_paths_list))
219-223: Move success log toelseblock per static analysis hint.The print statement on Line 219 executes even when retries were exhausted but zero controls remain. Moving it to an
elseblock ensures it only logs on genuine success.Proposed fix
while len(controls) == 0 and retries < MAX_RETRIES: retries += 1 print(f"Empty controls for {pdf_path}, retry {retries}/{MAX_RETRIES} with fallback prompt") controls_json = extract_controls_from_framework( pdf_text, framework_name, use_fallback_prompt=True ) controls = controls_json.get("controls", []) - print(f"Controls extracted from {pdf_path}") - return controls + else: + print(f"Controls extracted from {pdf_path}") + return controls + return controls except Exception as e: print(f"Error extracting from {pdf_path}: {e}") return []services/ai-service/src/embeddings/qdrant_manager.py (2)
111-115: Addstrict=Truetozip()to catch length mismatches early.Although Line 108-109 validates lengths, adding
strict=Trueprovides defense-in-depth and satisfies linter B905.Proposed fix
- for doc, embedding in zip(documents, embeddings): + for doc, embedding in zip(documents, embeddings, strict=True):
196-217: Chain exceptions withraise ... from efor better tracebacks.Multiple
raise RuntimeError(...)statements lose the original exception context. Usingraise ... from epreserves the chain for debugging.Proposed fix for exception chaining
except AttributeError as e: error_msg = str(e) raise RuntimeError( f"Qdrant client method error: {error_msg}. " ... - ) + ) from e except Exception as e: error_msg = str(e) if "gRPC" in error_msg or "grpc" in error_msg.lower(): raise RuntimeError( ... - ) + ) from e raise RuntimeError( ... - ) + ) from eservices/ai-service/src/embeddings/haystack_retriever.py (1)
87-88: Move import to module level to avoid repeated import overhead.The import inside the method is executed on every call. Unless this is intentional to avoid circular imports, move it to the top of the file.
Proposed fix
from haystack.dataclasses import Document from qdrant_client import QdrantClient from typing import List, Dict, Optional from .gemma_embedder import GemmaEmbedder +from .qdrant_manager import search_similar ... def retrieve_documents(self, query: str, top_k: Optional[int] = None) -> List[Dict]: ... - # Search in Qdrant directly (more efficient than Haystack for this use case) - from .qdrant_manager import search_similar - results = search_similar(services/ai-service/src/rag/_shared.py (1)
13-18: Singleton pattern is appropriate but not thread-safe.If multiple threads call
get_shared_embedder()simultaneously before initialization completes, multiple embedder instances could be created (race condition). For a single-threaded FastAPI/CLI context this is fine, but worth noting if concurrency increases.Thread-safe alternative using threading.Lock
from typing import Any, Optional +import threading from src.embeddings import load_gemma_embedder _cached_embedder: Optional[Any] = None +_lock = threading.Lock() def get_shared_embedder(): """Return a single embedder instance, creating and caching on first use.""" global _cached_embedder - if _cached_embedder is None: - _cached_embedder = load_gemma_embedder() + if _cached_embedder is None: + with _lock: + if _cached_embedder is None: # Double-checked locking + _cached_embedder = load_gemma_embedder() return _cached_embedderservices/ai-service/src/rag/retrieval.py (2)
73-76: Qdrant client initialized on every call; consider caching.Each invocation of
retrieve_control_detailscreates a new Qdrant client and potentially re-creates the collection. For frequently-called agent tools, this adds overhead. Consider caching the client similarly to how the embedder is cached.Proposed caching approach
+from functools import lru_cache + +@lru_cache(maxsize=16) +def _get_qdrant_client(collection: str, dim: int): + return initialize_qdrant(collection_name=collection, vector_size=dim) + def retrieve_control_details( control_id: str, framework_name: str, *, top_k_pdf: int = 5, ) -> Dict[str, Any]: ... embedder = get_shared_embedder() collection = f"{framework_name}_rag" dim = embedder.get_embedding_dim() - client = initialize_qdrant(collection_name=collection, vector_size=dim) + client = _get_qdrant_client(collection, dim)
52-109: No error handling for missing collection or embedder failures.If the collection doesn't exist or the embedder fails to load, the function will raise an unhandled exception. For an agent tool, returning a structured error response may be preferable.
Example error handling wrapper
def retrieve_control_details( control_id: str, framework_name: str, *, top_k_pdf: int = 5, ) -> Dict[str, Any]: + try: embedder = get_shared_embedder() collection = f"{framework_name}_rag" ... return { "json_cards": json_cards, "pdf_chunks": pdf_chunks, } + except Exception as e: + return { + "error": str(e), + "json_cards": [], + "pdf_chunks": [], + }services/ai-service/src/rag/ingestion.py (1)
65-68: Log PDF extraction errors instead of silently continuing.Bare exception handling without logging makes missing or corrupt PDFs difficult to detect and debug. Catch only the specific exceptions that
extract_text_from_pdfcan raise (FileNotFoundErrorandValueError) and log them at warning level:Suggested adjustment
+import logging + +logger = logging.getLogger(__name__) + for pdf_path in pdf_paths: try: raw = extract_text_from_pdf(pdf_path) - except Exception: + except (FileNotFoundError, ValueError) as exc: + logger.warning("Skipping PDF %s: %s", pdf_path, exc) continueservices/ai-service/run.py (1)
1-3: Docstring mentions--reloadbut the script doesn't support it.The docstring suggests using
--reloadfor auto-restart, butuvicorn.run()is called without parsing CLI arguments. Either update the docstring or add argument parsing.Additionally,
uvicornshould be explicitly declared in dependencies if not already present viafastapi[standard].💡 Option to enable reload support
-"""Run the API: python run.py (from ai-service directory). Use --reload for auto-restart on code changes.""" +"""Run the API: python run.py (from ai-service directory).""" import uvicorn -uvicorn.run("src.api.app:app", host="0.0.0.0", port=8000) +uvicorn.run("src.api.app:app", host="0.0.0.0", port=8000, reload=True)Or for production without reload:
-"""Run the API: python run.py (from ai-service directory). Use --reload for auto-restart on code changes.""" +"""Run the API: python run.py (from ai-service directory)."""services/ai-service/src/api/app.py (2)
18-24: CORS configuration is development-only.The allowed origins are hardcoded to localhost. Consider using environment variables for production flexibility.
💡 Environment-based CORS configuration
+import os + +ALLOWED_ORIGINS = os.getenv( + "CORS_ORIGINS", "http://localhost:3000,http://127.0.0.1:3000" +).split(",") + app.add_middleware( CORSMiddleware, - allow_origins=["http://localhost:3000", "http://127.0.0.1:3000"], + allow_origins=ALLOWED_ORIGINS, allow_credentials=True, allow_methods=["*"], allow_headers=["*"], )
42-51: GlobalValueErrorhandler may mask programming errors.Catching all
ValueErrorexceptions globally could hide bugs in application code that raiseValueErrorunintentionally. Consider using a custom exception class for input validation errors instead.💡 Use a custom validation exception
+class ValidationError(Exception): + """Raised for invalid user input.""" + def __init__(self, message: str): + self.message = message + super().__init__(message) + -@app.exception_handler(ValueError) -async def value_error_handler(request, exc: ValueError): +@app.exception_handler(ValidationError) +async def validation_error_handler(request, exc: ValidationError): return JSONResponse( status_code=400, content={ "error": "bad_request", - "message": str(exc), + "message": exc.message, "detail": None, }, )Then raise
ValidationErrorin routers instead ofValueError.services/ai-service/src/services/framework_service.py (2)
88-91: Unused loop variablesection_nameshadows outer scope.The loop unpacks
section_namebut only usescontentandsafe_name. Use_for the unused variable to improve clarity.🔧 Proposed fix
- for (section_name, content), safe_name in zip(pdf_sections, section_names_order): + for (_, content), safe_name in zip(pdf_sections, section_names_order, strict=True): out_pdf = vector_db_dir / f"{safe_name}.pdf" out_pdf.write_bytes(content) pdf_paths_for_rag.append(str(out_pdf))
98-98:datetime.utcnow()is deprecated.
datetime.utcnow()is deprecated since Python 3.12. Usedatetime.now(timezone.utc)instead for timezone-aware UTC timestamps.🔧 Proposed fix
-from datetime import datetime +from datetime import datetime, timezone ... - "created_at": datetime.utcnow(), + "created_at": datetime.now(timezone.utc),
| def main(): | ||
| """Main entry point.""" | ||
| print("\n" + "=" * 60) | ||
| print("Compliance Framework Extraction System") | ||
| print("=" * 60) | ||
|
|
||
| paths = get_input_paths() | ||
| print("\nDirectory structure:") | ||
| print(f" Input PDFs (Frameworks): {paths['frameworks']}") | ||
| print(f" Input PDFs (Applicants): {paths['applicants']}") | ||
| print(f" Framework outputs: config/frameworks/") | ||
|
|
||
| print("\nUsage:") | ||
| print(" setup_framework('data/inputs/frameworks/<dir>', 'framework_name')") | ||
| print("=" * 60 + "\n") |
There was a problem hiding this comment.
main() function is defined but not called.
The __main__ block directly calls setup_framework() with hardcoded paths instead of calling main(). Either call main() or remove it if unused.
🔧 Option 1: Call main() and move setup logic there
def main():
"""Main entry point."""
print("\n" + "=" * 60)
print("Compliance Framework Extraction System")
print("=" * 60)
paths = get_input_paths()
print("\nDirectory structure:")
print(f" Input PDFs (Frameworks): {paths['frameworks']}")
print(f" Input PDFs (Applicants): {paths['applicants']}")
- print(f" Framework outputs: config/frameworks/")
+ print(" Framework outputs: config/frameworks/")
print("\nUsage:")
print(" setup_framework('data/inputs/frameworks/<dir>', 'framework_name')")
print("=" * 60 + "\n")
+
+ # Example invocation
+ framework_path = "data/inputs/frameworks/NDI"
+ framework_name = "NDI"
+ setup_framework(framework_path, framework_name)
if __name__ == "__main__":
- framework_path = "data/inputs/frameworks/NDI"
- framework_name = "NDI"
- setup_framework(framework_path, framework_name)
+ main()Also applies to: 68-71
🧰 Tools
🪛 Ruff (0.14.14)
[error] 61-61: f-string without any placeholders
Remove extraneous f prefix
(F541)
🤖 Prompt for AI Agents
In `@services/ai-service/main.py` around lines 51 - 65, The file defines main()
but never calls it; instead the if __name__ == "__main__" block directly invokes
setup_framework() with hardcoded paths. Fix by consolidating startup logic into
main() (use get_input_paths(), print messages and call setup_framework as
appropriate) and replace the direct call in the __main__ block with a single
call to main(); alternatively remove the unused main() if you prefer the current
inline behavior—update the __main__ block and/or main() so only one entrypoint
(main or the inline setup_framework call) remains.
| print("\nDirectory structure:") | ||
| print(f" Input PDFs (Frameworks): {paths['frameworks']}") | ||
| print(f" Input PDFs (Applicants): {paths['applicants']}") | ||
| print(f" Framework outputs: config/frameworks/") |
There was a problem hiding this comment.
Remove extraneous f-string prefix.
This string has no placeholders, so the f prefix is unnecessary.
🔧 Proposed fix
- print(f" Framework outputs: config/frameworks/")
+ print(" Framework outputs: config/frameworks/")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| print(f" Framework outputs: config/frameworks/") | |
| print(" Framework outputs: config/frameworks/") |
🧰 Tools
🪛 Ruff (0.14.14)
[error] 61-61: f-string without any placeholders
Remove extraneous f prefix
(F541)
🤖 Prompt for AI Agents
In `@services/ai-service/main.py` at line 61, The print statement print(f"
Framework outputs: config/frameworks/") uses an unnecessary f-string; replace it
with a normal string literal by removing the leading 'f' so it becomes print("
Framework outputs: config/frameworks/") — locate the exact print call (print(f"
Framework outputs: config/frameworks/")) in main.py and update it accordingly.
| [project] | ||
| name = "ai-service" | ||
| version = "0.1.0" | ||
| description = "Add your description here" |
There was a problem hiding this comment.
Update the placeholder description.
The description "Add your description here" is a placeholder that should be replaced with meaningful project metadata describing the AI service's purpose.
📝 Suggested fix
-description = "Add your description here"
+description = "AI service for governance framework controls extraction, RAG, and policy evaluation"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| description = "Add your description here" | |
| description = "AI service for governance framework controls extraction, RAG, and policy evaluation" |
🤖 Prompt for AI Agents
In `@services/ai-service/pyproject.toml` at line 4, Replace the placeholder
description value in pyproject.toml (the description = "Add your description
here" entry) with a concise, meaningful string that describes the AI service's
purpose (e.g., what the service does, its domain or primary feature). Update the
description field only, keeping TOML syntax intact and using double quotes, and
ensure the new text succinctly summarizes the project for package metadata
consumers.
| ``` | ||
| data/ | ||
| ├── inputs/ | ||
| │ ├── frameworks/ # Place framework PDF files here | ||
| │ └── applicants/ # Place applicant PDF files here | ||
| ``` | ||
|
|
||
| **Usage:** | ||
| - **Framework PDFs**: Place compliance framework PDFs in `data/inputs/frameworks/` | ||
| - **Applicant PDFs**: Place applicant documents to evaluate in `data/inputs/applicants/` | ||
|
|
||
| ### Output Directories (Generated automatically) | ||
|
|
||
| ``` | ||
| config/ | ||
| ├── frameworks/ # Framework outputs (one JSON per section) | ||
| │ └── {framework_name}/ | ||
| │ └── {section_name}.json | ||
| └── vector_db/ # Qdrant vector database storage | ||
|
|
||
| data/ | ||
| └── outputs/ | ||
| └── evaluations/ # Evaluation reports (JSON files) | ||
| ``` |
There was a problem hiding this comment.
Add language identifiers to the two directory-tree fenced blocks.
markdownlint MD040 flags the fences at Line 9 and Line 22 as missing language. Adding text keeps lint clean and improves rendering.
Suggested fix
-```
+```text
data/
├── inputs/
│ ├── frameworks/ # Place framework PDF files here
│ └── applicants/ # Place applicant PDF files here
-```
+```
-```
+```text
config/
├── frameworks/ # Framework outputs (one JSON per section)
│ └── {framework_name}/
│ └── {section_name}.json
└── vector_db/ # Qdrant vector database storage
data/
└── outputs/
└── evaluations/ # Evaluation reports (JSON files)
-```
+```🧰 Tools
🪛 markdownlint-cli2 (0.20.0)
[warning] 9-9: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 22-22: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
In `@services/ai-service/README.md` around lines 9 - 32, Update the two fenced
code blocks showing the directory trees in README.md so they include a language
identifier (`text`) after the opening triple backticks; specifically add ```text
to the block starting with the `data/` tree and the block starting with the
`config/` tree (the blocks that show the `data/ inputs/ frameworks/ applicants/`
tree and the `config/ frameworks/` / `data/ outputs/ evaluations/` tree) to
satisfy markdownlint MD040 and improve rendering.
| - `GET /health` — Health check. | ||
| - `POST /api/v1/frameworks/setup` — Setup framework: form fields `framework_name`, `section_names` (list), `files` (list of PDFs). Same order for section_names and files. | ||
|
|
||
| **Run:** From `ai-service/`: `python src/api/app.py` or `python run.py`. Docs: `/api/docs`, `/api/redoc`. |
There was a problem hiding this comment.
Align the run-from path with the repo layout.
Line 14 points to ai-service/, but this service lives under services/ai-service/. Aligning avoids confusion for new users.
Suggested fix
-**Run:** From `ai-service/`: `python src/api/app.py` or `python run.py`. Docs: `/api/docs`, `/api/redoc`.
+**Run:** From `services/ai-service/`: `python src/api/app.py` or `python run.py`. Docs: `/api/docs`, `/api/redoc`.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| **Run:** From `ai-service/`: `python src/api/app.py` or `python run.py`. Docs: `/api/docs`, `/api/redoc`. | |
| **Run:** From `services/ai-service/`: `python src/api/app.py` or `python run.py`. Docs: `/api/docs`, `/api/redoc`. |
🤖 Prompt for AI Agents
In `@services/ai-service/src/api/README.md` at line 14, Update the README run
instruction to use the correct repository path: replace the "From `ai-service/`"
text with "From `services/ai-service/`" in the Run line in README.md so the
example command (`python src/api/app.py` or `python run.py`) accurately reflects
the service location; verify both docs endpoints `/api/docs` and `/api/redoc`
remain unchanged.
| scroll_result = self.qdrant_client.scroll( | ||
| collection_name=self.collection_name, | ||
| limit=10000, | ||
| with_payload=True, | ||
| with_vectors=False | ||
| ) |
There was a problem hiding this comment.
Hardcoded limit=10000 may not retrieve all documents.
If the collection grows beyond 10,000 documents, this scroll will silently truncate results. Consider using pagination via the offset returned by scroll() or documenting this limitation.
🤖 Prompt for AI Agents
In `@services/ai-service/src/embeddings/haystack_retriever.py` around lines 43 -
48, The scroll call in haystack_retriever.py uses a hardcoded limit=10000 on
qdrant_client.scroll which can truncate results; update the code in the method
that calls qdrant_client.scroll (look for the scroll_result variable and
collection_name usage) to paginate properly by looping and using the
offset/next_offset returned by scroll() (or repeatedly calling scroll with
updated offset until no more results) instead of a single large fixed limit, or
alternatively expose/configure the page size and document this limitation.
| try: | ||
| client.get_collection(collection_name) | ||
| except Exception: | ||
| client.create_collection( | ||
| collection_name=collection_name, | ||
| vectors_config=VectorParams( | ||
| size=vector_size, | ||
| distance=Distance.COSINE | ||
| ) | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, locate the file and check if it exists
find . -name "qdrant_manager.py" -type fRepository: AFAskar/Governance-Agent
Length of output: 122
🏁 Script executed:
# Read the relevant section (lines 50-70) to see the current code
sed -n '50,70p' services/ai-service/src/embeddings/qdrant_manager.pyRepository: AFAskar/Governance-Agent
Length of output: 684
🏁 Script executed:
# Check the imports at the top of the file to see what's already imported
head -30 services/ai-service/src/embeddings/qdrant_manager.pyRepository: AFAskar/Governance-Agent
Length of output: 704
🏁 Script executed:
# Search for exception handling related to Qdrant in the codebase
rg "UnexpectedResponse|NotFoundError|RpcException" --type py -nRepository: AFAskar/Governance-Agent
Length of output: 291
🌐 Web query:
qdrant-client Python exception types collection not found
💡 Result:
In qdrant-client (Python), a “collection not found” coming back from the HTTP/REST API is raised as:
qdrant_client.http.exceptions.UnexpectedResponse(base:ApiException), withe.status_code == 404and the server error ine.content/e.structured().[1][2]- In some call paths, you may instead see
qdrant_client.http.exceptions.ResponseHandlingException, which wraps the underlying exception ine.source.[2]
Example (recommended way to branch on “collection missing”):
from qdrant_client.http.exceptions import UnexpectedResponse, ResponseHandlingException
try:
client.get_collection("my_collection")
except UnexpectedResponse as e:
if e.status_code == 404:
# collection does not exist
...
else:
raise
except ResponseHandlingException as e:
# unwrap if needed
if isinstance(e.source, UnexpectedResponse) and e.source.status_code == 404:
...
else:
raiseYou’ll typically see it printed like: qdrant_client.http.exceptions.UnexpectedResponse: Unexpected Response: 404 (Not Found) when the collection doesn’t exist.[3]
Note: if you’re using the local (embedded) client/backend, some “not found” conditions may be raised as plain ValueError instead of HTTP exceptions.[4]
Sources:
[1] Qdrant client exceptions source (UnexpectedResponse, status_code, structured()) https://python-client.qdrant.tech/_modules/qdrant_client/http/exceptions
[2] Qdrant client exceptions API docs (ApiException, UnexpectedResponse, ResponseHandlingException) https://python-client.qdrant.tech/qdrant_client.http.exceptions
[3] Example of 404 surfaced as UnexpectedResponse in practice (GitHub issue) qdrant/qdrant#4739
[4] Local client code raising ValueError for not-found scenarios https://python-client.qdrant.tech/_modules/qdrant_client/local/qdrant_local
Catching bare Exception masks specific Qdrant errors and hides unexpected failures.
Line 59 catches all exceptions to check collection existence, but this code supports both local (embedded) and remote (HTTP/REST) modes. The local client raises ValueError for missing collections, while the remote client raises qdrant_client.http.exceptions.UnexpectedResponse with status code 404. Catching all exceptions prevents proper error handling and debugging.
Catch the appropriate exceptions for each mode:
Proposed fix
+from qdrant_client.http.exceptions import UnexpectedResponse
+
# Create collection if it doesn't exist
try:
client.get_collection(collection_name)
- except Exception:
+ except (ValueError, UnexpectedResponse) as e:
+ # ValueError: local mode; UnexpectedResponse: remote mode
client.create_collection(
collection_name=collection_name,
vectors_config=VectorParams(
size=vector_size,
distance=Distance.COSINE
)
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| try: | |
| client.get_collection(collection_name) | |
| except Exception: | |
| client.create_collection( | |
| collection_name=collection_name, | |
| vectors_config=VectorParams( | |
| size=vector_size, | |
| distance=Distance.COSINE | |
| ) | |
| ) | |
| from qdrant_client.http.exceptions import UnexpectedResponse | |
| # Create collection if it doesn't exist | |
| try: | |
| client.get_collection(collection_name) | |
| except (ValueError, UnexpectedResponse) as e: | |
| # ValueError: local mode; UnexpectedResponse: remote mode | |
| client.create_collection( | |
| collection_name=collection_name, | |
| vectors_config=VectorParams( | |
| size=vector_size, | |
| distance=Distance.COSINE | |
| ) | |
| ) |
🧰 Tools
🪛 Ruff (0.14.14)
[warning] 59-59: Do not catch blind exception: Exception
(BLE001)
🤖 Prompt for AI Agents
In `@services/ai-service/src/embeddings/qdrant_manager.py` around lines 57 - 66,
The current bare except around client.get_collection masks real errors; update
the try/except in qdrant_manager.py so you catch ValueError (embedded/local
client missing collection) and qdrant_client.http.exceptions.UnexpectedResponse
(remote client) specifically, and only create the collection when ValueError or
an UnexpectedResponse with status_code == 404 is encountered; re-raise any other
UnexpectedResponse or unexpected exceptions so failures aren't hidden. Locate
the try around client.get_collection and the subsequent client.create_collection
call (referencing collection_name, vector_size, VectorParams, Distance) and
implement the two-case exception handling described.
| sections_out: list[dict[str, Any]] = [] | ||
| total_controls = 0 | ||
|
|
||
| for section_name, controls_array in zip(section_names_order, controls_arrays): |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Add strict=True to zip() to catch length mismatches.
If section_names_order and controls_arrays have different lengths due to a bug, zip() will silently truncate. Using strict=True raises an error on mismatch.
🔧 Proposed fix
- for section_name, controls_array in zip(section_names_order, controls_arrays):
+ for section_name, controls_array in zip(section_names_order, controls_arrays, strict=True):📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| for section_name, controls_array in zip(section_names_order, controls_arrays): | |
| for section_name, controls_array in zip(section_names_order, controls_arrays, strict=True): |
🧰 Tools
🪛 Ruff (0.14.14)
[warning] 66-66: zip() without an explicit strict= parameter
Add explicit value for parameter strict=
(B905)
🤖 Prompt for AI Agents
In `@services/ai-service/src/services/framework_service.py` at line 66, The loop
using zip(section_names_order, controls_arrays) can silently truncate if the two
iterables differ in length; update that call to zip(section_names_order,
controls_arrays, strict=True) so Python raises a ValueError on length mismatch.
Locate the for loop that currently reads "for section_name, controls_array in
zip(section_names_order, controls_arrays):" in framework_service.py and change
it to include strict=True; if any surrounding code relies on older Python
versions, ensure the runtime supports Pythons with zip(..., strict=True) or add
an explicit length check (len(section_names_order) != len(controls_arrays))
before iterating.
| if sub.is_dir(): | ||
| pdfs = sorted(sub.glob("*.pdf")) + sorted(sub.glob("*.PDF")) | ||
| if pdfs: | ||
| return pdfs | ||
| pdfs = sorted(vdb.glob("*.pdf")) + sorted(vdb.glob("*.PDF")) |
There was a problem hiding this comment.
Potential duplicate paths on case-insensitive filesystems.
On macOS and Windows, glob("*.pdf") and glob("*.PDF") may return the same files, leading to duplicates in the result list.
🔧 Proposed fix using case-insensitive glob
- if sub.is_dir():
- pdfs = sorted(sub.glob("*.pdf")) + sorted(sub.glob("*.PDF"))
- if pdfs:
- return pdfs
- pdfs = sorted(vdb.glob("*.pdf")) + sorted(vdb.glob("*.PDF"))
- return pdfs
+ def _get_pdfs(directory: Path) -> List[Path]:
+ # Use case-insensitive matching to avoid duplicates on case-insensitive filesystems
+ return sorted(set(directory.glob("*.[pP][dD][fF]")))
+
+ if sub.is_dir():
+ pdfs = _get_pdfs(sub)
+ if pdfs:
+ return pdfs
+ return _get_pdfs(vdb)🤖 Prompt for AI Agents
In `@services/ai-service/src/utils/framework_utils.py` around lines 96 - 100, The
combined glob calls for "*.pdf" and "*.PDF" can produce duplicate Path objects
on case-insensitive filesystems; update the logic in the block that builds pdfs
(the variables and expressions using sub.is_dir(), sub.glob("*.pdf"),
sub.glob("*.PDF"), and vdb.glob(...)) to deduplicate results after collecting
both patterns—e.g., normalize each Path (resolve() or use name.lower()) and keep
a single entry per real file while preserving sort/order; apply the same
deduping for both the sub directory branch and the vdb fallback so pdfs contains
unique files only.
| { | ||
| "name": "stderr", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "/Users/mohammedbalkhair/Documents/Governance-Agent/services/ai-service/.venv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", | ||
| " from .autonotebook import tqdm as notebook_tqdm\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "import os\n", | ||
| "os.environ[\"HF_HUB_OFFLINE\"] = \"1\" \n", | ||
| "from src.rag import retrieve_control_details, RETRIEVE_CONTROL_DETAILS_TOOL_SCHEMA\n", | ||
| "\n", | ||
| "\n", | ||
| "\n", | ||
| "# 2. Retrieve (e.g. from an agent)\n", | ||
| "out = retrieve_control_details(\"DG.1\", \"NDI\", top_k_pdf=5)\n", | ||
| "# out[\"json_cards\"] → structured control summaries\n", | ||
| "# out[\"pdf_chunks\"] → relevant PDF passages\n", | ||
| "\n", | ||
| "# 3. Register as tool\n", | ||
| "# Use RETRIEVE_CONTROL_DETAILS_TOOL_SCHEMA in your agent’s tool list." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 2, | ||
| "id": "a9e7ac20", | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "data": { | ||
| "text/plain": [ | ||
| "{'json_cards': [{'text': 'Control DSI.OE.02. This metric measures the percentage of systems shared by the entity with the National Data Lake (NDL) against the total number of the entity’s systems requested by the NDL team. Note that a system will be considered integrated only if all required dimensions of its data are fully sourced to NDL. Calculation: Number of systems integrated with NDL by the entity / Total number of the entity’s systems requested by NDL * 100. Threshold: 70%. Scale: Unacceptable: ≤ 70%; Low: (70%, 75%]; Fair: (75%, 80%]; Good: (80%, 85%]; Excellent: (85%, 90%]; Leader: > 90%.',\n", | ||
| " 'metadata': {'chunk_id': 'json_NDI_DSI_OE_02_OperationalExcellence-OE',\n", | ||
| " 'framework_name': 'NDI',\n", | ||
| " 'source': 'json',\n", | ||
| " 'control_id': 'DSI.OE.02',\n", | ||
| " 'source_pdf': 'OperationalExcellence-OE'}}],\n", | ||
| " 'pdf_chunks': [{'text': 'NDL) against the total number of the entity’s systems requested\\nby the NDL team. Note that a system will be considered integrated only if all required\\ndimensions of its data are fully sourced to NDL.\\nThis metric aims to accelerate the efforts to enrich NDL with high-value and wide-\\nspectrum data assets generated by various government entities. It also helps in\\n10\\nمﺎﻋ\\n\\n18/11/2025\\nElement Name Element Details\\nachieving the goal of making NDL the unified single source of truth for analytical\\ndata assets.\\nDomain Name Data Sharing and Interoperability (DSI)\\nData Platforms National Data Lake (NDL)\\nDefinitions • Number of systems integrated with NDL by the entity\\n• Total number of the entity’s systems requested by NDL\\nCalculation = Number of systems integrated with NDL by the entity / Total number of the\\nentity’s systems requested by NDL * 100\\nMeasurement Unit Percentage\\nAcceptable Threshold 70%\\nScale Intervals Unacceptable: ≤ 70%\\nLow: (70%, 75%]\\nFair: (75%, 80%]\\nGood: (80%, 85%]\\nExcellent: (85%, 90%]\\nLeader: > 90%\\nVersion History\\nDependencies\\nElement Name Element Details\\nMetric ID DSI.OE.03\\nMetric Name Data sharing agreement processing\\nMetric Description This metric measures the amount of time taken by the data producer to process the\\ndata sharing requests raised by the consumer entities. This metric considers the time\\ntaken for either approving or rejecting the requests as part of the processing time.',\n", | ||
| " 'score': 0.84288440527831,\n", | ||
| " 'metadata': {'chunk_id': 'pdf_OperationalExcellence-OE_11',\n", | ||
| " 'framework_name': 'NDI',\n", | ||
| " 'start_char': 14175,\n", | ||
| " 'end_char': 15595,\n", | ||
| " 'length': 1419,\n", | ||
| " 'chunk_size': 1500,\n", | ||
| " 'overlap': 200,\n", | ||
| " 'source': 'pdf',\n", | ||
| " 'source_pdf': 'OperationalExcellence-OE'}}]}" | ||
| ] |
There was a problem hiding this comment.
Clear notebook outputs to avoid PII and content leakage.
The committed outputs include a local user path and extracted control text. Please clear outputs before committing to avoid leaking local paths and potentially sensitive content.
If helpful, you can clear outputs with:
jupyter nbconvert --clear-output --inplace services/ai-service/test.ipynb
🧰 Tools
🪛 Ruff (0.14.14)
[warning] 13-13: Comment contains ambiguous ’ (RIGHT SINGLE QUOTATION MARK). Did you mean ``` (GRAVE ACCENT)?
(RUF003)
[warning] 17-17: Found useless expression. Either assign it to a variable or remove it.
(B018)
🤖 Prompt for AI Agents
In `@services/ai-service/test.ipynb` around lines 9 - 60, The notebook contains
committed cell outputs leaking a local path and extracted control text; clear
all outputs before committing by running a notebook-output clear (e.g., jupyter
nbconvert --clear-output --inplace <notebook> or use the Jupyter UI) and remove
the printed results from the cell that calls retrieve_control_details (the cell
that invoked retrieve_control_details("DG.1", "NDI", top_k_pdf=5) and references
RETRIEVE_CONTROL_DETAILS_TOOL_SCHEMA), then recommit the cleaned notebook.
|
Note Docstrings generation - SUCCESS |
Docstrings generation was requested by @Mohammed-Balkhair-hub. * #2 (comment) The following files were modified: * `services/ai-service/main.py` * `services/ai-service/src/api/app.py` * `services/ai-service/src/api/routers/frameworks.py` * `services/ai-service/src/api/routers/health.py` * `services/ai-service/src/core/evaluator.py` * `services/ai-service/src/core/framework_extraction.py` * `services/ai-service/src/embeddings/gemma_embedder.py` * `services/ai-service/src/embeddings/haystack_retriever.py` * `services/ai-service/src/embeddings/qdrant_manager.py` * `services/ai-service/src/processing/pdf_parser.py` * `services/ai-service/src/processing/text_chunker.py` * `services/ai-service/src/rag/_shared.py` * `services/ai-service/src/rag/ingestion.py` * `services/ai-service/src/rag/retrieval.py` * `services/ai-service/src/services/framework_service.py` * `services/ai-service/src/utils/framework_utils.py`
1 participant
this branch completed setup governance framework controls extraction endpoint as well as establishing Vector DB for for framework files , controls & polices
Summary by CodeRabbit
Release Notes
New Features
Documentation
✏️ Tip: You can customize this high-level summary in your review settings.