Repository navigation
Conversation
A choice list declares its displayed labels and its saved values in two <items> groups; the generator concatenated them into options such as ['non', 'oui', '0', '1']. The model could answer '0', and 'oui' was never translated to '1', so the radio stayed empty. Nine catalogue fields are fixed in place and the generator now reads the save="1" group apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
- XFA bool fields collapsed anything but '1'/'true' to '0': 'oui' and 'On' were written unchecked while the AcroForm layer showed them checked. - Checkbox normalisation did not understand French answers. - A /Ch list received the label in /V and drew it as is; it now stores the export value and draws the label, whichever of the two it is given. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
- A preset that is only the blank state of a field (unchecked box, yes/no
question on 'non', counter at 0) no longer excludes it from extraction:
91 catalogue fields were never filled.
- Repeated structures get their context and rank ('période d'incapacité de
travail n°2 sur 4'): 177 fields shared identical questions.
- Occurrences of a repeated structure stay in the same batch.
- Values are normalised before display and fill: not-found answers become
empty, dates JJ.MM.AAAA, yes/no canonical, option labels to export values.
- Chosen answers (yes/no, options) are grounded by their quote only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
- Chunks carry their source (document, page, section) and are sized for the reranker; tiny sections are merged. - The prompt context is measured in tokens: a dossier that fits is sent whole, once, as a prefix shared by every batch; otherwise chunks are interleaved by rank so no field of a batch is left without sources. - The prompt puts the full synthesis before the documents, documents prevailing; no more '...' example for the model to copy. - Responses are constrained by a JSON schema (falls back to json_object), parsed leniently, and a failed batch is shrunk, split or re-asked instead of being lost; 4xx errors are no longer retried blindly. app.py switches to this module in a following commit; the former prompt builder stays until then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
Conservative token estimate, explicit max_tokens, 900 s timeout and one retry; a failed direct synthesis falls back to per-document summaries, oversized documents are split, and an impossible merge is replaced by a deterministic one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
…f failing
- Run and rerun share _extract_fields and _fill_pdf: the rerun copy lost the
computed fields and the canton recipient, so a re-run AI form went back to
the template's default office. Rerun now also waits for a pipeline slot.
- The synthesis is passed whole to every batch: the per-section filter
(SECTION_SYNTHESIS_KEYS, removed) only matched the former AVS layout, and
hid incapacity, diagnosis and physician data from most other forms.
- One unreadable document is skipped with a warning instead of failing the
job (and leaving the embed consumer looping); OCR retries match the cause.
- Embedding and rerank outages degrade retrieval instead of failing the job.
- Actionable failures ('no readable text') reach the user; /status exposes
warnings.
- /chat requires the job token like /fields and /download.
- Debug logs holding full patient text are purged with the job
(KEEP_DEBUG_LOGS to keep them); the purge loop no longer dies on error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
…ng errors A single failed /status poll abandoned a job still running on the server; polling now retries with backoff. A failed rerun no longer leaves the rerun button disabled, and job warnings are shown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
- GitHub Actions: orchestrator tests (no GPU, services mocked) and frontend build. - .gitignore no longer ignores the whole eval/ tree it tracks; tracked bytecode removed. - .dockerignore keeps tests and the medForms cache out of the image. - Pipeline-RAG wiki rewritten for the current pipeline; new settings documented in .env.example and next to --max-model-len. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
Deploying doctorfill with
|
| Latest commit: |
8adb863
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://54195983.doctorfill.pages.dev |
| Branch Preview URL: | https://claude-laughing-allen-i8izo9.doctorfill.pages.dev |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved retrieval, extraction retry, and debug-log retention issues remain.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (3)
What changed in this PR
This PR hardens the RAG pipeline from extraction through PDF filling, with improved resilience, normalization, security, testing, and documentation.
Changes:
- Adds contextualized, token-aware extraction and synthesis fallbacks.
- Unifies processing and reruns with improved PDF field handling.
- Adds frontend warnings, CI, integration tests, and operational configuration.
| File | Summary |
|---|---|
services/orchestrator/tools/gen_template.py |
Separates display labels from export values. |
services/orchestrator/tests/test_pipeline.py |
Adds end-to-end pipeline coverage. |
services/orchestrator/tests/test_fill.py |
Tests PDF and checkbox filling. |
services/orchestrator/tests/test_fields.py |
Tests field preparation and normalization. |
services/orchestrator/tests/test_extraction.py |
Tests resilient extraction and context budgeting. |
services/orchestrator/tests/pdf_factory.py |
Provides PDF test fixtures. |
services/orchestrator/tests/conftest.py |
Configures isolated tests. |
services/orchestrator/tests/__init__.py |
Defines the test package. |
services/orchestrator/template/Form_LCA_IncapaciteTravail.json |
Updates form export mappings. |
services/orchestrator/template/Form_LAMal_RapportInitial.json |
Updates form export mappings. |
services/orchestrator/template/Form_LAM_RapportIntermediaire.json |
Updates form export mappings. |
services/orchestrator/template/Form_LAA_RapportInitial.json |
Updates form export mappings. |
services/orchestrator/template/Form_LAA_RapportAbrege.json |
Updates form export mappings. |
services/orchestrator/template/Form_LAA_PriseEnChargeHospitaliere.json |
Updates form export mappings. |
services/orchestrator/template/Form_LAA_PremierDiagnostic_MTBI.json |
Updates form export mappings. |
services/orchestrator/requirements-dev.txt |
Adds development test dependencies. |
services/orchestrator/prompts.py |
Revises extraction and synthesis prompts. |
services/orchestrator/medical_synthesis.py |
Adds budgets and synthesis fallbacks. |
services/orchestrator/extraction.py |
Adds token-aware, resilient extraction. |
services/orchestrator/core/provenance.py |
Supports response provenance. |
services/orchestrator/core/fill.py |
Preserves boolean states during XFA filling. |
services/orchestrator/core/fields.py |
Centralizes field preparation and normalization. |
services/orchestrator/core/checkbox.py |
Recognizes boolean responses. |
services/orchestrator/core/acroform.py |
Handles choice values and appearances. |
services/orchestrator/app.py |
Integrates orchestration, cleanup, and security changes. |
services/orchestrator/.dockerignore |
Excludes unnecessary image content. |
services/docker-compose.yml |
Documents model context configuration. |
services/.env.example |
Documents pipeline settings. |
README.md |
Updates test instructions. |
frontend/src/App.tsx |
Improves polling, warnings, chat authentication, and reruns. |
docs/wiki/Pipeline-RAG.md |
Rewrites pipeline documentation. |
.gitignore |
Corrects repository exclusions. |
.github/workflows/tests.yml |
Adds backend and frontend CI. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The canton table names its fields without their structure (street, zip, ean, input) and was applied by name alone. For a patient treated in NE, the IV office address replaced the patient's address, the office EAN replaced the physician's GLN, and free-text remarks received 'exaddress#@#…'. It now only writes recipient blocks and the insurer/recipient structure. Also found by the catalogue-wide tests: in LAA_PriseEnChargeHospitaliere the d/e blocks are procedure slots (coding preset CHOP) but asked for ICD-10 diagnoses. Their questions are fixed and they are numbered as interventions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
…-ask empty replies Addresses the Copilot review: - Job files and debug logs left by a previous process are purged once past retention (at startup and on each cleanup pass); job files move to a dedicated JOBS_DIR so the purge never touches the rest of /tmp. - Any retrieval failure, including question embeddings, now takes the degraded path with a warning instead of failing the job. - A valid but empty JSON reply is asked again once, like partial replies. The mocked services move to tests/fakes.py for reuse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
- eval/truth.py derives ground truth for all 21 forms from the scenarios that generated eval/dossiers (324 fields, each present in the dossier). - eval/regression.py runs the dossiers through the API, scores them and compares with eval/baseline.json; exit 1 on regression, 2 on failure. - test_catalogue: template invariants, context budget, and a coverage snapshot so no form silently loses extracted fields. - test_catalogue_fill: a hybrid XFA + AcroForm PDF per form; every extracted field must land in both layers (radios, combos, checkboxes, canton). - test_eval_dossiers: the 21 dossiers through the HTTP API with a model answering the ground truth; 100% in /fields and in the delivered PDF, and the regression gate itself is shown to catch a wrong value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
Native text skips OCR, scans and partly scanned documents go to OCR, broken font encodings are not trusted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
- tests.yml: lint (ruff, errors only), orchestrator tests with a 70% coverage floor, marker-ocr tests, frontend build; superseded runs are cancelled. Existing lint errors fixed (unused imports, f-string). - eval.yml: runs eval/regression.py on a self-hosted 'dgx' runner against the real backend, on demand, and publishes the report. - docs/wiki/Tests.md: what each level guarantees, and how to make the checks required so a red PR cannot be merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contexte
Audit du dépôt suite au constat : le RAG n'est pas parfait, le remplissage est incomplet et des erreurs surviennent. Cette MR regroupe toutes les corrections, ainsi qu'un dispositif de tests et de non-régression. Chaque commit se relit seul, importe
appet passe ses tests (vérifié commit par commit).Audit : causes trouvées
Remplissage incomplet ou faux
presetexcluait le champ de l'extraction, y compris l'état vierge d'un champ (case « /Off », question oui/non sur « non », compteur à 0)phone/phone[1], diagnostics a/b/d/e)street,zip,ean,input…) — bug antérieur, trouvé par les nouveaux testsexaddress#@#…(5 formulaires : 4 AI et LAM_FeuilleMaladieAccident)LAA_PriseEnChargeHospitaliere: blocsd/eréservés aux interventions (CHOP) mais questions « code CIM-10 du diagnostic »Erreurs
max_tokens2048max_tokensni repli, estimation à 4 caractères par token/statusen échec abandonnait le suiviSécurité / confidentialité
/chatne vérifiait pas le jeton du job.Modifications
core/fields.py(nouveau) :extraction.py(nouveau) :prompts.py,medical_synthesis.py: prompt révisé ; synthèse avec budgets explicites et replis.app.py:/status;/chat;JOBS_DIR)./Ch(valeur d'export dans/V, libellé affiché) ; générateur de templates corrigé./chat, suivi relancé avec délai croissant, avertissements affichés.Tests et non-régression
Deux niveaux, documentés dans
docs/wiki/Tests.md.1. CI à chaque PR (
.github/workflows/tests.yml, sans GPU, services simulés) : jobslint,orchestrator(plancher de couverture 70 %, actuellement 75 %),marker-ocr,frontend.test_cataloguetest_catalogue_filltest_eval_dossierseval/dossiers/par l'API HTTP, modèle simulé répondant la vérité terrain : 100 % dans/fieldset dans le PDF livré ; le garde-fou qualité est lui-même testétest_pipeline,test_extraction,test_fields,test_fill,test_textlayerChaque garde-fou a été vérifié par mutation. Réintroduire l'ancien bug des cases fait échouer 8 formulaires, le destinataire appliqué par nom 6 tests, l'exclusion des cases vierges 6 tests.
2. Qualité du vrai modèle :
eval/regression.pysoumet les 21 dossiers au backend et les note contre 324 champs de vérité terrain dérivés des scénarios (contre 26 sur un seul formulaire jusqu'ici). Il compare ensuite àeval/baseline.jsonet renvoie 1 en cas de régression, 2 en cas d'échec d'exécution..github/workflows/eval.ymlle lance à la demande sur un runner auto-hébergédgx.Vérifications
pytestorchestrateur : 265 passed (75 % de couverture) ;marker_ocr: 4 passed ;ruff: propre ;npm run build: vert.Non vérifié / à faire
main, aveclint,orchestrator,marker-ocretfrontendexigés (voirTests.md). Sans cette règle, une PR rouge reste fusionnable.python eval/regression.py --api http://localhost:8080 --api-key $KEY --update-baselinesurmainaprès merge, puis committereval/baseline.json. Rien de cette MR n'a tourné contre les vrais services (pas de GPU ici, medforms.ch inaccessible).eval.yml: enregistrer un runner auto-hébergé portant le labeldgxet créer le secretDOCTORFILL_API_KEY.response_format: json_schemaavec la version de vLLM de l'image NGC : repli automatique surjson_objectsi refusé.recipientInfoBlock/recipientBlockAddressFooter, absents des templates. Il faut vérifier sur un PDF réel que ces blocs n'affichent pas l'office par défaut.🤖 Generated with Claude Code
https://claude.ai/code/session_01CAQEkcJGzQXzySYYsRCKWW