Confirming first that #5 is fixed, because I re-ran it rather than taking the word for it. At 7557fe0 on a machine with no CUDA, python:3.12-slim, torch 2.14.0+cpu:
GET /health -> {"ok": true, "model": "Mapika/decider-2b", "device": "cpu"}
POST /v1/systemone -> 200 refund noul 0.9781 · team choice "billing" 0.9983
POST /decide -> 200 {"Which team should handle this?": {"choice": "billing", "confidence": 0.9387, …}}
python -m pytest tests 110 passed, 9 skipped
and the three refusals all name the requirement:
DECIDER_DEVICE=cuda RuntimeError: DECIDER_DEVICE=cuda, but torch.cuda.is_available() is False on this
machine. Set DECIDER_DEVICE=mps or cpu (or leave it at auto).
DECIDER_DEVICE=mps RuntimeError: DECIDER_DEVICE=mps, but torch.backends.mps.is_available() is False …
DECIDER_DEVICE=bogus RuntimeError: DECIDER_DEVICE=bogus: expected auto, cuda, cuda:<index>, mps or cpu.
Thank you — that was a thorough fix, and /health reporting the device is more than I asked for.
The new thing
While exercising /decide I sent two bodies a first-time reader might send, and both come back as 500 Internal Server Error with an unhandled traceback in the log:
| body |
result |
{"context": "…"} — no schema at all |
500, AttributeError: 'NoneType' object has no attribute 'items' |
{"context": "…", "schema": {"Which team?": ["billing", "technical"]}} |
500, AttributeError: 'list' object has no attribute 'get' |
{"context": "…", "schema": null} |
422, clean |
{"context": "…", "schema": ["a", "b"]} |
422, clean |
{"schema": {…}} — no context |
422, clean |
The second row is the interesting one: the module docstring says POST /decide {"context": str, "schema": {...}}, and a reader who has just seen d.decide(ctx, [{"question": …, "options": [...]}]) in the README can reasonably guess that a schema maps a question to its options. The real shape is {question: {"type": "choice", "options": [...]}}, which is documented in a comment above _schema_to_questions in infer.py rather than anywhere near the endpoint.
schema_: dict = None on Req is what lets both through: None is a valid default, and any dict satisfies the annotation, so validation passes and _prepare_decide is the first thing to look inside it.
/v1/systemone sets the bar here — {"type": "choice", …} with 300 options gets 422 {"detail":"choice criteria: a map of 2..255 options"}, which tells you exactly what is wrong.
Suggestion
Make schema_ required and typed, e.g. dict[str, dict], so a missing key and a list-valued entry are both 422s before the executor is reached. A one-line shape check in _prepare_decide raising HTTPException(422, …) with the expected form would do as well, and would let the message name the three field types.
Unrelated small note: POST /v1/systemone with "questions": {} returns 200 {"answers": {}}. Refusing an empty question map would match /decide's own 2..255 instinct, and would stop a caller reading "no answers" as "no findings".
Found while re-checking decider for mrjev.com after your fix to #5. Everything above ran locally against your released weights; no hosted API was called.
Confirming first that #5 is fixed, because I re-ran it rather than taking the word for it. At 7557fe0 on a machine with no CUDA,
python:3.12-slim,torch 2.14.0+cpu:and the three refusals all name the requirement:
Thank you — that was a thorough fix, and
/healthreporting the device is more than I asked for.The new thing
While exercising
/decideI sent two bodies a first-time reader might send, and both come back as500 Internal Server Errorwith an unhandled traceback in the log:{"context": "…"}— noschemaat allAttributeError: 'NoneType' object has no attribute 'items'{"context": "…", "schema": {"Which team?": ["billing", "technical"]}}AttributeError: 'list' object has no attribute 'get'{"context": "…", "schema": null}{"context": "…", "schema": ["a", "b"]}{"schema": {…}}— nocontextThe second row is the interesting one: the module docstring says
POST /decide {"context": str, "schema": {...}}, and a reader who has just seend.decide(ctx, [{"question": …, "options": [...]}])in the README can reasonably guess that a schema maps a question to its options. The real shape is{question: {"type": "choice", "options": [...]}}, which is documented in a comment above_schema_to_questionsininfer.pyrather than anywhere near the endpoint.schema_: dict = NoneonReqis what lets both through:Noneis a valid default, and anydictsatisfies the annotation, so validation passes and_prepare_decideis the first thing to look inside it./v1/systemonesets the bar here —{"type": "choice", …}with 300 options gets422 {"detail":"choice criteria: a map of 2..255 options"}, which tells you exactly what is wrong.Suggestion
Make
schema_required and typed, e.g.dict[str, dict], so a missing key and a list-valued entry are both 422s before the executor is reached. A one-line shape check in_prepare_decideraisingHTTPException(422, …)with the expected form would do as well, and would let the message name the three field types.Unrelated small note:
POST /v1/systemonewith"questions": {}returns200 {"answers": {}}. Refusing an empty question map would match/decide's own2..255instinct, and would stop a caller reading "no answers" as "no findings".Found while re-checking decider for mrjev.com after your fix to #5. Everything above ran locally against your released weights; no hosted API was called.