Skip to content

Agentic SQL & routing: higher tiers unsolvable, Verify solve can be passed without the tool path #31

Open

Description

@afshinlamei

Summary

Functional testing of /agentic-tools (done for #29) found four problems that need backend or UI changes, so they are not in that content PR.

1. F1–F4 cannot be solved through the tools

From F1 upward the fragment filter rejects %, _, anything containing bob, and fragments under 3 alphanumeric characters. Since the only username to match is bob, no allowed fragment can return Bob's row. An exhaustive search of ~1.5M fragments (all strings up to 5 characters from a set including b, o, case variants, wildcards and punctuation) found zero leaks at F1–F4; F0 leaks with b. At F3/F4 the table name must exactly match an allowlisted name, so it is not injectable either.

Question for maintainers: are F1–F4 meant to be defenses that hold, or challenges with an intended bypass? If the latter, the F1 filter is too strict as written. The test named test_catering_sql_f1_partial_fragment_still_exfil_mocked asserts solved is False, which suggests this changed at some point.

2. "Verify solve" can be passed without exploiting the tool path

The solve check can be satisfied at every tier, including F4, without any tool call. Because the lab is used for CTFs and workshops, I am not posting the reproduction here; I will share it with the maintainers directly.

Expected: a solve counts only when another user's token was returned by a tool result.

3. generated_sql, level_name and level_hint are returned but never shown

The page says each response includes generated_sql, but the chat only renders the combined reply. Seeing the executed SQL is the core learning point of the lab; currently it is only visible in DevTools.

4. A missing Ollama model shows up as a bare chat reply

With no model pulled, the assistant replies Error: Ollama status 404 with no guidance. Pointing to Lab Setup would help workshop attendees.

Context

Field Value
Module Agentic SQL & routing (/agentic-tools)
Environment Docker compose, ghcr.io/owasp/pwnzzai:latest, Ollama llama3.2:1b
Related #29
Priority Medium (items 1–2), Low (items 3–4)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions