Skip to content

feat: add Qwen Autopilot Trust Agent hackathon MVP - #910

Open
safal207 wants to merge 30 commits into
mainfrom
agent/qwen-autopilot-agent
Open

feat: add Qwen Autopilot Trust Agent hackathon MVP#910
safal207 wants to merge 30 commits into
mainfrom
agent/qwen-autopilot-agent

Conversation

@safal207

@safal207 safal207 commented Jul 16, 2026

Copy link
Copy Markdown
Owner

What changed

  • adds a standalone FastAPI MVP for the Qwen Cloud Autopilot Agent track;
  • integrates Alibaba Cloud Model Studio through the OpenAI-compatible Qwen API;
  • validates Qwen structured output before it can affect a decision;
  • adds a deterministic policy floor so model output cannot weaken safety controls;
  • implements ALLOW, HUMAN_APPROVAL, and BLOCK decisions;
  • persists human approval records in SQLite with WAL/busy-timeout handling;
  • makes repeated approval resolution explicit and idempotent;
  • adds an interactive three-scenario demo UI;
  • adds Docker deployment assets, architecture documentation, tests, CI, a deployment verifier, and submission-ready Devpost copy;
  • adds an explicit one-shot live Qwen smoke workflow guarded by a commit-message marker;
  • records redacted immutable evidence from a real Qwen Cloud response.

Why

Autonomous agents need a control point before actions with external, financial, credential, production, or irreversible side effects. This MVP combines Qwen semantic reasoning with deterministic fail-closed policy and explicit human review.

Current exact head

11d27d156e8b301a9eaffdf084c4f37cfb78dcf9

Live Qwen validation

A real Model Studio request completed successfully on exact tested head:

24ab32d04871798b240b95e2daf6d3e18049dfec

Evidence:

  • workflow run: 29706522814;
  • live job: 88244366628 — success;
  • artifact: 8448074218;
  • artifact digest: sha256:6fa93b99181c67d3696d4924c16ffb7e33a8ab171e0a4d733f6f931316737ae2;
  • model: qwen3.7-plus;
  • Qwen status: COMPLETED;
  • validated result: LOW / ALLOW, confidence 0.98;
  • execution: NOT_EXECUTED;
  • authority: advisory_only.

The API key remained masked by GitHub Actions and is not included in logs or committed evidence. Temporary PR access to the secret was removed after the test; future live calls require an explicit [qwen-live-smoke] push marker.

Automated validation

  • policy tests cover allow, human-review, block, and non-weakening behavior;
  • API smoke tests cover /healthz, completed Qwen assessment, fail-closed behavior, and idempotent approval resolution;
  • python -m py_compile app/*.py runs in the dedicated workflow;
  • dedicated Qwen Autopilot Agent CI is configured for this subtree.

Submission assets

  • hackathons/qwen-autopilot-agent/DEVPOST_SUBMISSION.md
  • hackathons/qwen-autopilot-agent/docs/architecture.md
  • hackathons/qwen-autopilot-agent/docs/DEPLOYMENT_EVIDENCE.md
  • hackathons/qwen-autopilot-agent/docs/evidence/LIVE_QWEN_SMOKE.md
  • hackathons/qwen-autopilot-agent/docs/evidence/qwen-live-smoke-24ab32d0.json
  • hackathons/qwen-autopilot-agent/scripts/verify_deployment.sh

Remaining external work

  • deploy the container on Alibaba Cloud;
  • add the repository secret to the cloud service as an environment variable, never as source code;
  • obtain a public HTTPS URL;
  • run the deployment verifier and capture deployed qwen.status = COMPLETED evidence;
  • record the public demo video under three minutes;
  • add the public URL, screenshots, and final immutable commit links to Devpost.

Boundary

The live smoke proves Qwen Cloud connectivity and structured-response validation. It does not yet prove Alibaba Cloud hosting. Deployment remains the final external requirement before submission.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@safal207, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 56 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f4141989-3bde-4f8d-9e81-b93124292140

📥 Commits

Reviewing files that changed from the base of the PR and between 3a63e9c and 11d27d1.

📒 Files selected for processing (20)
  • .github/workflows/qwen-autopilot-agent.yml
  • hackathons/qwen-autopilot-agent/.env.example
  • hackathons/qwen-autopilot-agent/DEVPOST_SUBMISSION.md
  • hackathons/qwen-autopilot-agent/Dockerfile
  • hackathons/qwen-autopilot-agent/README.md
  • hackathons/qwen-autopilot-agent/app/__init__.py
  • hackathons/qwen-autopilot-agent/app/main.py
  • hackathons/qwen-autopilot-agent/app/policy.py
  • hackathons/qwen-autopilot-agent/app/qwen.py
  • hackathons/qwen-autopilot-agent/app/store.py
  • hackathons/qwen-autopilot-agent/docs/DEPLOYMENT_EVIDENCE.md
  • hackathons/qwen-autopilot-agent/docs/architecture.md
  • hackathons/qwen-autopilot-agent/docs/evidence/LIVE_QWEN_SMOKE.md
  • hackathons/qwen-autopilot-agent/docs/evidence/qwen-live-smoke-24ab32d0.json
  • hackathons/qwen-autopilot-agent/requirements-dev.txt
  • hackathons/qwen-autopilot-agent/requirements.txt
  • hackathons/qwen-autopilot-agent/scripts/verify_deployment.sh
  • hackathons/qwen-autopilot-agent/static/index.html
  • hackathons/qwen-autopilot-agent/tests/test_api.py
  • hackathons/qwen-autopilot-agent/tests/test_policy.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/qwen-autopilot-agent

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

LS multi-model PR review

  • Exact head: 11d27d156e8b301a9eaffdf084c4f37cfb78dcf9
  • Base: 3a63e9ce81d00d4db063cd017e897d1c1b2f73a0
  • Provider: openrouter
  • Status: PARTIAL
  • Aggregate verdict: COMMENT
  • Mode: advisory
  • High-risk route: true
  • Diff truncated: true
  • Files represented in bounded evidence: 18/20
  • Policy would block: true

Model executions

Role Model Status Verdict
architecture_and_governance_reviewer nvidia/nemotron-3-ultra-550b-a55b:free INVALID -
fast_diff_reviewer cohere/north-mini-code:free INVALID -
deep_implementation_reviewer poolside/laguna-xs-2.1:free INVALID -
independent_challenger tencent/hy3:free VALID REQUEST_CHANGES

Confirmed findings

No finding reached independent two-model confirmation.

Candidate findings

  • critical .github/workflows/qwen-autopilot-agent.yml:47 Malformed secret reference in CI workflow — support 1
  • high hackathons/qwen-autopilot-agent/docs/evidence/LIVE\_QWEN\_SMOKE.md:11 Live smoke evidence references different head SHA — support 1
  • medium hackathons/qwen-autopilot-agent/app/policy.py:38 Substring policy matching causes false positives — support 1
  • low hackathons/qwen-autopilot-agent/app/main.py:58 Qwen failure mode must be surfaced explicitly — support 1
  • low hackathons/qwen-autopilot-agent/app/store.py:31 SQLite concurrency limited to single process — support 1

Incomplete lanes

  • diff\_coverage: {'key': 'diff_coverage', 'reason': 'bounded diff was truncated; the review cannot claim complete PR coverage', 'reviewed_files': ['.github/workflows/qwen-autopilot-agent.yml', 'hackathons/qwen-autopilot-agent/.env.example', 'hackathons/qwen-autopilot-agent/DEVPOST_SUBMISSION.md', 'hackathons/qwen-autopilot-agent/Dockerfile', 'hackathons/qwen-autopilot-agent/README.md', 'hackathons/qwen-autopilot-agent/app/__init__.py', 'hackathons/qwen-autopilot-agent/app/main.py', 'hackathons/qwen-autopilot-agent/app/policy.py', 'hackathons/qwen-autopilot-agent/app/qwen.py', 'hackathons/qwen-autopilot-agent/app/store.py', 'hackathons/qwen-autopilot-agent/docs/DEPLOYMENT_EVIDENCE.md', 'hackathons/qwen-autopilot-agent/docs/architecture.md', 'hackathons/qwen-autopilot-agent/docs/evidence/LIVE_QWEN_SMOKE.md', 'hackathons/qwen-autopilot-agent/docs/evidence/qwen-live-smoke-24ab32d0.json', 'hackathons/qwen-autopilot-agent/requirements-dev.txt', 'hackathons/qwen-autopilot-agent/requirements.txt', 'hackathons/qwen-autopilot-agent/scripts/verify_deployment.sh', 'hackathons/qwen-autopilot-agent/static/index.html'], 'omitted_files': ['hackathons/qwen-autopilot-agent/tests/test_api.py', 'hackathons/qwen-autopilot-agent/tests/test_policy.py']}
  • tie\_breaker: {'key': 'tie_breaker', 'requested_model': 'gpt-oss-120b', 'candidates': ['openai/gpt-oss-120b:free'], 'reserved_candidates': []}

Authority boundary

This output is evidence for human review. It cannot approve or merge the PR, and a single-model finding remains a candidate rather than a gate decision.

@safal207
safal207 marked this pull request as ready for review July 19, 2026 21:03

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Please review exact head 878e90e3054df1812778609f7d901e7f665f0d8f. Focus on the Qwen response validation, SQLite approval resolution semantics, API smoke tests, deployment verifier, and submission evidence accuracy.

@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Revision =='
git rev-parse HEAD
git status --short

printf '%s\n' '== Relevant files =='
fd . hackathons/qwen-autopilot-agent -t f 2>/dev/null | sort
printf '%s\n' '== Workflows mentioning Qwen =='
rg -n -i -C2 'qwen|autopilot' .github/workflows 2>/dev/null || true

printf '%s\n' '== Python module outlines =='
fd -e py . hackathons/qwen-autopilot-agent -t f -x ast-grep outline {} --items all --view expanded 2>/dev/null || true

printf '%s\n' '== Test / script / documentation contents =='
fd -t f . hackathons/qwen-autopilot-agent \
  | rg '(^|/)(test[^/]*|.*\.(py|sh|md|ya?ml|json|toml|Dockerfile))$' \
  | sort \
  | while IFS= read -r file; do
      printf '\n===== %s =====\n' "$file"
      cat -n "$file"
    done

Length of output: 48826


🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='878e90e3054df1812778609f7d901e7f665f0d8f'

printf '%s\n' '== Requested revision availability and relationship =='
git cat-file -t "$target"
git merge-base --is-ancestor "$target" HEAD && echo 'target is an ancestor of HEAD' || echo 'target is not an ancestor of HEAD'
git diff --stat "$target" HEAD -- hackathons/qwen-autopilot-agent .github/workflows/qwen-autopilot-agent.yml
git diff --unified=25 "$target" HEAD -- hackathons/qwen-autopilot-agent .github/workflows/qwen-autopilot-agent.yml

printf '%s\n' '== Exact target files: implementation, tests, deployment, workflow =='
for file in \
  hackathons/qwen-autopilot-agent/app/main.py \
  hackathons/qwen-autopilot-agent/app/qwen.py \
  hackathons/qwen-autopilot-agent/app/store.py \
  hackathons/qwen-autopilot-agent/tests/test_api.py \
  hackathons/qwen-autopilot-agent/tests/test_policy.py \
  hackathons/qwen-autopilot-agent/scripts/verify_deployment.sh \
  hackathons/qwen-autopilot-agent/Dockerfile \
  .github/workflows/qwen-autopilot-agent.yml \
  hackathons/qwen-autopilot-agent/static/index.html \
  hackathons/qwen-autopilot-agent/docs/architecture.md \
  hackathons/qwen-autopilot-agent/docs/DEPLOYMENT_EVIDENCE.md \
  hackathons/qwen-autopilot-agent/DEVPOST_SUBMISSION.md; do
  printf '\n===== %s @ %s =====\n' "$file" "$target"
  git show "$target:$file" | cat -n
done

printf '%s\n' '== Exact target: assertions and public-facing claims =='
git grep -n -I -E 'COMPLETED|idempoten|execut|approval|Qwen|SQLite|verified|qwen3\.7' "$target" -- \
  hackathons/qwen-autopilot-agent .github/workflows/qwen-autopilot-agent.yml || true

Length of output: 65240


@​safal207, reviewed exact head 878e90e3054df1812778609f7d901e7f665f0d8f (the sandbox checkout was a descendant with only a later .env.example change).

Findings

  1. High — approval resolution has no reviewer authentication or authorization
    hackathons/qwen-autopilot-agent/app/main.py, Lines 94–99 accepts any caller-provided reviewer string and transitions an approval through ApprovalStore.resolve. The atomic SQLite condition correctly prevents overwrite, but it does not establish that a human reviewer authorized the transition. Anyone who obtains an approval_id can approve or reject it and forge the audit identity.
    Add an authenticated reviewer boundary (and derive reviewer identity server-side), or explicitly scope the feature/documentation as an unauthenticated demo queue rather than human authorization.

  2. Medium — architecture/evidence overstate what the service proves or produces
    hackathons/qwen-autopilot-agent/docs/architecture.md, Line 10 shows an “Advisory execution token,” but app/main.py, Line 78 only returns execution.status = NOT_EXECUTED; no token is issued. Additionally, docs/DEPLOYMENT_EVIDENCE.md, Line 42 says qwen.status = COMPLETED proves the backend reached Qwen Cloud, though this status is set by the application after a successful call to an operator-configurable OpenAI-compatible QWEN_BASE_URL.
    Rename the diagram’s ALLOW output to an advisory assessment/decision, and qualify the verifier as confirming a successful assessment through the configured endpoint—not cryptographic proof that the endpoint is Qwen Cloud.

The focused implementation otherwise has solid fail-closed behavior: Pydantic constrains the Qwen decision payload; exceptions from parsing/validation fall back to HUMAN_APPROVAL; and the UPDATE ... WHERE status='PENDING' transition correctly preserves the first approval resolution. The existing API smoke tests validate happy-path completion, missing credentials, and store-level idempotence, but should be extended alongside the authorization boundary and malformed-Qwen-output path.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Final review target: exact head c4f51d9b94044908a418c8b4172d5b907232f5ae. Earlier review requests are stale because endpoint documentation was simplified after them.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

@safal207, I’ll review the final target c4f51d9b94044908a418c8b4172d5b907232f5ae, treating the earlier endpoint-documentation feedback as stale.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant