Skip to content

The packaged NASim red configuration repeats one exploit #105

Open

Description

@doublewhy

Problem

The packaged NASim red participant repeats one exploit, while every authored description of that participant names the run_bruteforce_agent baseline. On dev (0272949):

  • src/raes_adapters/nasim/examples/nasim-tiny/participant/nasim-red-bruteforce.configuration.json L18-28 has one value, red-action-contract-address = participant.action-contract.service-exploit.
  • src/raes_adapters/nasim/researcher.py L232-235 accepts only that contract, and _red_action_request sends normalized_arguments=() (L257), so no target reaches the driver.
  • The driver resolves a semantic action kind and an optional target to one flat action (src/raes_adapters/nasim/backend/driver.py L291-318). With no target it resolves the same flat action on every step: index 4, the exploit on host (1, 0).
  • The SDL describes the attacker as "run_bruteforce_agent baseline attacker cycling the flat action space" (nasim-tiny.sdl.yaml L136, L277), ledger rows 16 and 17 (mapping/source-ledger.jsonl) cite nasim/agents/bruteforce_agent.py, and the task (nasim-tiny.task.exp.json L6, experiment/nasim-tiny.task.exp.yaml L7) names the same baseline. That baseline cycles all 18 flat actions.

A probe recorded in #98 with the pinned nasim 0.12.0 wheel truncated at step 1000 with cumulative reward -1000 and the goal unreached. Every step costs 1 and root is never taken.

The fix changes a packaged configuration and the researcher run path, which the 2026-08-13 hold on #87 does not allow until OpenRAE/rae#1023 is settled.

Scope

  • Make the packaged red participant behave as the authored run_bruteforce_agent baseline (cycling the flat action space through portable actions and targets), or change the SDL, ledger rows 16 and 17 and the task to describe the behavior that actually runs.
  • Keep the configuration digest, pack content manifest and spec references consistent with the change.

Done when

  • The packaged red participant's behavior and the SDL, ledger and task descriptions agree.
  • A test drives the packaged configuration through the researcher run path and fails if every step resolves to the same flat action while the baseline is claimed.
  • Pack validation and the release checks pass.

Related

Activity

  1. added
    bugSomething isn't working
    blockedBlocked on an upstream gate or dependency
    on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedBlocked on an upstream gate or dependencybugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions