Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion docs/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -337,14 +337,50 @@ message tag.
| `_ChipWorker.init()` | Load sim context and host runtime, then bind each module's logger state | `src/common/worker/chip_worker.cpp` |
| `simpler_init` | Onboard maps the bound threshold to CANN; attach and take executor binaries | `src/common/platform/{onboard,sim}/host/c_api_shared.cpp` |
| Nested host load | Bind generated host orchestration/AICore logger state before entry | runtime maker / sim device runner |
| AICPU init | Sim binds the live host state; onboard snapshots CANN policy | platform AICPU init |
| AICPU init | Sim binds the live host state; onboard snapshots CANN policy **and latches `InitArgs.log_level` once for the Worker's life** | platform AICPU init |

The Python level is still sampled during worker initialization. Calling
`logger.setLevel(...)` does not itself call the native setter; recreate or
reinitialize the worker to apply a new Python configuration. Within a process,
all bound host modules observe a native state update immediately instead of
requiring threshold fan-out to every DSO.

### The threshold is live on the host and fixed on the device

Two different lifetimes, and the boundary is the silicon rather than the
language:

| reader | when a `set_level` takes effect |
| ------ | ------------------------------- |
| Every host module in the process, including a `dlopen`ed one | **immediately** — each reads `state()->threshold` on every record |
| Sim AICPU | **immediately** — it runs in the host process as a bound consumer of the same state |
| **Onboard AICPU** | **never; it keeps the threshold it was given at device init** |

Onboard AICPU receives the threshold once, in `InitArgs.log_level`, and
`simpler_aicpu_init` pushes it into a device-side flag. That entry runs once per
Worker, so the value it latched is the value for that Worker's life.

**This is a design decision, not a gap.** Raising verbosity mid-run to chase a
device-side problem is not a workflow this runtime supports: recreate the Worker
with the level you want. Two reasons it is not worth supporting:

- A `Worker` is cheap to recreate, so the workaround costs nothing a debugging
session would notice.
- Device log volume is exactly where an accidental `DEBUG` is most expensive —
`codestyle.md` rule 7 forbids logging on AICPU hot paths precisely because
`device_log` writes serialize on the single AICPU op and can trip the
op-execute timeout. A threshold that can be raised into that from outside is a
liability rather than a feature.

Delivering it would also cost more than it looks. The threshold could ride an
existing per-run payload, but making it *immediately* live needs either a device
launch per `set_level` — the opposite direction from #2092, which exists to make
`simpler_aicpu_init` launch exactly once — or a new host-writable device-resident
location polled outside any launch.

So: **any claim that "the log level is live" is scoped to host modules.** State
it that way when writing one.

### The Python logger is a client, not a second system

Seeding the native threshold also installs a handler on the `simpler` logger that
Expand Down
2 changes: 2 additions & 0 deletions src/a2a3/platform/onboard/aicpu/kernel.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,8 @@ extern "C" __attribute__((visibility("default"))) int simpler_aicpu_init(void *a
}

InitArgs *init_args = reinterpret_cast<InitArgs *>(arg);
// One-shot for the Worker's life: this entry runs once and no host-side
// set_level reaches here afterwards. Deliberate — see docs/logging.md.
set_log_level(static_cast<int>(init_args->log_level));
set_orch_device_id(static_cast<int>(init_args->device_id));
set_scheduler_timeout_ms(static_cast<int>(init_args->scheduler_timeout_ms));
Expand Down
2 changes: 2 additions & 0 deletions src/a5/platform/onboard/aicpu/kernel.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@ extern "C" __attribute__((visibility("default"))) int simpler_aicpu_init(void *a
}

InitArgs *init_args = reinterpret_cast<InitArgs *>(arg);
// One-shot for the Worker's life: this entry runs once and no host-side
// set_level reaches here afterwards. Deliberate — see docs/logging.md.
set_log_level(static_cast<int>(init_args->log_level));
set_orch_device_id(static_cast<int>(init_args->device_id));
set_scheduler_timeout_ms(static_cast<int>(init_args->scheduler_timeout_ms));
Expand Down
8 changes: 8 additions & 0 deletions src/common/log/include/host_log.h
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,14 @@ class SIMPLER_HOST_LOG_LOCAL HostLogger {
// Owner initialization starts the bounded writer by default. Hierarchical
// workers defer it until their final local fork so no process forks with a
// C++ thread already running.
//
// Takes effect immediately for every host module in the process, since each
// reads the bound state's threshold per record — sim AICPU included, because
// it is one of them. It does NOT reach onboard AICPU, which latches
// InitArgs.log_level once in simpler_aicpu_init and keeps it for the
// Worker's life; that is deliberate, not missing (see docs/logging.md,
// "The threshold is live on the host and fixed on the device"). Recreate the
// Worker to change the device threshold.
void set_level(simpler::log::LogLevel level, bool defer_writer = false);
bool start_writer();

Expand Down
4 changes: 4 additions & 0 deletions src/common/platform/onboard/host/device_runner_base.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -567,6 +567,10 @@ int DeviceRunnerBase::ensure_aicpu_init_launched() {

InitArgs init_args{};
init_args.device_id = static_cast<uint32_t>(device_id_);
// The device threshold is set here and never again: this entry launches once
// per Worker, so a later host-side set_level does not reach the AICPU. That is
// the intended contract, not a missing refresh — recreate the Worker to change
// it. docs/logging.md records why.
init_args.log_level = static_cast<uint32_t>(HostLogger::get_instance().level());
// Per-device scheduler watchdog override, resolved once at attach into
// timeout_config_. 0 -> the AICPU scheduler keeps its compile-time default.
Expand Down
Loading