Skip to content

feat(gap9): untiled baseline, and fix two defects in PR #21's approach - #75

Merged
runwangdl merged 2 commits into
develfrom
feat/gap9-untiled-baseline
Sep 13, 2026
Merged

runwangdl merged 2 commits into
develfrom
feat/gap9-untiled-baseline

Conversation

@runwangdl

Copy link
Copy Markdown
Owner

Adds DEEPLOY_L1_AS_L2, the compile-time switch that turns a tiled deployment into the untiled baseline the tiling ablation needs.

What "untiled" means here

PULP cluster cores cannot dereference HyperRAM addresses, so an untiled L3 deployment is not "no movement". It is a single tile whose L1-annotated buffers are physically allocated in FC L2, with the cluster computing over the L2 fabric. The switch redirects pi_l1_malloc to pi_l2_malloc and replaces the mchan pushes with memcpy.

On PR #21

Its memcpy substitution is correct and I was wrong to call it lazy: the mchan hardware routes loc into L1 banks unconditionally, so with L1-as-L2 there is genuinely no DMA left to program. Two things were missing:

  1. Only mchan_transfer_push_1d was substituted, so any tile taking the 2-D path hung. push_2d is substituted too.
  2. The define never reached the code it was meant to switch. It was set with target_compile_definitions(${ProjectId} PRIVATE ...), but ProjectId compiles only CycleCounter.c and deeploytraintest.c. Every L1 allocation and every mchan call lives in the generated network code, which is a separate target (${NETWORK_LIB} / optimizer_network). Both arms therefore produced the same binary — DS-CNN measured 3,044,109 cycles untiled against 3,044,050 tiled, a 59-cycle "difference". It is now a directory-scope add_compile_definitions in DeeployTest/CMakeLists.txt; deeploylib is an INTERFACE target so it takes INTERFACE, not PUBLIC.

Measured once the define actually applied

GAP9, 8 cores, gvsoc, --l2 1024000, every run Errors: 0 out of 4:

network tiled untiled slowdown opt tiled opt untiled slowdown
Autoencoder 2,504,870 62,385,885 24.9× 1,241,834 31,799,906 25.6×
DS-CNN 3,044,050 45,917,437 15.1× 130,744 2,978,135 22.8×
CCT-2 256,721,885 2,093,477,099 8.2× 7,335,848 226,170,099 30.8×
MobileNetV1 177,150,391 2,171,487,469 12.3× 9,102,093 233,206,261 25.6×
ResNet-8 184,286,767 1,070,880,302 5.8× 3,193,912 85,103,850 26.6×

Two things worth reading off this:

  • The optimizer column is a near-constant 23–31× across all five networks. SGD is elementwise and memory-bound, with no arithmetic to hide the access latency, so that column reads out the TCDM-vs-L2-fabric cost directly.
  • The training column varies with arithmetic intensity, 5.8× to 24.9×. ResNet-8 is lowest because convolutions over large feature maps reuse each fetched byte many times; the autoencoder is highest because it is a pure MLP (0.99 MACs per parameter against 115–160 for the convolutional benchmarks). Tiling's benefit is not a fixed multiplier — it is a function of 1/arithmetic intensity.

The Autoencoder and DS-CNN rows predate PR #70's MLperf-compliant fixtures and are being re-measured; the three convolutional/transformer rows are unaffected (those fixtures are byte-identical).

🤖 Generated with Claude Code

https://claude.ai/code/session_012vb2GVs3qbiyLV1KkV2baN

runwangdl and others added 2 commits September 13, 2026 19:15
Adds DEEPLOY_L1_AS_L2, the compile-time switch that turns a tiled
deployment into the untiled baseline the tiling ablation needs.

On what "untiled" means here: PULP cluster cores cannot dereference
HyperRAM addresses, so an untiled L3 deployment is not "no movement". It is
a single tile whose L1-annotated buffers are physically allocated in FC L2,
with the cluster computing over the L2 fabric. The switch therefore
redirects pi_l1_malloc to pi_l2_malloc and replaces the mchan pushes with
memcpy.

PR #21 had the right idea and I was wrong to call its memcpy substitution
lazy: the mchan hardware routes `loc` into L1 banks unconditionally, so
with L1-as-L2 there is genuinely no DMA left to program. Two things were
missing from it:

- only mchan_transfer_push_1d was substituted, so any tile taking the 2-D
  path hung. push_2d is substituted too.
- the define was set with target_compile_definitions(${ProjectId} PRIVATE),
  but ProjectId compiles only CycleCounter.c and deeploytraintest.c. Every
  L1 allocation and every mchan call lives in the GENERATED network code,
  which is a separate target (${NETWORK_LIB} / optimizer_network). The
  define never reached the code it was meant to switch, and the two arms
  produced the same binary -- DS-CNN measured 3,044,109 untiled against
  3,044,050 tiled, a 59-cycle "difference". It is now a directory-scope
  add_compile_definitions in DeeployTest/CMakeLists.txt. deeploylib is an
  INTERFACE target, so it takes INTERFACE rather than PUBLIC.

Measured once the define actually applied (GAP9, 8 cores, gvsoc, every run
Errors: 0 out of 4):

  network       tiled        untiled       slowdown
  Autoencoder   2,504,870    62,385,885    24.9x
  DS-CNN        3,044,050    45,917,437    15.1x
  CCT-2       256,721,885 2,093,477,099     8.2x
  MobileNetV1 177,150,391 2,171,487,469    12.3x
  ResNet-8    184,286,767 1,070,880,302     5.8x

The optimizer step is 23-31x slower across all five, a near constant: SGD
is elementwise and memory-bound, so that column reads out the TCDM-vs-L2
access cost with nothing to hide it. The training column varies with
arithmetic intensity instead -- ResNet-8 lowest because convolutions over
large feature maps reuse each fetched byte many times, the autoencoder
highest because it is a pure MLP.

(The Autoencoder and DS-CNN rows above predate PR #70's MLperf-compliant
fixtures and are being re-measured.)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012vb2GVs3qbiyLV1KkV2baN
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012vb2GVs3qbiyLV1KkV2baN
@runwangdl
runwangdl merged commit c642537 into devel Sep 13, 2026
29 checks passed
@runwangdl
runwangdl deleted the feat/gap9-untiled-baseline branch September 13, 2026 21:30
runwangdl added a commit that referenced this pull request Sep 14, 2026
AllocateTemplate.py: take devel's version, which already carries this
branch's NULL check on the L1 arena allocation (#75).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013w8xXNTjEDT261WWj3QnJC
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant