Methods & reproducibility — Strix Halo throughput

1 · Hardware

A single box (“trixi”): a Corsair AI Workstation 300 — AMD Ryzen AI Max+ 395 APU with the Radeon 8060S integrated GPU (gfx1151, RDNA 3.5, no FP8), 124 GB usable unified LPDDR5X (~215 GB/s, shared by CPU and GPU), Ubuntu, single-node k0s. The unified-memory pool and its bandwidth are central to the findings (see RQ3 and the batch-size result in RQ2).

Shared box. The same iGPU also serves an LLM via a llamacpp Deployment; the single iGPU can’t train and serve at once (concurrent compute segfaults). Every run frees the GPU first (kubectl scale deploy/llamacpp --replicas=0) and restores it after. A watchdog re-scales the service back up on a ~30-min cadence, so long runs re-assert the scale-down and verify no contention (throughput is flat across the boundary).

2 · Software stack

All runs go through one Docker image and wrapper so the ROCm/torch stack and device mapping are identical every time. The working build is the axolotl image (torch 2.8.0+rocm 7.12) driving plain transformers + peft (the hf engine). An earlier unsloth image’s torch 2.7.1 segfaulted at first kernel dispatch on gfx1151 — a broken build, not a hardware limit.

The one env flag that matters for RQ1: gfx1151 is “experimental” for AOTriton, so without TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 torch’s SDPA flash/efficient backends report “no available kernel” and attn=sdpa silently falls back to the math kernel — which would void the eager-vs-AOTriton A/B. The toolbox defaults it on; it’s captured in every run’s environment manifest.

3 · How a run is measured

Steady state over a fixed step count, never epochs: discard warmup steps (compile/warmup), then time a fixed measure window. Time-to-first-step is reported separately.
≥3 repeats per A/B config; we report mean and spread, not a lone number.
Time-boxed at ≤15 min per run so a misconfig can’t burn hours. The thermal run is the one sanctioned 60-min exception.
Both real and padded tok/s are recorded; the charts use real (non-pad) tokens/sec. A synthetic fixed-length corpus keeps token accounting exact.
Failures are data: OOM / NaN / time-box each produce a record with a status field, never a silent drop. The harness is config-driven, idempotent, and resumable (results keyed by a hash of the config).

Per-arm window sizing. Step time grows with model size and method (4B ~11 s, 20B ~15 s, 8B ~19 s, 12B full-SFT ~88 s per optimizer step), so the measurement window is sized per arm to fit the 900-s box — the workload (batch / grad-accum / seq) is unchanged, only the number of timed steps. The secondary sweep uses a still-shorter window, which reads a few % higher in absolute tok/s; compare within a sweep, not across.

4 · Per-arm configuration

Each arm’s baseline config, pinned to an exact Hugging Face revision. Derived from the run records:

arm	model @ revision	method	lora	seq	bs×ga	optim	warmup+measure	A/B winner

5 · Per-arm bring-up (what each arm needed)

Two arms needed framework fixes before they would train on gfx1151 — each surfaced one at a time by the smoke run, all now in the harness:

GPT-OSS-20B (MoE): explicit LoRA target_modules (PEFT can’t infer them for gpt_oss); Mxfp4Config(dequantize=True) (MXFP4 weights can’t be trained packed — dequantized to bf16, ~54 GB); and experts_implementation="eager" (the default grouped-GEMM MoE path calls torch._grouped_mm, “not supported on ROCM”). gpt-oss has no SDPA path, so it’s eager-only.
Gemma-3-12B (full SFT): requires token_type_ids as a training input — supplied as all-zeros (text) by the collator for this architecture.

6 · Reproduce it

From the repo on the GPU box (everything runs through ./toolbox.sh):

# free the shared GPU (always restore =1 when done)
sudo k0s kubectl -n llamacpp scale deploy/llamacpp --replicas=0

# 1. validate the environment + a tiny smoke run (confirms the SDPA toggle dispatches a fast kernel)
./toolbox.sh python3 -m harness.validate --smoke configs/arms/qwen3-4b.yaml

# 2. primary eager-vs-AOTriton A/B (per arm)  →  RQ1
./toolbox.sh python3 -m harness.sweep --config configs/sweeps/eager-vs-aotriton.yaml

# 3. secondary sweep (seq / batch / rank / packing)  →  RQ2
./toolbox.sh python3 -m harness.sweep --config configs/sweeps/secondary-qwen3-4b.yaml

# 4. max-batch search (memory ceiling)  →  RQ3
./toolbox.sh python3 -m harness.fit --config configs/arms/qwen3-8b.yaml --candidates 1,2,4,8

# 5. bounded thermal long run (the sanctioned 60-min exception)  →  RQ4
./toolbox.sh python3 -m harness.run --config configs/thermal-qwen3-4b.yaml

# 6. regenerate the analysis tables from the raw log
./toolbox.sh python3 -m analysis.report

sudo k0s kubectl -n llamacpp scale deploy/llamacpp --replicas=1   # restore serving

Sweeps are idempotent and resumable — a crash or OOM mid-sweep loses nothing; re-running skips finished configs. Off-GPU, the orchestration is unit-tested (make test, 19 passing) without a GPU.

7 · This site

Self-contained and data-driven. site/build_site.py reads data/runs.jsonl and emits site/data.js; the pages render hand-built SVG from it — no figure is hand-typed, and the whole site regenerates from a clean checkout:

python site/build_site.py # data/runs.jsonl → site/data.js

← Data Overview →