RQ3 · where the 124 GB pool bites

Unified-memory ceiling

Max-batch search (harness.fit): peak memory vs per-device batch, ascending until OOM. The iGPU and CPU share one 124 GB LPDDR5X pool.

Peak memory (GB) vs batch size — dashed line is the unified-memory pool
Activation memory grows ~linearly with batch (~14–15 GB/step). The 8B bs8 cross is the study’s only OOM, recorded as a failure (status: oom), not dropped.

Memory fits big batches; throughput won’t use them

The LoRA arms fit a surprisingly large micro-batch — bs8 for 4B (97% of the 124 GB pool), bs4 for 8B and 20B — but RQ2 showed every batch step above 1 lowers tok/s. So the practical limit is set by throughput, not memory: stay at batch 1 and the headroom is enormous.

Full-SFT memory is flat in batch. Gemma-3-12B sits at ~64 GB at bs1, bs2, and bs4 alike — with gradient checkpointing the peak is dominated by weights + grads + Adafactor state (fixed), not activations. So the heaviest arm has the most batch headroom; the limiter everywhere is throughput, not the pool. The only memory failure in the study is the bs8-8B OOM, recorded as data.

At batch 1, peak ranges 15 GB (4B LoRA sdpa) → 64 GB (12B full-SFT) — no arm near the ceiling. attn=sdpa buys ~7 GB for the LoRA arms, nothing for full-SFT.

← RQ2 · Scaling RQ4 · Thermal →