block  /  storage & volumes controllers/block
Chapter 04

Block, Storage & Volumes

Talos has no /etc/fstab and runs no install scripts. Storage is a declarative, controller-reconciled resource graph: disks, partitions, tmpfs mounts, directories, overlays and encrypted volumes are all VolumeConfig documents that the VolumeManagerController drives toward a mounted VolumeStatus. The mental model is everything-is-a-volume — and the on-disk design is immutable + ephemeral + state: a read-only squashfs root, writable overlays, and a tiny encrypted partition that holds the node's identity.

#The immutable + ephemeral + state design

A Talos node's operating system never writes to itself. The OS root is a read-only squashfs (rootfs.sqsh), loop-mounted read-only — nothing under /usr is writable, ever. The image is reproducible and tamper-evident, and "configuration drift" of the system is structurally impossible. The single line that enforces this is in the mount helpers:

internal/pkg/mount/v3/helpers.gogo
dev, err := losetup.Attach(squashfsFile, 0, true) // true => read-only loop

But a Kubernetes node obviously needs to write — kubelet state, container images, etcd data, a handful of system config directories. Talos splits those needs across three tiers, each backed by a different mechanism:

🧊

Immutable root

The squashfs is loop-mounted read-only. Paths that must be writable (/usr/libexec/kubernetes, /opt, and config overlays like /etc/kubernetes, /etc/cni) are overlayfs mounts listed in constants.Overlays, with their upper dirs backed by EPHEMERAL.

📌

STATE — node identity

STATE mounts at /system/state and holds the machine config plus encryption metadata. It is small, persistent, and typically LUKS2-encrypted. This is what makes a wiped-and-reinstalled disk still this node.

🌊

EPHEMERAL — all the churn

EPHEMERAL mounts at /var and holds container/kubelet/etcd data and every overlay upper-dir. It is provisioned to grow into all free space. To "reset" a node you wipe EPHEMERAL; STATE and META survive.

Beneath those sits META: a 1 MiB partition of out-of-band key/value entries that survives a wipe. META is special enough to be exempt from volume teardown, and — crucially — it is where the encryption configuration for STATE is stashed so that an encrypted STATE can be unlocked on first boot before its own config is even readable. The net effect: a reproducible, tamper-evident OS image; node identity in a small encrypted STATE; and all mutable churn isolated in a disposable EPHEMERAL.

the running rootfs — a stack, not a disk overlayfs (writable) /usr/libexec/kubernetes · /opt · /etc/kubernetes · /etc/cni — upper dirs on EPHEMERAL read-only squashfs root rootfs.sqsh · losetup.Attach(file, 0, true) · /usr never writable overlays + /var backed by partitions below the physical disk — one GPT, partitioned by the volume manager EFI /boot/EFI bootloader / UKI ~100 MiB survives wipe ✓ META (no mount) out-of-band kv 1 MiB survives wipe ✓ STATE /system/state machine config · enc meta 100 MiB · XFS · LUKS2 survives wipe ✓ EPHEMERAL /var containers · kubelet · etcd · overlay upper-dirs min 2 GiB · Size:0 Grow:true → all free space wiped on reset ✗
The running root is a read-only squashfs with writable overlays on top; the disk below carries EFI · META · STATE · EPHEMERAL. Only EPHEMERAL is disposable.

#The partition table

Partition labels come from pkg/machinery/constants/constants.go and GPT type GUIDs from internal/pkg/partition/constants.go. The full layout for a GRUB-booted node:

LabelMountGPT type GUIDSizeHolds
EFI/boot/EFIC12A7328-F81F-11D2-BA4B-00A0C93EC93B100 MiB (GRUB) / 101 MiB+ (UKI)bootloader / UKI
BIOS21686148-6449-6E6F-744E-6565644546491 MiBGRUB stage-2 (BIOS)
BOOT/boot0FC63DAF-8483-4772-8E79-3D69D8477DE4bootSizekernel / initramfs (GRUB)
METALinuxFilesystemData1 MiBout-of-band META kv (survives wipe)
STATE/system/stateLinuxFilesystemData100 MiB (XFS)machine config, encryption meta
EPHEMERAL/varLinuxFilesystemDatamin 2 GiB, growscontainer / kubelet / etcd, overlays

With UKI (Unified Kernel Image) boot, the BIOS and BOOT partitions disappear entirely — kernel, initramfs and bootloader fold into a single larger EFI partition. EPHEMERAL is always provisioned with Size: 0, Grow: true so it consumes the largest contiguous block of free space, and SELinux labels (EphemeralSelinuxLabel, StateSelinuxLabel) are baked into the filesystems at format time. The three GUID constants the partition code allocates against:

internal/pkg/partition/constants.gogo
const (
	EFISystemPartition  Type = "C12A7328-F81F-11D2-BA4B-00A0C93EC93B"
	BIOSBootPartition   Type = "21686148-6449-6E6F-744E-656564454649"
	LinuxFilesystemData Type = "0FC63DAF-8483-4772-8E79-3D69D8477DE4"
)

#Discovery: disks & volumes

Before anything can be provisioned, the machine has to know what hardware it has. Two controllers feed the volume manager. DisksController (controllers/block/disks.go) opens each block device and records its size, I/O size, sector size, read-only flag, model, serial and WWID into a Disk resource. DiscoveryController (controllers/block/discovery.go) blkid-probes every device and emits a DiscoveredVolume describing any filesystem or partition signature it already carries. The VolumeManagerController then consumes both — plus SystemDisk, DevicesStatus, PCRStatus and EncryptionSalt — as the inputs to reconciliation.

A whole controller family.

Registered in v1alpha2_controller.go: DevicesController, DiscoveryController, DisksController, MountController, MountRequestController, MountStatusController, SymlinksController, SystemDiskController, UserDiskConfigController, VolumeConfigController, VolumeManagerController and VolumeTrimController. This page focuses on the manager and its helpers — the rest are the COSI plumbing around it, the same engine the COSI runtime chapter dissects.

#The volume phase machine

Every volume is driven through a small state machine. VolumeStatusSpec.Phase is one of eight values declared in pkg/machinery/resources/block/volumephase.go:

pkg/machinery/resources/block/volumephase.gogo
const (
	VolumePhaseWaiting     VolumePhase = iota // waiting
	VolumePhaseFailed                         // failed
	VolumePhaseMissing                        // missing
	VolumePhaseLocated                        // located
	VolumePhaseProvisioned                    // provisioned
	VolumePhasePrepared                       // prepared
	VolumePhaseReady                          // ready
	VolumePhaseClosed                         // closed
)

The happy path is Waiting → Located → Provisioned → Prepared → Ready, with Failed and Closed off to the side. The heart of volume_manager.go is a single switch in processVolumeConfig that runs exactly one helper per phase — each helper does its work and advances the volume to the next phase:

internal/app/machined/pkg/controllers/block/volume_manager.gogo
case block.VolumePhaseWaiting, block.VolumePhaseMissing:
	if err := volumes.LocateAndProvision(ctx, logger, volumeContext); err != nil { return err }
case block.VolumePhaseLocated:
	if err := volumes.Grow(ctx, logger, volumeContext); err != nil { return err }
case block.VolumePhaseProvisioned:
	if err := volumes.HandleEncryption(ctx, logger, volumeContext); err != nil { return err }
case block.VolumePhasePrepared:
	if err := volumes.Format(ctx, logger, volumeContext); err != nil { return err }

Read it top to bottom: LocateAndProvision finds an existing volume or carves a new partition; Grow expands a located partition to its target size; HandleEncryption formats or opens LUKS2; Format lays down the filesystem (xfs / ext4 / vfat). Once a volume reaches Ready, the MountController performs the actual kernel mount. On a retryable error a volume flips to Failed but stashes its PreFailPhase so the next tick resumes exactly where it left off, and a shouldRetry flag re-runs the offender on a 30-second ticker.

Waiting / Missing Located adopt / carved Provisioned partition exists Prepared decrypted Ready formatted LocateAnd Provision Grow Handle Encryption Format MountController kernel mount → /var, /system/state… Failed PreFailPhase stashed retry on 30s ticker Closed teardown reversed
One helper per phase. A Ready volume is handed to MountController; a Failed volume stashes PreFailPhase and resumes on the 30s ticker.

#CEL selection & provisioning waves

The clever part lives inside LocateAndProvision (internal/volumes/locate.go). It is deliberately locate-or-provision and idempotent: first it tries to locate an existing volume by evaluating the config's CEL Locator.Match against every DiscoveredVolume; only if nothing matches does it provision a new one — picking a target disk by evaluating the CEL DiskSelector.Match against each Disk, then calling CreatePartition to carve GPT space. Because adoption is tried before creation, re-applying a config or reinstalling the OS does not blow away EPHEMERAL — the existing partition is simply located again.

CEL everywhere.

Both disk selection and volume location are CEL expressions evaluated against the disk/volume specs — e.g. disk.transport == "nvme" or disk.size > 100u * GiB. This is what makes "install on the biggest SSD" a declarative one-liner instead of a script that greps lsblk.

Volumes are also wave-ordered. Each carries a wave number — WaveSystemDisk = -1, WaveUserVolumes = 0, WaveLegacyUserDisks = 1000000 — and a wave is not considered fullyProvisionedWave until all its volumes reach Ready. The manager refuses to start a later wave until earlier ones are complete, so EPHEMERAL (wave −1) is fully provisioned before any user volume (wave 0) is touched. This is what prevents a user volume from racing EPHEMERAL for the same free space. Disk locking reinforces it: CreatePartition and HandleEncryption each take a 10-second exclusive lock, and lock contention is treated as Retryable — it self-heals on the same 30s ticker.

Teardown is the machine run backwards. A VolumeLifecycle resource plus COSI finalizers drive Close, which unmounts the volume, closes any LUKS mapping, and walks the phase back to Closed. META is exempt from teardown — it has to outlive everything else because it bootstraps STATE's encryption.

#LUKS2 encryption

Talos supports exactly one encryption provider: LUKS2 (EncryptionProviderLUKS2; None is a pass-through). An EncryptionSpec carries numbered LUKS key slots, and each slot is an EncryptionKey with a Type. The dispatch lives in keys.NewHandler — a single switch that builds the right key handler:

internal/pkg/encryption/keys/keys.gogo
switch cfg.Type {
case block.EncryptionKeyStatic:
	handler = NewStaticKeyHandler(key, k)
case block.EncryptionKeyNodeID:
	handler = NewNodeIDKeyHandler(key, opts.VolumeID, opts.GetSystemInformation)
case block.EncryptionKeyKMS:
	handler, err = NewKMSKeyHandler(key, cfg.KMSEndpoint, opts.GetSystemInformation)
case block.EncryptionKeyTPM:
	handler, err = NewTPMKeyHandler(key, cfg.TPMCheckSecurebootStatusOnEnroll, cfg.TPMPCRs, opts.TPMLocker)
}
if cfg.LockToSTATE {
	handler = NewSaltedHandler(handler, opts.SaltGetter)
}

The four key types span the spectrum from convenient to hardware-bound:

🔑

static

A passphrase carried directly in the machine config. Simplest, least secure — the key is in the config.

🆔

nodeID

Derived from the SMBIOS UUID plus the partition label: encryption.NewKey(slot, []byte(nodeUUID+partitionLabel)). No secret in config, but bound to the chassis.

🛰️

kms

Sealed and unsealed by a remote KMS over gRPC at KMSEndpoint. The node must reach the KMS to unlock.

🔐

tpm

Sealed to a TPM 2.0, optionally bound to PCRs and secure-boot state — the disk only unlocks on a machine whose measured boot matches.

The LockToSTATE flag wraps any handler in a SaltedHandler that mixes in the per-node EncryptionSalt, binding the derived key to this node's STATE partition. Unlocking is handled by HandleEncryptionWithHandler: it locks the device and blkid-probes it; if there is no filesystem it calls FormatAndEncrypt, and if it is already LUKS it opens it. Handler.Open runs tryHandlers, iterating the configured key handlers in slot order until one succeeds. It then calls syncKeys to converge the LUKS key slots toward the configured set — and here is a real gotcha: a single failing slot (say an unreachable KMS) is not fatal. The volume still mounts via a working slot, and the failure is surfaced in VolumeStatusSpec.EncryptionFailedSyncs rather than blocking boot. The opened mapping appears as luks2-<volumeID> and that device path becomes the volume's Status.MountLocation.

The chicken-and-egg of encrypted STATE.

On first boot the STATE partition is encrypted but its own config hasn't been read yet — because the config lives inside STATE. Talos breaks the cycle with META: the STATE encryption configuration is recovered from META via UnmarshalEncryptionMeta, so STATE can be unlocked before it is readable. This is precisely why META is exempt from teardown and survives wipes.

#User volumes

Beyond the system volumes, operators declare their own storage with a UserVolumeConfig v1alpha1 document (config/types/block/user_volume_config.go). Each becomes a partition labeled u-<name> (the UserVolumePrefix = "u-") and is auto-mounted at /var/mnt/<name>. The document carries a name, a volumeType (directory / disk / partition), a provisioning block (CEL DiskSelector plus min/max/grow sizing), a filesystem (xfs by default), and optional encryption, mount and trim settings. A UserVolumeTransformer turns each one into a VolumeConfig at WaveUserVolumes — so it flows through the exact same phase machine as EPHEMERAL, just one wave later. Sibling document kinds round out the model: RawVolumeConfig, ExistingVolumeConfig, ExternalVolumeConfig and SwapVolumeConfig.

Two final wrinkles.

Config-only mounts get hardened with nosuid,nodev,noexec (the Secure mount flag), while hosts that must execute code — /opt, kubelet plugin dirs — opt out. And in container mode, where there are no real block devices, partition-backed volumes gracefully degrade to plain directory volumes so the same configs still reconcile.

That is the whole storage story: a read-only image, a tiny encrypted identity, a disposable data partition, and one phase machine that locates-or-provisions every volume on the disk. The other controller family doing this much work is the network stack — next, Chapter 05 traces how Talos reconciles addresses, links, routes and the rest of the machine's networking the same declarative way.