SYNOPSIS
/usr/local/etc/nvme/nvme-fabrics.conf /usr/local/etc/nvme/nvme-fabrics.conf.d/*.conf
DESCRIPTION
An INI-format file describing which NVMe over Fabrics (NVMe-oF) controllers a host should connect to, and with what parameters. This file is read by nvme connect-all, nvme discover and nvme connect, and by the nvme-discoverd daemon.
Other orchestrators read the same format through the same parser, but from a file of their own: nvme-stas reads its configuration through the Python bindings. The file name is a parameter of every entry point, so each orchestrator names its own and is unaffected by the others'.
A configuration is a main file plus an optional drop-in directory, the same foo.conf + foo.conf.d/ pattern systemd uses. Both parts are read independently, and either may be absent; an absent configuration is not an error; it simply yields no connections.
--config FILE (on connect-all/discover/connect) names the main file; its drop-in directory is always FILE.d/. Omitting --config uses the default path above.
SECTIONS
Each section is one or more key = value lines. An empty value (key =) resets the key to the kernel default, overriding whatever an outer scope set (see RESETTING A KEY below). Unknown keys are ignored. Boolean values accept 1/yes/y/true/t/on and 0/no/n/false/f/off (case-insensitive).
- [Host]
-
Optional, at most one per file. The identity used by every connection in that file: hostnqn, hostid, hostsymname, plus any SECURITY key below. A file with no [Host] connects as the system default identity (/usr/local/etc/nvme/hostnqn / hostid). A drop-in that defines [Host] must set hostnqn explicitly.
- [Discovery Controller Defaults], [I/O Controller Defaults]
-
Optional, at most one of each per file. Default TUNABLE values for connections of that class, drawn from below. A drop-in’s copy overlays the top-level file’s, for that drop-in’s own connections only.
- [Discovery Controller]
-
One Discovery Controller to connect to. Repeatable. nqn = defaults to the well-known discovery NQN if omitted.
- [Subsystem]
-
One I/O subsystem to connect to, named by nqn =. Repeatable.
Both endpoint sections accept any TUNABLE or SECURITY key as a per-section override, plus one or more controller = lines (below). A section with no controller = line makes no connection. It can still give the KX-HMAC-CHAP secrets of a discovered controller (see DISCOVERED CONTROLLERS below).
CONTROLLER LINES
Each controller = line under a [Discovery Controller] or [Subsystem] section is one path to that endpoint; repeating the key expresses multipath (native NVMe multipath groups all paths sharing a subsysnqn). The value is a ;-separated key=value list:
controller = transport=tcp;traddr=10.0.0.9;trsvcid=4420;host-iface=eth0
transport and traddr are required; trsvcid, host-traddr and host-iface are optional. Any TUNABLE key may also appear on the line as a per-path override — SECURITY keys may not, since they are bound to the (host, subsystem) pair, not to a single path.
KEYS
TUNABLE (type default, [Host], endpoint section, or controller = line):
nr-io-queues, nr-write-queues, nr-poll-queues, queue-size,
keep-alive-tmo, reconnect-delay, ctrl-loss-tmo, fast-io-fail-tmo,
tos, duplicate-connect, disable-sqflow, hdr-digest, data-digest
SECURITY ([Host] or endpoint section only, never a controller = line):
tls, tls-key, tls-key-identity, keyring, concat, kxchap-secret,
kxchap-ctrl-secret, tls-key-source
IDENTITY ([Host] only):
hostnqn, hostid, hostsymname
Every key here except tls-key-source is the same name as its nvme connect long option (hyphenated). See nvme-connect(1) for what each one means.
tls-key-source says where the TLS PSK comes from:
- inline
-
The default. tls-key is the key itself.
- systemd-creds
-
tls-key is the name of a systemd credential. nvme-keysd reads the credential and puts the PSK in the keyring. The connection does not use tls-key: the kernel finds the PSK in the keyring by the host NQN and the subsystem NQN. See nvme-keysd(8).
- kmip:<name>
-
tls-key is the Name of a key on the key management server (KMS) <name>, configured in nvme-keysd.conf. nvme-keysd gets the key and puts it in the keyring. kmip alone is enough when only one KMS is configured. See nvme-keysd(8).
Any other value also means that tls-key is not the key. libnvme does not check the value: nvme-keysd does.
RESETTING A KEY
An empty assignment (key =) resets that key to the kernel default, undoing whatever an outer scope (a *Defaults section or [Host]) set. This differs from simply not writing the key, which instead inherits through the cascade below.
PRECEDENCE
A key resolves most-specific first:
controller = line > endpoint section > [Host] > type defaults > kernel default
The type-defaults level is chosen by the connection’s class: a Discovery Controller draws from [Discovery Controller Defaults], an I/O Controller from [I/O Controller Defaults].
DISCOVERED CONTROLLERS
A controller found through discovery (an I/O controller or a referral in a discovery log page, or a Discovery Controller found through mDNS) has no section of its own. It gets kxchap-secret and kxchap-ctrl-secret from the endpoint section with the same nqn =: a [Subsystem] section for an I/O controller, a [Discovery Controller] section for a Discovery Controller. The section must be in the same file as the Discovery Controller that reported the controller (the top-level file for mDNS), because a secret belongs to one (host, subsystem) pair. The section’s controller = lines do not matter: the addresses come from discovery.
A secret that no such section sets comes from the Discovery Controller: its kxchap-secret for nvme connect-all, the defaults of its file for nvme-discoverd. The kxchap-ctrl-secret of the Discovery Controller is never used. A [Discovery Controller] section with the well-known discovery NQN never matches.
A Discovery Controller with the well-known discovery NQN allows only unidirectional authentication. A [Discovery Controller] section with that NQN, or with no nqn =, may set kxchap-secret. If it sets kxchap-ctrl-secret, a warning is logged.
Two sections of the same type with the same NQN in one file share their secrets. If they set different values for the same secret, the file is rejected.
[Subsystem]
nqn = nqn.2026-10.com.example:sub1
kxchap-secret = DHHC-1:00:...
kxchap-ctrl-secret = DHHC-1:00:...
EXAMPLE
# /usr/local/etc/nvme/nvme-fabrics.conf
[Discovery Controller Defaults]
keep-alive-tmo = 30
[I/O Controller Defaults]
keep-alive-tmo = 5
[Host]
hostsymname = lab-host-01
[Discovery Controller]
controller = transport=tcp;traddr=192.168.1.10;trsvcid=8009
[Subsystem]
nqn = nqn.2024-01.com.example:data.vol1
controller = transport=tcp;traddr=192.168.1.20;trsvcid=4420
controller = transport=tcp;traddr=192.168.1.21;trsvcid=4420
See /usr/local/etc/nvme/nvme-fabrics.conf.sample for a longer, commented example.
SEE ALSO
nvme-connect-all(1) nvme-discover(1) nvme-connect(1) nvme-config-convert(1) https://github.com/linux-nvme/nvme-cli/blob/master/libnvme/design/CONFIG.md
NVME
Part of the nvme-user suite