NVMe topologies

[Eli Gerber]

Hi,

I have a question regarding different NVMe topologies.
Up until recently I iterated only over nvme devices under a controller, but after I installed new devices for which my discovery procedure failed I found that these devices appear directly under the subsystem hierarchy and not uder a controller.
My question - Is this a valid topology? if so what does it mean that a device appears/does not appear under a controller hierarchy?
can it point to a configuration issue? compatiblity issue?
I am using libnvme version 1.0 BTW if it matters.

Thanks in advance to all helpers!

[Keith Busch]

If your devices claim to support multipath, and your kernel is using native nvme multipath, then namespaces will appear under subsystem rather than controller.

[Daniel Wagner]

You can check the supported features of the device via the id-ctrl command:

# nvme id-ctrl -H /dev/nvme0
NVME Identify Controller:
[...]
cmic      : 0
  [3:3] : 0x1   ANA supported
  [2:2] : 0     PCI
  [1:1] : 0x1   Multi Controller    // XXX multipath device
  [0:0] : 0x1   Multi Port
[...]

and the kernel support via

# cat /sys/module/nvme_core/parameters/multipath 
Y

[Eli Gerber]

Thanks guys, you helped me a lot!