nvmf-connect-nbft.service service dependencies
tbzatek opened this issue · comments
This is a follow-up on #2078
Cc: @mwilck
I've been trying to get nvmf-connect-nbft.service
working on RHEL 9 that is based on NetworkManager. Turns out the network-online.target
only fires once at boot since in the Boot-from-NBFT scenario at least one network interface (say nbft0
for illustration) is always up, thus satisfying the definition of Network is Online. (network-online.target
, see https://systemd.io/NETWORK_ONLINE/).
In reality, once a secondary network interface comes up (say nbft1
) no action is taken, except of NetworkManager-dispatcher.service
activation that doesn't trigger any other actions/services/units.
Ultimately my questions are:
- is the
nvmf-connect-nbft.service
supposed to react on any "link up" or "interface configured (DHCP)" events? - is this supposed to be a one-time-off service only as a secondary connect measure after switchroot?
- I see no interface name matching, reacting to events on any network interface?
Does this work this way with wicked
on your side? I'm trying to find a network management software-universal solution first.
My testing scenario looks like this:
- boot qemu with two network interfaces up, let the UEFI connect both defined boot attempts so that the ACPI NBFT table lists two HFIs
- while at the grub menu, set the link on a second interface down (e.g. through qemu console
set_link net1 off
command) - boot into linux, let the dracut nvmf module time out on a second connection
- after boot, bring the link up
- expected:
NetworkManager-dispatcher.service
sets the interface up andnvmf-connect-nbft.service
is triggered
1. is the `nvmf-connect-nbft.service` supposed to react on any "link up" or "interface configured (DHCP)" events?
No. This is what we should ultimately implement. "network online" is obviously much too coarse. I am unsure what the requirements would be though. There are no uevents for this type of event. So I guess we'd need some daemon listing for netlink events (?) and taking appropriate action.
iSCSI hasn't been able to come up with a solution to this problem in 20 years.
2. is this supposed to be a one-time-off service only as a secondary connect measure after switchroot?
Yes.
3. I see no interface name matching, reacting to events on any network interface?
No.
Does this work this way with
wicked
on your side? I'm trying to find a network management software-universal solution first.
Wicked includes a unit called wicked.service
which is ordered before network-online.service
. It's a oneshot
service that runs wicked --systemd ifup all
. This will try to bring up all interfaces that are configured as "mandatory". The details (like timeouts waiting for the interfaces) are configured in wicked's own config file. When this command finishes (and thus the unit is considered started), most interfaces will likely be up (although there's no guarantee).
I suppose that this is the difference wrt NM, where network-online.target
is reached if just a single interface is up.
Great, thanks for your answers, it's a good starting point now that we know where do we stand.
Noting some hints to follow-up on based on the recent Timberland SIG call:
- create a systemd unit for each interface that could be used as a dependency?
- handle DHCP lease renewal with a different IP address (--> i.e. do we want to tear down the kernel connection to prevent excessive recovery attempts?)
- look how iSCSI multipathing is handled across distros
All my experiments to find an elegant systemd-only upstream-ready solution led to nowhere. Even if I manage to come up with a systemd unit for a particular network interface and watch for link changes (very nasty solution, don't ask), the interface would not have the necessary addresses set up (at least in RHEL9 w/ NetworkManager) and any calls to nvme-cli would fail anyway.
Thus I resorted for a custom NM hook that I plan to keep downstream for the time being.
Let's close this until somebody comes with a good solution.
Thanks. I'd be grateful if you could publish your downstream solution, perhaps in the timberland repos somewhere?