Error messages with immutable `/etc`
abbradar opened this issue · comments
On NixOS we have most of /etc generated with our declarative configuration system and mounted read-only (sans /etc/{users,groups,shadow,gshadow} when we allow mutable users). For example, we have our GCE support described in one of our configuration modules.
Because of this approach parts of Guest Agent that set up system environment by modifying configuration files fail, producing error messages like those:
2022-01-10T14:38:39.5728Z GCEGuestAgent Error non_windows_accounts.go:107: Error creating google-sudoers file: open /etc/sudoers.d/google_sudoers: no such file or directory.
2022-01-10T14:38:39.6548Z GCEGuestAgent Error oslogin.go:91: Error updating SSH config: open /etc/ssh/sshd_config: read-only file system.
2022-01-10T14:38:39.6557Z GCEGuestAgent Error oslogin.go:99: Error updating PAM config: open /etc/pam.d/sshd: read-only file system.
2022-01-10T14:38:39.6558Z GCEGuestAgent Error oslogin.go:103: Error updating group.conf: open /etc/security/group.conf: no such file or directory.
2022-01-10T14:38:39.6564Z GCEGuestAgent Error oslogin.go:118: Error creating OS Login sudoers file: open /etc/sudoers.d/google-oslogin: no such file or directory.
Our current approach is adding instance configuration options for that, as described in #152. However, as @hopkiw mentioned extending these is to be avoided. What are the other possible solutions?
This is not a scenario we want to support in the guest agent. It would require significant additional implementation of options and configurations that would be specific to NixOS and we would not be continuing to maintain support for this sort of environment as we add future changes. It seems that with a sufficiently complex custom OS, you should be maintaining your own patches to the guest agent and compiling from source.
My vision is that this feature would be less about NixOS specifics and more about allowing distributions to opt out of "setup" phase of Guest Agent, in case their maintainers want to implement it by themselves. NixOS's declarative /etc is one example, but generally (in my experience) package maintainers try to set as much system configuration using their distribution's native tools as possible, to ease migration and possible clutter after package removal. Usually on-the-fly patching of system configuration (sans dynamic entries such as users) like implemented in Guest Agent is to be avoided, although of course Guest Agent's usage patterns are different from most packages out there.
An interesting possible alternative is to split Guest Agent into two parts -- an installation script, which does all the setup work and can be executed during package installation (or removal for cleanup) and the actual daemon part, which updates configuration based on metadata.
Still, I understand that such an undertaking is huge and in practice current way of doing it during daemon startup works well for overwhelming number of users and distributions -- so even our current patch may bring benefits too small to consider inclusion. If you decide to not support disabling or splitting of this so-called setup phase, we can maintain our out-of-tree patches instead.
Yes, the guest agent is a unique piece of software not intended for general consumption. A piece of software meant for general consumption may implement some paradigm similar to what you are suggesting, but the guest agent is targeted at the managed GCE user experience to provide maximum convenience. This is one reason it is not available in all distribution upstream repositories.
Unfortunately this proposal does not match our current intentions or our product requirements with the guest agent. Suggestions for features are welcome, but users who need more flexibility than we are able to offer must build their own agents, including carrying patches as necessary.
I understand your position, we will maintain our own patches. Thanks for the discussion!