Third-party plugin breakages in Certbot 2.x
alexzorin opened this issue · comments
#9161 removed the old IAuthenticator
, IInstaller
etc classes from certbot.interface
. We had a 14-month deprecation period for these classes and eventually removed them in the release of v2.0.0.
A number of third-party plugins no longer work on Certbot v2.0.0, due to them referencing the removed classes. We have received reports about this on the community forums, this issue tracker, and other project issue trackers.
@zope.interface.implementer(interfaces.IAuthenticator)
@zope.interface.provider(interfaces.IPluginFactory)
class Authenticator(dns_common.DNSAuthenticator):
Fixing this in Certbot v1.18.0 and later is simple: removing the zope
decorators. In the case of dns_common.DNSAuthenticator
, the plugin is still registered as an authenticator due to the way the new abc
-based classes work.
However, prior to Certbot v1.18.0, this results in the plugin no longer being registered:
# certbot certonly -d example.com --dry-run --preferred-challenges dns -a certbot-dns-inwx:dns-inwx
PluginEntryPoint#certbot-dns-inwx:dns-inwx does not provide IPluginFactory, skipping
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Could not choose appropriate plugin: The requested certbot-dns-inwx:dns-inwx plugin does not appear to be installed
The requested certbot-dns-inwx:dns-inwx plugin does not appear to be installed
Plugin authors are stuck on their ability to support both old Certbot and new Certbot. Some integration projects have pinned Certbot back to v1.32.0 or told users to install Certbot v1.32.0 instead of v2.x, in order to deal with this problem.
I think we should try to solve this issue for all the dns_common.DNSAuthenticator
-based certbot-dns-*
projects that appear to have real users (around 12,000+ based on UA statistics from April), if it is straightforward to do so.
My proposal is to reintroduce some of the removed zope
classes, without reintroducing zope itself. That approach can be seen here. The effect is that it will perform the required zope
-based class registration in Certbot < v1.18.0, and be a no-operation in newer versions.
One downside is that we cannot entirely fix the problem. I'm not sure whether we can do anything about plugins that inherit from certbot.plugins.common.Plugin
and then do zope
-based registration to IAuthenticator
or IInstaller
, without us backtracking on the zope
removal. Based on the stats, I think total breakage in that case is less than 700 users. This number excludes a bunch from the Nextcloud project who I think are not affected due to pinning Certbot in their snap (at least, I can't see any complaints about it).
This comment is a draft based on the proposed change.
My Certbot plugin doesn't work?
Plugin Type | Base Class | Action to support all versions of Certbot |
---|---|---|
Authenticator | `certbot.plugins.dns_common.DNSAuthenticator` |
Ensure your plugin depends on `zope.interface` and declare your authenticator in the following manner:
import zope.interface
from certbot import errors, interfaces
from certbot.plugins import dns_common
@zope.interface.implementer(interfaces.IAuthenticator)
@zope.interface.provider(interfaces.IPluginFactory)
class Authenticator(dns_common.DNSAuthenticator): |
Authenticator | `certbot.plugins.common.Plugin` | |
Installer | `certbot.plugins.common.Plugin` |
This plan sounds good to me. I think we potentially could try to do something for non-DNS plugins by doing clever/tricky things with our interfaces.I*
classes, but I personally don't think it's worth the trouble based on the number of people that should be affected.
Our messaging for plugin authors is a little subtle I think. I don't think plugins are required to inherit from any of our classes. Instead, they just need to register their implementation of the interface in certbot.interface
in a way that works for zope
and/or abc
. What I'd suggest is:
- We place this text in something like a community forum post we can link to so we can easily edit it in the future if needed.
- We briefly explain the switch to
abc
and the need to register with or inherit from these interfaces in Certbot 2.0+. We then say that if you want to also support Certbot<1.18.0, you can continue to depend onzope
and register your plugin with those interfaces as well. Finally, we can provide your example above as I think that's likely the only code snippet anyone will pull from the post.
What do you think?
Our messaging for plugin authors is a little subtle I think. I don't think plugins are required to inherit from any of our classes
This change only works because those DNS plugins are already inheriting from .dns_common.DNSAuthenticator
, which gives implicit abc
-based registration to .interfaces.Authenticator
.
Ditto with plugins inheriting from .common.Plugin
registering to .interfaces.Plugin
.
We otherwise lack a straightforward way for plugin authors to get registration to .interfaces.Authenticator
or .interfaces.Installer
in a way that is compatible with all versions of Certbot.
So:
- We are making a change in Certbot that should unbreak a bunch of popular third-party plugins, provided they inherit from
.dns_common.DNSAuthenticator
or.common.Plugin
and havezope.interface
as a dependency. - We don't have a full answer about creating widely compatible plugins. Every other plugin will need to make a decision about supporting older and newer versions of Certbot.
Does that seem accurate?
I don't particularly like (2). Having a compatibility adapter would be nice but I'm okay with it if we can capture most of the breakages in (1).
I agree with your (1) & (2).
Creating a way for plugins not based on .dns_common.DNSAuthenticator
to work with new and old Certbot versions without any changes to the plugin would be kind of ugly I think. The only idea that immediately comes to mind is to do something like trying to import zope
when loading plugins and being satisfied if the plugin claims to implement either the new interfaces or the old ones which would now just be empty shims. Since it seems the majority of 3rd party plugins are DNS plugins by a wide margin, I'm personally hesitant to do this.
We don't have a full answer about creating widely compatible plugins. Every other plugin will need to make a decision about supporting older and newer versions of Certbot.
Any plugin that wants to support new and old versions needs to depend on zope
, register themselves as implementing both the zope
and abc
interfaces, and do so in a way that works in new and old Certbot versions. It's not exactly pretty and how it would look exactly depends on what the plugin chooses to inherit from since I believe we have no strict requirements here, but for instance, if they were an installer that inherited from .common.Plugin
, I think one way to do this would be:
import zope.interface
from certbot import interfaces
from certbot.plugins import common
def try_registration(cls):
try:
interfaces.Installer.register(cls)
except AttributeError:
pass
return cls
@try_registration
@zope.interface.implementer(interfaces.IInstaller)
@zope.interface.provider(interfaces.IPluginFactory)
class Installer(common.Plugin):
...
What do you think? Do you want to either try using zope if it's available inside Certbot or also include a code snippet like this for non-DNS plugins?
Another idea I had was to publish a simple package on PyPI with decorators that handle all of this zope and conditional imports/accesses stuff for you, but that's yet another thing for us to maintain and I feel like the shim interfaces alone get us most of the way there due to the widespread subclassing of .dns_common.DNSAuthenticator
.
Thanks so much for all the iterations with this one.
I noticed that registering classes like this doesn't actually show up in .mro()
(i.e. in certbot plugins
output on the Interfaces:
line), but it doesn't seem to go beyond that display glitch - issubclass
still works correctly and the authenticators show up in the authenticators prompt.
I've opened a draft PR + draft forum post which reflects the strategy which I think we agreed upon here.
The forum post is a bit wordy, if you can see where places I can cut it down, I'd be glad to shorten it.