Can't Change ACME Challenge Type

Question

Can't Change ACME Challenge Type

blastagator opened this issue 4 years ago · comments

I wanted to change the existing Certera Cert to DNS challenge/validation. When I change the drop down and hit save it does not update. I thought this may be defined behavior for the Certera cert, so I tried making another cert for a different subdomain and then switching the challenge type which also failed.

Is this intended / needed behavior based on how LE works? If so, may want to tweak the drop down box to become grey/unchangeable once a cert is created.

Thanks!

certeraio · Answer 1 · Wed Oct 21 2020 06:45:53 GMT+0800 (China Standard Time)

I think this is a bug/omission on my part. Certera originally only did HTTP validation. I added DNS validation much later as a change in the codebase and looks like I forgot to add that ability when editing a cert. I can't see any reason why it can't be changed at will.
I'll flag this as a fix to make for a future release. In the meanwhile, I believe you should be able to delete and create the cert correctly. The key should be retained so that cert-pinning still works. It sucks, but at least that should work as a workaround.

Greg Wallace · Answer 2 · Wed Oct 21 2020 07:21:06 GMT+0800 (China Standard Time)

I backed up the certera folder to try your recommendation but trying to delete the site certificate yields "Cannot delete site certificate"
I may try to manually edit the database if I feel super motivated, or I can just make the http server available every 3 months (assuming this isn't fixed by then).
Thanks for helping and responding so fast!

certeraio · Answer 3 · Wed Oct 21 2020 08:03:39 GMT+0800 (China Standard Time)

Ah, sorry. I didn't get that it was the certera instance cert itself you were trying to change. I know there are limitations about being able to delete that one vs other certs, as you don't want to be unable to access the site.

I need to go back and double check whether the certera cert itself can be on dns validation. I know it starts off as HTTP since that's part of the initial setup. If you're trying to keep this mostly internal, I can see why you wouldn't want to open access for HTTP. Let me stew on this a bit more to see if having some dns config in place prior to setup would be sufficient to make DNS an option for the instance cert.

Greg Wallace · Answer 4 · Thu Oct 22 2020 03:04:08 GMT+0800 (China Standard Time)

Sure thing. I appreciate your help!

I am able to enable http support for the initial setup (and could do it manually for renewals), I'd just really prefer to have it fully automated like all of the other certs. I've also been writing some scripts for my various applications:

Cloudflare add / delete scripts
apache
plex

I'll start up a repo once I have them ironed out the way I like them and I'll share the link in case you (or anyone else using your great server) have interest. I'd love to help with the underlying project, but sadly I am not overly familiar with c or c# (aside from some very basic things).

certeraio · Answer 5 · Thu Oct 22 2020 04:55:20 GMT+0800 (China Standard Time)

It may be that the certera instance cert begins as HTTP validation, then it can be changed to DNS manually if the operator chooses (once the bug is fixed). I think that should all work. I'll check it out over the weekend and provide an update here.

As for the scripts, I'm sure it'll be helpful to others. I know there have been a few folks using different DNS providers and applying certs to various systems like nginx, IIS, etc. Thanks for your feedback and community contributions!

certeraio · Answer 6 · Wed Oct 28 2020 10:45:53 GMT+0800 (China Standard Time)

@blastagator I've created version 2.1.4 with the fix to change the certificate challenge type. You'll have to open things up for the initial provisioning, but then should be able to change the challenge type and lock things down. Hopefully that's a suitable workaround.

https://github.com/certera-io/certera/releases/tag/2.1.4

Greg Wallace · Answer 7 · Wed Oct 28 2020 11:21:07 GMT+0800 (China Standard Time)

Thanks! I installed the new version but after switching to dns challenge type the webui just shows "Error" when I try to request an update. Is there a log location I can check for more specific details? I verified another one of my certs still updates successfully, so the challenge type still seems to be working fine in general.

certeraio · Answer 8 · Wed Oct 28 2020 11:38:24 GMT+0800 (China Standard Time)

Check /var/syslog as it should put everything in there. The error will go a long way into figuring it out.

Greg Wallace · Answer 9 · Wed Oct 28 2020 11:54:23 GMT+0800 (China Standard Time)

I did a fresh request (to ensure I only captured the logging related to the error) and this is the output:

Oct 27 23:47:28 vm-certdp certera[22544]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 27 23:47:28 vm-certdp certera[22544]:       Error creating order
Oct 27 23:47:28 vm-certdp certera[22544]: System.NullReferenceException: Object reference not set to an instance of an object.
Oct 27 23:47:28 vm-certdp certera[22544]:    at Certera.Web.AcmeProviders.CertesAcmeProvider.BeginOrder()
Oct 27 23:47:28 vm-certdp certera[22544]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 27 23:47:28 vm-certdp certera[22544]:       [vm-certdp.home.gregtwallace.com] - error obtaining certificate:

certeraio · Answer 10 · Wed Oct 28 2020 12:10:29 GMT+0800 (China Standard Time)

Hmm, that's not too useful. You may need to change the Default log level in the app settings.json file to be Debug to get a little more info. I was hoping for a line number to make it easier.

I can try to reproduce it tomorrow too.

Greg Wallace · Answer 11 · Wed Oct 28 2020 12:31:33 GMT+0800 (China Standard Time)

Here is the log info with Debug level set. I'm not sure this is much more helpful though.

Oct 28 00:15:17 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:17 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - starting certificate acquisition
Oct 28 00:15:18 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:18 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - creating ACME order
Oct 28 00:15:20 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:15:20 vm-certdp certera[24834]:       Order created: https://acme-v02.api.letsencrypt.org/acme/order/99857896/5908461734
Oct 28 00:15:20 vm-certdp certera[24834]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:15:20 vm-certdp certera[24834]:       Error creating order
Oct 28 00:15:20 vm-certdp certera[24834]: System.NullReferenceException: Object reference not set to an instance of an object.
Oct 28 00:15:20 vm-certdp certera[24834]:    at Certera.Web.AcmeProviders.CertesAcmeProvider.BeginOrder()
Oct 28 00:15:20 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:20 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - setting DNS records
Oct 28 00:15:21 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:21 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - requesting ACME validation
Oct 28 00:15:21 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:21 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - completing order
Oct 28 00:15:21 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:21 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - cleaning up DNS records
Oct 28 00:15:21 vm-certdp certera[24834]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:21 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - error obtaining certificate:
Oct 28 00:15:21 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:15:21 vm-certdp certera[24834]:       [vm-certdp.home.gregtwallace.com] - done
Oct 28 00:15:47 vm-certdp certera[24834]: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Certera.Web.Services.HostedServices.DomainScanIntervalService[0]
Oct 28 00:15:47 vm-certdp certera[24834]:       Domain scanning job started.
Oct 28 00:15:47 vm-certdp certera[24834]: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Certera.Web.Services.HostedServices.DomainScanIntervalService[0]
Oct 28 00:15:47 vm-certdp certera[24834]:       No domains needing scan
Oct 28 00:15:47 vm-certdp certera[24834]: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Certera.Web.Services.HostedServices.DomainScanIntervalService[0]
Oct 28 00:15:47 vm-certdp certera[24834]:       Domain scanning job completed.

certeraio · Answer 12 · Wed Oct 28 2020 12:46:53 GMT+0800 (China Standard Time)

One thing that's interesting is this here: https://acme-v02.api.letsencrypt.org/acme/order/99857896/5908461734

In the identifiers, it says "type": "dns", which is correct.

Then, when you follow the "authorizations" link: https://acme-v02.api.letsencrypt.org/acme/authz-v3/8028529186

It shows http-01 in the challenges. I'm not sure if that's correct or not (if that shows up for all request types or only for HTTP-01). In my test quick test, I can't see much as some of those URLs don't work in when using the staging Let's Encrypt endpoints. I'll keep digging.

Greg Wallace · Answer 13 · Wed Oct 28 2020 13:09:08 GMT+0800 (China Standard Time)

Here is the log when I renew a different cert (that is successful):

Oct 28 00:56:37 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:56:37 vm-certdp certera[24834]:       [vm-apache-test.home.gregtwallace.com] - starting certificate acquisition
Oct 28 00:56:37 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:56:37 vm-certdp certera[24834]:       [vm-apache-test.home.gregtwallace.com] - creating ACME order
Oct 28 00:56:38 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:56:38 vm-certdp certera[24834]:       Order created: https://acme-v02.api.letsencrypt.org/acme/order/99857896/5909817639
Oct 28 00:56:39 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:56:39 vm-certdp certera[24834]:       [vm-apache-test.home.gregtwallace.com] - setting DNS records
Oct 28 00:57:26 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:57:26 vm-certdp certera[24834]:       Process completed with exit code 0
Oct 28 00:57:27 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:57:27 vm-certdp certera[24834]:       Process output:
Oct 28 00:57:27 vm-certdp certera[24834]:       /usr/local/lib/python2.7/dist-packages/lexicon/providers/googleclouddns.py:30: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
Oct 28 00:57:27 vm-certdp certera[24834]:         from cryptography.hazmat.backends import default_backend
Oct 28 00:57:27 vm-certdp certera[24834]:       RESULT
Oct 28 00:57:27 vm-certdp certera[24834]:       ------
Oct 28 00:57:27 vm-certdp certera[24834]:       True
Oct 28 00:57:27 vm-certdp certera[24834]:
Oct 28 00:57:27 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:57:27 vm-certdp certera[24834]:       [vm-apache-test.home.gregtwallace.com] - requesting ACME validation
Oct 28 00:57:27 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:57:27 vm-certdp certera[24834]:       [vm-apache-test.home.gregtwallace.com] - completing order
Oct 28 00:57:27 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:57:27 vm-certdp certera[24834]:       0 incomplete authorizations.
Oct 28 00:57:28 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:57:28 vm-certdp certera[24834]:       [vm-apache-test.home.gregtwallace.com] - cleaning up DNS records
Oct 28 00:57:28 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:57:28 vm-certdp certera[24834]:       Process completed with exit code 0
Oct 28 00:57:28 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 00:57:28 vm-certdp certera[24834]:       Process output:
Oct 28 00:57:28 vm-certdp certera[24834]:
Oct 28 00:57:28 vm-certdp certera[24834]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 00:57:28 vm-certdp certera[24834]:       [vm-apache-test.home.gregtwallace.com] - done

certeraio · Answer 14 · Wed Oct 28 2020 13:14:20 GMT+0800 (China Standard Time)

See how in the authz-v3 URL that the challenges are dns-01 type: https://acme-v02.api.letsencrypt.org/acme/authz-v3/8032295765

I need to see if a cert can be changed from starting under one challenge type and moving to another. I'll do some digging and let you know what I find out.

certeraio · Answer 15 · Thu Oct 29 2020 03:31:31 GMT+0800 (China Standard Time)

@blastagator I've created build 2.1.5: https://github.com/certera-io/certera/releases/tag/2.1.5

This should give better debugging information (file names and line numbers) in the exception. 🤞

I've received word that changing from using HTTP-01 to using DNS-01 should be just fine, i.e. there isn't some strange thing on Let's Encrypt that would prevent that.

If all else fails, I can try to add more logging to give visibility into what things are set to and what things look like. This should help shed more light.

Greg Wallace · Answer 16 · Thu Oct 29 2020 06:13:16 GMT+0800 (China Standard Time)

I installed the new version but it doesn't look like much changed in the log when trying to update:

Oct 28 18:11:04 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:04 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - starting certificate acquisition
Oct 28 18:11:05 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:05 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - creating ACME order
Oct 28 18:11:06 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 18:11:06 vm-certdp certera[57268]:       Order created: https://acme-v02.api.letsencrypt.org/acme/order/99857896/5908461734
Oct 28 18:11:07 vm-certdp certera[57268]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Oct 28 18:11:07 vm-certdp certera[57268]:       Error creating order
Oct 28 18:11:07 vm-certdp certera[57268]: System.NullReferenceException: Object reference not set to an instance of an object.
Oct 28 18:11:07 vm-certdp certera[57268]:    at Certera.Web.AcmeProviders.CertesAcmeProvider.BeginOrder()
Oct 28 18:11:07 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:07 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - setting DNS records
Oct 28 18:11:07 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:07 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - requesting ACME validation
Oct 28 18:11:07 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:07 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - completing order
Oct 28 18:11:07 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:07 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - cleaning up DNS records
Oct 28 18:11:07 vm-certdp certera[57268]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:07 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - error obtaining certificate:
Oct 28 18:11:07 vm-certdp certera[57268]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Oct 28 18:11:07 vm-certdp certera[57268]:       [vm-certdp.home.gregtwallace.com] - done
Oct 28 18:11:15 vm-certdp certera[57268]: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Certera.Web.Services.HostedServices.DomainScanIntervalService[0]
Oct 28 18:11:15 vm-certdp certera[57268]:       Domain scanning job started.
Oct 28 18:11:15 vm-certdp certera[57268]: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Certera.Web.Services.HostedServices.DomainScanIntervalService[0]
Oct 28 18:11:15 vm-certdp certera[57268]:       No domains needing scan
Oct 28 18:11:15 vm-certdp certera[57268]: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Certera.Web.Services.HostedServices.DomainScanIntervalService[0]
Oct 28 18:11:15 vm-certdp certera[57268]:       Domain scanning job completed.

Alexander Reinert · Answer 17 · Mon Jan 24 2022 16:31:10 GMT+0800 (China Standard Time)

Same issue here. Because of the logging in my DNS scripts I can see, that the scripts are not called.

Jan 24 08:54:38 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:38 dns certera[1562145]:       [certera.cyberterrorist.de] - starting certificate acquisition
Jan 24 08:54:38 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:38 dns certera[1562145]:       [certera.cyberterrorist.de] - creating ACME order
Jan 24 08:54:40 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Jan 24 08:54:40 dns certera[1562145]:       Order created: https://acme-v02.api.letsencrypt.org/acme/order/365194870/57261720160
Jan 24 08:54:40 dns certera[1562145]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.AcmeProviders.CertesAcmeProvider[0]
Jan 24 08:54:40 dns certera[1562145]:       Error creating order
Jan 24 08:54:40 dns certera[1562145]: System.NullReferenceException: Object reference not set to an instance of an object.
Jan 24 08:54:40 dns certera[1562145]:    at Certera.Web.AcmeProviders.CertesAcmeProvider.BeginOrder()
Jan 24 08:54:41 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:41 dns certera[1562145]:       [certera.cyberterrorist.de] - setting DNS records
Jan 24 08:54:41 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:41 dns certera[1562145]:       [certera.cyberterrorist.de] - requesting ACME validation
Jan 24 08:54:41 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:41 dns certera[1562145]:       [certera.cyberterrorist.de] - completing order
Jan 24 08:54:41 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:41 dns certera[1562145]:       [certera.cyberterrorist.de] - cleaning up DNS records
Jan 24 08:54:41 dns certera[1562145]: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:41 dns certera[1562145]:       [certera.cyberterrorist.de] - error obtaining certificate:
Jan 24 08:54:41 dns certera[1562145]: #033[40m#033[37mdbug#033[39m#033[22m#033[49m: Certera.Web.Services.CertificateAcquirer[0]
Jan 24 08:54:41 dns certera[1562145]:       [certera.cyberterrorist.de] - done

For me, it seems like the (already expired) HTTP authorization of the current order will be reused and no new DNS authorization will be created.