dev-certificates on Linux -- how to get dotnet-to-dotnet comms to work?
fluffynuts opened this issue · comments
Description
I have an asp.net core application with a self-hosted IdentityServer component. I can convince a browser (Chrome and Firefox) to visit the site, but I'm having trouble getting the IdentityServer component to be able to talk back to localhost due to certificate errors, specifically:
fail: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[3]
Exception occurred while processing message.
System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://localhost:44303/.well-known/openid-configuration'. ---> System.IO.IOException: IDX20804: Unable to retrieve document from: 'https://localhost:44303/.well-known/openid-configuration'. ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
I realise that certificate installation can differ from distro to distro, however, I've tried this first on my (more native) Gentoo, and also in an Ubuntu vm:
- export the dev cert to a pfx with
dotnet dev-certs https -ep /tmp/dnc.pfx -p SomePassword - extract the crt from the pfx with
openssl pkcs12 -in /tmp/dnc.pfx -clcerts -nokeys -out /tmp/dnc.crt - copy to /usr/local/share/ca-certificates and run update-ca-certificates. In both cases (Gentoo and Ubuntu), see a message about 1 added certificate
3.5 in the case of Gentoo, I was also able to use certutil to install the crt into my ~/.pki/nssdb; in the case of Ubuntu, this dir does not exist and I'm not sure how to bootstrap it - Fire up the asp.net site with angular frontend -- on Gentoo, where certutil worked, Chrome happily visits the site; on both (iirc) I had to add a security exception for Firefox and store, but could get to the site.
- Attempts to log in fail because all auth requests go through the primary api, which then attempts to contact the same host/port (localhost:44303) for IdentityServer comms -- and that's where the failure occurs with the message above: I can get no further
To Reproduce
Set up an asp.net core project with self-hosted IdentityServer on Linux, such that the config for identity server points back to the same hosting application, using dev certs. Any route requiring authorization will result in a 500 error with the above logging in place. Using dotnet dev-certs https --trust on Windows, same code, no problem.
Expected behavior
I expect to be able to get dotnet-to-dotnet comms working with the dev cert. I'm probably missing something )':
Ubuntu dotnet info:
.NET Core SDK (reflecting any global.json):
Version: 2.2.103
Commit: 8edbc2570a
Runtime Environment:
OS Name: ubuntu
OS Version: 18.10
OS Platform: Linux
RID: ubuntu.18.10-x64
Base Path: /usr/share/dotnet/sdk/2.2.103/
Host (useful for support):
Version: 2.2.1
Commit: 878dd11e62
.NET Core SDKs installed:
2.2.103 [/usr/share/dotnet/sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
Gentoo dotnet info:
.NET Core SDK (reflecting any global.json):
Version: 2.2.103
Commit: 8edbc2570a
Runtime Environment:
OS Name: ubuntu
OS Version: 18.10
OS Platform: Linux
RID: ubuntu.18.10-x64
Base Path: /usr/share/dotnet/sdk/2.2.103/
Host (useful for support):
Version: 2.2.1
Commit: 878dd11e62
.NET Core SDKs installed:
2.2.103 [/usr/share/dotnet/sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
I have also tried with dotnet binaries from https://dotnet.microsoft.com/download/thank-you/dotnet-sdk-2.2.103-linux-x64-binaries
If it would be helpful, I can possibly set up a self-hosting IdentityServer project at some point, given a little time. I can't supply the full code I'm working on because it's for a client.
@fluffynuts Were you able to resolve this in any way?
I have the same exact problem.
@maq1823 no, I didn't )': I've read that this is an issue within dotnet itself (specifically to do with cert trusting on linux), but I don't know any more about it. I eventually had to stop development under linux and boot into windows as this was a show-stopper preventing any progress on the project.
AFAIK, the only solution right now would be to have a proper certificate issued from an authority and use a hosts file redirection for that domain.
I'm having the same issue - I can find a lot of details for how to enable host-to-container SSL trust, but not for container-to-container trust.
Facing the same issue. I am trying to setup ASP.NET Core 3.0 Web Api, with self-hosted OpenIddict middleware, for handling OAuth token generation.
In Windows, things work fine, even with dotnet generated TLS certificate.
But things fall apart in Linux environment - I tried with WSL (ubuntu 18.04), as well as in a Ubuntu 18.04 VM. I created local self-signed certificate and setup the system to trust it - but still the issue persists.
There is no issue while trying to access the application URL, Token URL etc.
Once I get the token, and then make a cURL call to one of the protected API endpoints, application tries to internally talk to https://localhost:44320/.well-known/openid-configuration to get the OAuth configuration, and BOOM, it fails estabilish TLS session, as it thinks that the remote certificate is inavlid.
fail: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[3]
Exception occurred while processing message.
System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://localhost:44320/.well-known/openid-configuration'. ---> System.IO.IOException: IDX20804: Unable to retrieve document from: 'https://localhost:44320/.well-known/openid-configuration'. ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
This is blocking development/deployment of my project in Linux environment. Would really appreciate some one looking/responding to it.
Is this about development time or when deploying for production? The dev certs probably shouldn't be used for production purposes.
@poke development time. Unless the use cases work we can't be even thinking about prod.
https://github.com/dotnet/corefx/issues/40725
You need to trust it in libcurl I believe. Regardless, it's a corefx issue not an aspnet one.
@blowdart I retried by asking libcurl to trust the self signed certificate:
curl -X --insecure --cacert path/to/self-signed.crt GET \
https://localhost:44320/api/somesecureaction \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <token>'
But the issue still happens:
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/2 --insecure https://localhost:44320/api/Account/UserData
fail: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[3]
Exception occurred while processing message.
System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://localhost:44320/.well-known/openid-configuration'. ---> System.IO.IOException: IDX20804: Unable to retrieve document from: 'https://localhost:44320/.well-known/openid-configuration'. ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Security.SslStream.ThrowIfExceptional()
at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
--- End of inner exception stack trace ---
at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(String address, IDocumentRetriever retriever, CancellationToken cancel)
at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)
--- End of inner exception stack trace ---
at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)
at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
The problem does not seem to related to libcUrl though, since the following works fine:
curl -X GET https://localhost:44320/.well-known/openid-configuration
The problem happens when JWT middleware internally tries to fetch config from OAuth endpoint, by making HTTPS call to: https://localhost:44320/.well-known/openid-configuration.
I have already setup ASPNET to use my self-signed cert:
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder => {
webBuilder.UseKestrel(options => {
var certificate = new X509Certificate2("localhost.pfx", "<password>");
options.Listen(IPAddress.Loopback, 44320, listenOptions => {
listenOptions.UseHttps(certificate);
});
});
webBuilder.UseStartup<Startup>();
});
But somehow that does not stop JWT middleware to throw the error.
@terrajobst this appears to be a corefx problem rather than aspnet core. Can someone explain how to get corefx to trust a self signed cert?
I don't understand what the repro is exactly here. Can you attach the dev certificate so we can look at it?
Can someone explain how to get corefx to trust a self signed cert?
The problem is most likely not a "trust" issue. We have see that installing self-signed certificates into the trust list on Linux doesn't work because the certificate itself is usually missing some X509v3 attributes that Linux (due to OpenSssl) require but Windows doesn't. OpenSsl on Linux requires that self-signed certificates have the "Cert Signing" attribute on them. This is because a self-signed, trusted certificate has to be both a valid CA certificate (requires 'Cert Signing') as well as a "leaf node" certificate.
@davidsh please find attached the dev certificate file with which the issue happens.
dev-cert.zip
It would also be very helpful if you could point to any article where the proper step for generating self-signed certificates are provided, that would make linux/openssl happy.
I have this issue too. But for me is issue:
System.IO.IOException: The decryption operation failed, see inner exception. ---> Interop+OpenSsl+SslException: Decrypt failed with OpenSSL error - SSL_ERROR_SSL. ---> Interop+Crypto+OpenSslCryptographicException: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
--- End of inner exception stack trace ---
at Interop.OpenSsl.Decrypt(SafeSslHandle context, Byte[] outBuffer, Int32 offset, Int32 count, SslErrorCode& errorCode)
at System.Net.Security.SslStreamPal.EncryptDecryptHelper(SafeDeleteContext securityContext, ReadOnlyMemory`1 input, Int32 offset, Int32 size, Boolean encrypt, Byte[]& output, Int32& resultSize)
--- End of inner exception stack trace ---
at System.Net.Security.SslStreamInternal.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)
at Microsoft.AspNetCore.Server.Kestrel.Core.Adapter.Internal.AdaptedPipeline.ReadInputAsync(Stream stream)
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.ReadAsync(CancellationToken token)
at System.IO.Pipelines.Pipe.DefaultPipeReader.ReadAsync(CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.BeginRead(ValueTask`1& awaitable)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequestsAsync[TContext](IHttpApplication`1 application)
It would also be very helpful if you could point to any article where the proper step for generating self-signed certificates are provided, that would make linux/openssl happy.
So, I've researched this and determined that on Linux, self-signed certificates will not work unless you have "Certificate Signing" as an additional attribute on the KeyUsage field.
This limitation is not a .NET Core thing. It is an OpenSsl change (between 1.0.2e and 1.0.2g) where self-signed certificates need to have this additional attribute set on them. Even when you install the certificate (public key portion) using 'update-ca-certificates' on Linux, the certificate will still not be "trusted" if the "Certificate Signing" bit is missing.
There is an open issue still on the OpenSsl repo about this problem: openssl/openssl#1418
This is a client-side problem. For example, I created an ASP.NET Core website using kestrel and hosted it on my Windows box. I first used the default "ASP.NET Core HTTPS development certificate" for the website. I then installed the certificate (public key portion) into the trusted certificates list on my Linux machine using 'update-ca-certificates'. When I used 'curl' to send a request to the website, I got an error.
When I changed the website to use a self-signed certificate that has the added "Certificate Signing" attribute, then 'curl' worked from the Linux machine.
I got the same results using HttpClient from the Linux machine. I got errors about the certificate when I used the default ASP.NET Core development certificate. But when I updated the certificate to include "Certificate Signing" then HttpClient worked again.
As a result of this investigation, we need to fix the 'dotnet dev-certs' tool so that it will generate a self-signed certificate that can be used on both Windows and Linux properly. @Tratcher where is the source code to this tool? Perhaps I can submit a PR to fix that.
Instead of using the default ASP.NET development certificate, you can generate your own self-signed certificate using Windows PowerShell or Linux OpenSsl tools. Just make sure to specify the 'Certificate Signing' attribute in addition to other things.
For example, this is the command I used to generate a self-signed certificate on Windows.
New-SelfSignedCertificate -DnsName "localhost" -KeyUsage CertSign, DigitalSignature, KeyEncipherment -FriendlyName "Better ASP.NET Core HTTPS development certificate" -CertStoreLocation "Cert:\CurrentUser\My"I then exported the public key portion and installed that on my Linux machine and ran 'update-ca-certificates' to put it into the trusted list.
cc: @wfurt
As @davidsh mentioned, OpenSSL is more picky about correct flags. If curl does not work without -k (insecure) option, .Net will not work either. I look at one @sntnupl and it does not seems to have what is needed.
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
15:01:f7:60:0f:09:82:85:5a:e6:ab:1e:ba:1e:a4:16:5b:1d:9e:06
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, ST=New York, L=Rochester, O=localhost, OU=Development, CN=localhost
Validity
Not Before: Sep 20 01:17:47 2019 GMT
Not After : Sep 19 01:17:47 2020 GMT
Subject: C=US, ST=New York, L=Rochester, O=localhost, OU=Development, CN=localhost
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:d0:26:ea:7d:0c:a2:67:81:d1:4b:41:2c:2f:ac:
e3:f9:81:77:96:4c:df:dc:df:20:39:d5:e6:1c:4a:
b3:34:60:a3:77:b2:e2:c0:3c:a6:25:59:0b:90:6f:
6b:9d:50:8c:d1:b3:45:95:e0:f6:30:b6:d3:5e:4c:
f2:a6:8c:a6:1c:25:f8:42:b2:e7:cb:88:dc:95:6f:
f2:37:5c:dc:40:62:cb:b1:57:16:1b:0b:03:63:90:
e4:3f:13:74:1c:e7:ff:97:e9:67:57:f5:7f:e0:ac:
84:1d:23:5d:03:c9:ea:04:4f:60:75:8a:77:8f:b0:
94:b4:27:4e:08:c5:6d:f3:1f:e5:02:a4:3b:74:a2:
6e:7d:b2:a3:e8:96:19:a3:94:2e:49:f6:a1:e0:b1:
46:ce:9e:c9:21:b8:ea:28:4a:25:ad:3d:6c:c0:1b:
88:12:5a:f3:1d:d5:fc:6b:e7:b5:f6:ef:66:b6:01:
fa:ef:71:b0:ed:d9:71:65:98:22:64:09:4e:e8:c4:
ba:73:c0:bb:31:24:57:6c:69:cc:18:a4:c1:8e:eb:
70:e3:5f:a5:2a:8f:31:df:4b:a4:2f:d4:a2:26:fc:
77:9c:c0:01:f1:7a:55:29:70:c1:4e:76:5c:f5:f0:
6c:cb:5d:14:b7:c0:07:08:1a:89:cd:91:ba:67:96:
bb:0b
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Alternative Name:
DNS:localhost, DNS:127.0.0.1
Signature Algorithm: sha256WithRSAEncryption
There is longer discussion about this here: https://github.com/dotnet/corefx/issues/37516
I check certificate generated by asp.net on my system and I get:
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 Extended Key Usage: critical
TLS Web Server Authentication
X509v3 Subject Alternative Name: critical
DNS:localhost
1.3.6.1.4.1.311.84.1.1:
my suggestions would be to clear old certificate, create new set, verify v3 extensions and verify that curl can connect without -k.
my suggestions would be to clear old certificate, create new set, verify v3 extensions and verify that curl can connect without -k.
As I discussed above, the current 'dotnet dev-certs' tool is generating a self-signed certificate that works only on Windows but can't work on Linux due to OpenSsl requirements that self-signed certificates have the additional 'Certificate Signing' bit set on them. So, "clear old certificate, create new set" will not work at all unless that new set contains the additional 'Certificate Signing' bit.
And as you pointed out, the current ASP.NET 'dotnet dev-certs' tool is not creating a certificate that can make Linux clients happy.
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
That Key Usage attribute is missing the 'Certificate Signing' bit.
So, the best fix for everyone is to 1) understand the problems that OpenSsl on Linux has regarding self-signed certificates, and 2) change our 'dotnet dev-certs' tool so that the automatically generated ASP.NET Core development certificate has the proper attributes so that it works on both Windows and Linux. That will reduce pain for developers.
cc: @bartonjs
If this were to change it should only be on Linux. Creating a trusted cert which can issue other certs goes beyond self signed, it's creating a root CA and that is dangerous. I'd argue OpenSSL isn't being correct here, but it's easier for us to cope with their bad security decision than it is to get it rolled back, but if we do so we don't want to expose Windows users to additional risks just because of OpenSSL.
As for Linux to Windows communication, the dev certificate is limited to localhost anyway, so it should be rejected for cross machine communication, unless you're disabling subject validation which is not something we want to encourage at all, so I view that as out of scope and where folks need to start considering their own development certificate infrastructure and creation.
If this were to change it should only be on Linux. Creating a trusted cert which can issue other certs goes beyond self signed, it's creating a root CA and that is dangerous.
What's more interesting is that the certificate itself has a Basic Constraint that says it isn't a CA but an end-entity certificate.
It's just that OpenSsl has some logic that says any self-signed certificate can't be "trusted" unless it has the "Certificate Signing" attribute in the KeyUsages field.
I'm going to explore other possibilities for creating the self-signed certificate to see what works on both Windows and Linux.
Perhaps if the Basic Constraints were changed from
"Subject Type=End Entity, Path Length Constraint=None"
to
" Subject Type=End Entity, Path Length Constraint=0"
then it would further reduce risk if someone tried to use this development certificate to issue further certificates from it.
@davidsh thanks for your reply.
Following your advice, I created crt file with KeyUsage field having Certificate Signing" attribute.
My development platform is linux, so I used openssl to do the same.
Here's what the config file looked like:
[req]
default_bits = 2048
default_keyfile = localhost.key
distinguished_name = req_distinguished_name
req_extensions = req_ext
x509_extensions = v3_ca
[req_distinguished_name]
countryName = Country Name (2 letter code)
countryName_default = US
stateOrProvinceName = State or Province Name (full name)
stateOrProvinceName_default = New York
localityName = Locality Name (eg, city)
localityName_default = Rochester
organizationName = Organization Name (eg, company)
organizationName_default = localhost
organizationalUnitName = organizationalunit
organizationalUnitName_default = Development
commonName = Common Name (e.g. server FQDN or YOUR name)
commonName_default = localhost
commonName_max = 64
[req_ext]
subjectAltName = @alt_names
[v3_ca]
basicConstraints = CA:FALSE
subjectAltName = @alt_names
keyUsage = keyCertSign, nonRepudiation, digitalSignature, keyEncipherment
[alt_names]
DNS.1 = localhost
DNS.2 = 127.0.0.1
I transferred the crt file to windows machine, and checked the KeyUsage field, and Certificate Sigining was indeed added.
However, the issue continues to happen. Same exception as I mentioned in my preview comments.
I used curl with --insecure field, but it didn't help.
Can there be any other attribute missing?
Cert file attached.
dev-cert-updated.zip
@davidsh thanks for your reply.
Following your advice, I created crt file with KeyUsage field having Certificate Signing" attribute.
How did you generate the self-signed certificate exactly? What was the openssl command you used? What version of openssl do you have?
Is the self-signed certificate (*.PFX file with private key) installed on the Linux machine? How/where did you install that? Is the Linux machine the server machine?
Is the self-signed certificate public key portion (*.CRT file) installed in the machine trusted CA store with commands similar to?
sudo cp ~/dev-cert.crt /usr/local/share/ca-certificates/dev-cert.crt
sudo update-ca-certificatesHow did you generate the self-signed certificate exactly? What was the openssl command you used? What version of openssl do you have?
I used the following commands (localhost.conf commented above)
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout localhost.key -out localhost.crt -config localhost.conf -passin pass:crypticpassword
sudo openssl pkcs12 -export -out localhost.pfx -inkey localhost.key -in localhost.crt
Delete previously installed certificate named "localhost"
certutil -D -d sql:${HOME}/.pki/nssdb -n "localhost"
Install new certificate
certutil -D -d sql:${HOME}/.pki/nssdb -n "localhost"
I am using OpenSSL version 1.1.1
Is the self-signed certificate (*.PFX file with private key) installed on the Linux machine? How/where did you install that?
Yes, using the commands above.
Is the Linux machine the server machine?
Yes.
Is the self-signed certificate public key portion (*.CRT file) installed in the machine trusted CA store
Not installed but I call curl like this:
curl -X GET --insecure --cacert ~/.aspnet/localhost.crt https://localhost:44320/api/Account/User -H 'Accept: application/json' -H 'Authorization: Bearer <Access_Token>'
Is this an ASP.NET Core server? How is the ASP.NET Core Server finding the self-signed certificate (w/ private key)?
I've never tried your particular set of instructions, i.e. using 'certutil' on the Linux machine.
What happens if a Windows machine tries to access this Linux http://localhost:44320 server? And assume that you have installed the self-signed certificate (public key portion) into the Windows trusted root store. Will that work without any browser errors?
Yes, it is a ASP.NET Core server.
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder => {
webBuilder.UseKestrel(options => {
var certificate = new X509Certificate2("localhost.pfx", "crypticpassword");
options.Listen(IPAddress.Loopback, 44320, listenOptions => {
listenOptions.UseHttps(certificate);
});
options.AddServerHeader = false;
});
webBuilder.UseStartup<Startup>();
});
Above code shows how the server finds the certificate.
I did not try connecting a windows client to this server. But I had, a few weeks back, follow similar steps to generate a self signed cert in windows (using this Microsoft blog/tutorial), and same code worked fine windows client w/ windows server.
Also, as I mentioned above,
curl -X GET --insecure https://localhost:44320/.well-known/openid-configuration works fine.
I call another api endpoint to get access token, and then, as I make an api call to access a protected end point,
ASP.NET Core JWT middleware internally tries to fetch config from OAuth endpoint, by making HTTPS call to: https://localhost:44320/.well-known/openid-configuration. The issue happens here. Only in Linux.
Using curl --insecure is invalid test. You need to get to point where trust is esteablished and curl just works. Without it, you will keep getting validation errors.
So, I did a test where I created a sef-signed certificate on Windows using PowerShell. This self-signed certificate has the 'Certificate Signing' attribute on it.
PS C:\Users\davidsh> New-SelfSignedCertificate -DnsName "localhost" -TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.1","2.5.29.19= {critical}{text}false") -KeyUsage CertSign, DigitalSignature, KeyEncipherment -FriendlyName "V2 ASP.NET Core HTTPS development certificate" -CertStoreLocation "Cert:\CurrentUser\My"I then ran an ASP.NET Core web server on Windows using this self-signed certificate.
I was able to access it from a Linux WSL window using 'curl' and making sure the self-signed certificate was trusted (via passing the .crt via a parameter to curl).
davidsh@davidsh2:/mnt/d/dotnet/web1$ curl --cacert ./localhost-v2.crt --verbose https://localhost:5001/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5001 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: ./localhost-v2.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=localhost
* start date: Oct 11 17:27:55 2019 GMT
* expire date: Oct 11 17:47:55 2020 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: CN=localhost
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fffbe609b40)
> GET / HTTP/2
> Host: localhost:5001
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Fri, 11 Oct 2019 17:44:35 GMT
< server: Kestrel
<
* Connection #0 to host localhost left intact
Hello World!davidsh@davidsh2:/mnt/d/dotnet/web1$I will try your openssl instructions to generate the self-signed certificate on Linux itself. And then try to run the Linux server and use curl against it.
But our respective repros are not actually the same because you have a Linux solution where you have the ASP.NET server do an outbound request back to itself over TLS. Can you post a simple complete repro for that?
We have too many variables at work here and we need to get a standardized repro.
I am using OpenSSL version 1.1.1
Also, what distro are you using? Can you show the 'dotnet --info' output?
I did another test using only Linux machines (actually 2 separate Linux distros using WSL on Windows).
Server:
Runtime Environment:
OS Name: debian
OS Version: 9
OS Platform: Linux
RID: debian.9-x64
OpenSSL 1.1.0j 20 Nov 2018
Then I created a new ASP.NET Core application:
dotnet new web --name web1
I generated the public/private key using your 'localhost.conf' and 'openssl' instructions above.
I set the following environment variables so that ASP.NET Core would use my *.pfx file I generated:
export ASPNETCORE_Kestrel__Certificates__Default__Password=password
export ASPNETCORE_Kestrel__Certificates__Default__Path=localhost.pfx
Then I started the server with "dotnet run"
My client machine is a second WSL machine:
Hello World!davidsh@davidsh2:/mnt/d/dotnet/web1$ more /etc/os*
NAME="Ubuntu"
VERSION="18.04.1 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.1 LTS"
OpenSSL 1.1.0g 2 Nov 2017
Then I used 'curl' and was able to do an HTTPS request to my first Linux machine. The '/mnt/d/localhost.crt' is the public key of the self-signed certificate that I made on the first Linux WSL machine.
davidsh@davidsh2:/mnt/d/dotnet/web1$ curl --cacert /mnt/d/localhost.crt https://localhost:5001/
Hello World!davidsh@davidsh2:/mnt/d/dotnet/web1$
Update
I then did a modification to the ASP.NET Core server I created above on the first machine. I added another mapped endpoint. This new endpoint will do an outbound request using HttpClient back to itself.
endpoints.MapGet("/outbound", async context =>
{
using var client = new HttpClient();
string result = await client.GetStringAsync("https://localhost:5001/");
await context.Response.WriteAsync($"Outbound result: {result}\n");
});I also made sure to install the self-signed certificate (public key) portion into the machine's trusted store on this first Linux WSL machine.
davidsh@davidsh2:~/dotnet/web1$ sudo cp localhost.crt /usr/local/share/ca-certificates
davidsh@davidsh2:~/dotnet/web1$ ls /usr/local/share/ca-certificates
localhost.crt
davidsh@davidsh2:~/dotnet/web1$ sudo update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Then I repeated the experiment using the second Linux WSL machine and using curl.
davidsh@davidsh2:/mnt/d/dotnet/web1$ curl --cacert /mnt/d/localhost.crt https://localhost:5001/outbound
Outbound result: Hello World!
Finally, to simply things on the second Linux WSL machine, I then installed the self-signed certificate (public key) portion into its trusted store (similar to what I did on the first Linux WSL machine). Now, I can use a simpler 'curl' command since the certificate is now trusted on this machine:
davidsh@davidsh2:/mnt/d/dotnet/web1$ sudo cp /mnt/d/localhost.crt /usr/local/share/ca-certificates
davidsh@davidsh2:/mnt/d/dotnet/web1$ sudo update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
davidsh@davidsh2:/mnt/d/dotnet/web1$ curl https://localhost:5001/outbound
Outbound result: Hello World!
davidsh@davidsh2:/mnt/d/dotnet/web1$
So, this repro is similar to yours in that the Linux ASP.NET Core server does a TLS request back to itself.
Try reproducing my results using your own Linux machines. It should work.
@davidsh Many thanks for your detailed explanation!
I am pleased to inform that after installing the self-signed certificate (public key) portion into the machine's trusted store, I am able to make my code work 😀
It's resolved?
It's resolved?
No. The automatic dev certificate that the 'dotnet dev-certs' tool generates still doesn't work on Linux since it is missing the 'Certificate Signing' attribute. A PR is still required to fix that (I have plans to put up a PR soon).
However, the manual instructions in this discussion thread above show steps to create a self-signed certificate that does work on Linux.
I managed to make a self-signed certificate using the method above, and set the environment variable to point to the certificate. But when I run the application, the old default certificate is used instead of the new one by the web-application. Can you show me the changes to be made in the code of the application so that it uses the newly made self-signed certificate is used instead of the old one?(I am asking about the method to point out to krestrel the new self-signed certificate)
and set the environment variable to point to the certificate. But when I run the application, the old default certificate is used instead of the new one by the web-application.
Not sure why the environment variables aren't working. That is an option that Kestrel usually looks at to specify the PFX file for the server (instead of using the default ASP.NET "dev" certificate). Can you show the exact environment variables you're using?
I've used something like this in my code to use a specific PFX file for the Kestrel server:
public class Program
{
public static void Main(string[] args)
{
CreateHostBuilder(args).Build().Run();
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseKestrel(options =>
{
options.Listen(IPAddress.Loopback, 5000);
options.Listen(IPAddress.Loopback, 5001, listenOptions =>
{
listenOptions.UseHttps("certificate.pfx", "password");
});
});
webBuilder.UseStartup<Startup>();
});
}It appears that the program is unable to find the pfx file even if I specified it in the envrironment variables. The error looks like
Using launch settings from '/home/lord-amateur/RazorPagesMovie/Properties/launchSettings.json' [Profile 'RazorPagesMovie']... Unhandled exception. Interop+Crypto+OpenSslCryptographicException: error:2006D080:BIO routines:BIO_new_file:no such file at Interop.Crypto.CheckValidOpenSslHandle(SafeHandle handle) at Internal.Cryptography.Pal.OpenSslX509CertificateReader.FromFile(String fileName, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags) at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(String fileName, String password) at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions, String fileName, String password) at RazorPagesMovie.Program.<>c.<CreateHostBuilder>b__1_2(ListenOptions listenOptions) in /home/lord-amateur/RazorPagesMovie/Program.cs:line 29
log.txt
(the whole error log is attached)
I used the same Export commands to add to the environment variables.
checking the environment variables:
printenv | grep Kestrel
gives:
ASPNETCORE_Kestrel__Certificates__Default__Password=mypassword
ASPNETCORE_Kestrel__Certificates__Default__Path=localhost.pfx
The localhost.pfx file is located at my home folder
@Lord-Amateur I had a similar error due the file permissions.
ASPNETCORE_Kestrel__Certificates__Default__Path=localhost.pfx
The localhost.pfx file is located at my home folder
What is your 'home' folder? And how does it relate to where the project files are?
Since you aren't specifying an absolute path for the PFX file, it is assumed to be in the same folder as the binaries for the application. So, you should make sure you copy it there during the build process. Or perhaps use an absolute path for the PFX file when you set the environment variable.
I changed the permissions of the file localhost.pfx and also relocated it to the project folder and then curl request test worked. Thanks for the advice.
Just FYI the underlying OpenSSL issue has been resolved and will ship in 1.1.1h (stable) and 3.0 later this year.
Just FYI the underlying OpenSSL issue has been resolved and will ship in 1.1.1h (stable) and 3.0 later this year.
Should updating to OpenSSL 1.1.1h automatically fix this issue, or will we need to wait for some underlying change in the .NET Core framework/tooling before it's ready?
Currently running the OpenSSL 1.1.1h build (as the linked issue mentions that the fix is already included in it), but I'm still having issues with the SSL connection.
Should updating to OpenSSL 1.1.1h automatically fix this issue
As far as I'm aware.
Currently running the OpenSSL 1.1.1h build
If you've built it locally (since it hasn't released yet) double check that your version is actually getting loaded. Local builds tend to end up in /usr/local/lib/ which has (IIRC) lower binding than the system library path. So you'd need to use LD_PRELOAD to pre-load both libssl.so.11 and libcrypto.so.11 from your OpenSSL 1.1.1h build (or install the libraries to the normal system path).
No dice, sadly.
I've installed the OpenSSL 1.1.1-stable version from the OpenSSL GitHub repo, following their instructions:
# From the repo's root directory.
./config
make
make test
sudo make install
Running the /usr/bin/openssl version -a command yields the following output:
OpenSSL 1.1.1f 31 Mar 2020 (Library: OpenSSL 1.1.1h-dev xx XXX xxxx)
built on: Fri Aug 28 14:27:37 2020 UTC
platform: linux-x86_64
options: bn(64,64) rc4(8x,int) des(int) blowfish(ptr)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG
OPENSSLDIR: "/usr/local/ssl"
ENGINESDIR: "/usr/local/ssl/lib/engines-1.1"
Seeding source: os-specific
Which indicates that the file is still the old one.
Running the /usr/local/bin/openssl version -a command yields the following output:
OpenSSL 1.1.1h-dev xx XXX xxxx
built on: Fri Aug 28 14:27:37 2020 UTC
platform: linux-x86_64
options: bn(64,64) rc4(8x,int) des(int) idea(int) blowfish(ptr)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG
OPENSSLDIR: "/usr/local/ssl"
ENGINESDIR: "/usr/local/ssl/lib/engines-1.1"
Seeding source: os-specific
So that's where it got installed.
I move the current /usr/bin/openssl file to my home to keep it as a backup, then create a symlink to the newly created /usr/local/bin/openssl file to replace it. Now, running openssl version -a results in the 1.1.1h output.
I reboot, just in case.
If I try Identity Server 4's first quickstart example, it fails to establish an SSL connection (but it works fine on Windows).
I set LD_PRELOAD to /usr/local/lib/libcrypto.so.1.1:/usr/local/lib/libssl.so.1.1, and reboot (just in case).
I still get a failure to establish an SSL connection.
At this point I'm thinking of just trying to see if I can't get a Unix config and a Windows config and just use a hard-coded certificate with a script to generate them on Unix machines as part of our dev build process.
I don't know. Maybe I'm missing something here? Some people mentioned running sudo ldconfig -v, but that doesn't seem to link libraries that are in my /usr/local/ directory.
can you check /proc/<PID>/maps just to be sure? The .Net really does not care about the openssl executable. You can leave it as is. I would set LD_LIBRARY_PATH=/usr/local/lib to avoid interactions between preload and dlopen.
Also the loader would honor CLR_OPENSSL_VERSION_OVERRIDE environment. To be sure you can readme the libraries (or create symlink to something like libcrypto.so.1.1.my and do CLR_OPENSSL_VERSION_OVERRIDE=1.1.my
can you check
/proc/<PID>/mapsjust to be sure?
I'm guessing you mean the dotnet results from ps -A? If I look in them using less, I can see that they seem to interact with /usr/local/lib/libcrypto.so.1.1 and /usr/local/lib/libssl.so.1.1, which should be the 1.1.1h install.
LD_LIBRARY_PATH=/usr/local/lib
I've tried exporting this environment variable (both with and without LD_PRELOAD being exported), and it doesn't seem to change anything. Same thing with creating a symlink to libcrypto.so.11.my and setting CLR_OPENSSL_VERSION_OVERRIDE=1.1.my.
The question is if you see any other copy of libel or libcrypto. If not, than the .NET is running against the version you built.
If you build is right that would suggest that the 1.1.1h did not resolve your issue @micka190
Yeah, using grep to find all instances of libcrypto and libssl just gives me a list of calls to /usr/local/lib/libcrypto.so.1.1 and /usr/local/lib/libssl.so.1.1, which are the 1.1.1h ones.
Looks like I may have an issue unrelated to OpenSSL 1.1.1h... 😞
What is your issue @micka190? Can you post certificate and repro here (or create separate repo)
Do you have control over the cert creation? There is trivial workaround AFAIK by setting proper flags.
Also note the : OPENSSLDIR: "/usr/local/ssl"
Your version will probably look for trusted CAs in different directory and update-ca will not work if you want to add trust for your self-signet cert.
@wfurt, Essentially: we're trying to get a Blazor app going, but we need authentication for our API. So we're going through the Identity Server 4 documentation to get us started. Or at least we're trying to. We can't get past the first sample, because of issues with SSL. Running it on Windows works fine, but most of us are using Ubuntu 20.04 as our daily driver (the actual distro, not WSL).
In terms of code, that sample link has it all. We gave up and downloaded their sample code from their GitHub repo to make sure we weren't missing something. Works great on Windows, SSL connection issues on Ubuntu machines...
All we get is "The SSL connection could not be established, see inner exception.." But we're unable to view the inner exception. My understanding is that IS4 catches it in some deeply nested internal method and just adds the message to whatever throws this exception.
Since it only occurs on Ubuntu, and not on Windows, I assumed that it was probably related to this issue, since it's occurring in development, when using dotnet dev-certs https's certificates (which from my understanding are self-signed).
And this you make the cert trusted? Does it work when you run curl -v https://XXX/
Also you can always write simple console app with HttpClient. If you use 5.0 for that, it should have somewhat better error reporting on handshake failures.
Can you also check the certificate has the KeyUssage mentioned above?
You can always use your own certificate instead of the default.
I stepped out of the office for today. I did try it with curl, and I was getting security errors related to SSL (don't remember the exact wording), but I seem to recall that it was about self-signed certificates. I'll verify the exact output on Monday. I'll also take a look at using the 5.0 preview SDK with HttpClient to see if I can't narrow down the cause a bit more and check the KeyUsage.
You can always use your own certificate instead of the default.
I did try that using some of the above examples, but I think I got some stuff mixed up in all the examples because even specifically providing a self-generated certificate (be it through ASP .NET Core environment variables, appsettings.json configs, or through the actual code) was resulting in the same SSL error as I'm getting now (which kind of made me think it had to do with the OpenSSL version initially).
@wfurt Alright, so I'm back in my office.
Running curl -v https://localhost:5001 returns this:
* Trying 127.0.0.1:5001...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5001 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
Which is probably related to the OPENSSLDIR: "/usr/local/ssl" potential issue you pointed out earlier. I'll see if I can get the dotnet dev-certs https certificate inside that directory.
This doesn't work for me as well hmmm, just started using dotnet on mac though
I have managed to create a self-signed certificate that I can trust and use in Ubuntu. In addition the certificate can be imported into the .NET SDK and will then be used by ASP.NET Core as a development certificate without changing the config. You can find the script and more informations about this on my blog https://blog.wille-zone.de/post/aspnetcore-devcert-for-ubuntu
@BorisWilhelms your scripted worked out of the box for me
@BorisWilhelms worked like charm for me as well, thanks a ton!
@BorisWilhelms just out of curiosity, do you run the script in the project dir? I am running my project in a remote env and relying on port forwarding. Just can't seem to get it to work. I'm on ubuntu 20.04, dotnet 5. Thank you.
@nickwang0808 no. The script is not made for remote environments. The script is for local development machines where you run the browser and the server on the same machine. Could you please explain your use case more?
@BorisWilhelms I am running on vscode with the remote development extension pack, the host is ubuntu 20.04 and the client is running on windows 10. the way microsoft doc explains is that vscode setup automatic port-forwarding and I can just go to localhost:5001 to access the webpage built by .net.
it does seems like the certificate is there but chrome is not taking it?
One thing I noticed is that if I remove the certificate or just run the dotnet dec-certs https --clean , dotnet run will just throw error saying that certificate is missing. so after running the script something did work, just still not getting https connections.
Again really appreciate you time helping.
@nickwang0808 Are you using Devcontainer in VS Code? If so please check your devcontainer.json. The process to reuse your trusted development certificate is described there.
@BorisWilhelms not I am not, just a normal vm.
For folks interested in this. You can get this to work on Ubuntu following these instructions
#27344 (comment)
Similar steps can likely be taken for other distros.
@BorisWilhelms Thanks for your comment on stack overflow! This led me into the right direction regarding the 1.3.6.1.4.1.311.84.1.1 property needed on the cert.
As I faced the issue that the cert was either accepted in Chrome or in Firefox but not in both dependent on CA:TRUE/FALSE.
I got an 'NET::ERR_CERT_INVALID' in Chrome or an 'MOZILLA_PKIX_ERROR_CA_CERT_USED_AS_END_ENTITY' in Firefox.
I created a script, which sets up a Trust Chain in a way that both browsers are happy. Root CA and signed server certificate.
Repo: daretoIT/generate-dotnet-dev-certificate
I'm having issues with SSL invalid connection between service requests in WSL as well. In my case I get the following output:
The above error occurs during access token validation when in my API tries to access the Identity service.
To highlight the part I haven't found any related error on the web: The remote certificate is invalid because of errors in the certificate chain: PartialChain
I'm using the following powershell script to generate the certificate:
$cert = New-SelfSignedCertificate -DnsName "localhost", "auth" -KeyUsage DigitalSignature, CertSign, KeyEncipherment -CertStoreLocation "cert:\LocalMachine\My" -TestRoot
$certKeyPath = ".\certificates\.aspnet\https\aspnetapp.pfx"
$password = ConvertTo-SecureString 'the-password' -AsPlainText -Force
$cert | Export-PfxCertificate -FilePath $certKeyPath -Password $password
$rootCert = $(Import-PfxCertificate -FilePath $certKeyPath -CertStoreLocation 'cert:\LocalMachine\Root' -Password $password)
which I then use to generate a .crt file using openssl pkcs12 -in aspnetapp.pfx -clcerts -nokeys -out aspnetapp.crt and then trust in WSL with sudo update-ca-certificates .
When starting my docker containers I'm able to curl https://localhost:5000 (the Identity Service container that my service will call) and I get the expected response as for the https://localhost:5000/.well-known/openid-configuration endpoint. Since when using the dotnet dev-certs certificate the curl command curl https://localhost:5000 failed (probably due to what
@davidsh mentioned about the needed "Certificate Signing" on the Key Usage attribute) I'm assuming the certificate is now valid, and I haven't figured out what's causing the request to fail on the service.
I'm fairly new to certificates so I'm very likely missing something and I appreciate if you could point me towards any resource that could serve as guideline for the setup I'm trying to do.
Мне удалось создать самозаверяющий сертификат, которому я могу доверять и использовать в Ubuntu. Кроме того, сертификат можно импортировать в .NET SDK и затем использовать ASP.NET Core в качестве сертификата разработки без изменения конфигурации. Вы можете найти сценарий и дополнительную информацию об этом в моем блоге https://blog.wille-zone.de/post/aspnetcore-devcert-for-ubuntu
Good day! I am trying to run your script. I got the message: ./scripts/ubuntu-create-dotnet-devcert.sh: 2:.: Can't open ./common.sh. Please tell me how to solve the problem.
I am using VSL 2 debug application in Linux as described here https://docs.microsoft.com/ru-ru/visualstudio/debugger/debug-dotnet-core-in-wsl-2?view=vs-2019
This worked for me nicely:
I have managed to create a self-signed certificate that I can trust and use in Ubuntu. In addition the certificate can be imported into the .NET SDK and will then be used by ASP.NET Core as a development certificate without changing the config. You can find the script and more informations about this on my blog https://blog.wille-zone.de/post/aspnetcore-devcert-for-ubuntu
I'm on Ubuntu 20.04.
I was having trouble getting Identity authentication to work on our project, that said the docs really need to be updated because I spent a ridiculous amount of time not understanding what I was doing wrong, in fact I still can't get Firefox on Ubuntu 20.04 to fully trust the certs (the top-left lock still shows an exclamation mark but the authentication now works, Chrome seems to be fine).
Here are the links to the docs:
There should at least be a note on there that says there is known issue with dotnet dev-certs on Linux and redirect to a workaround, it would save a lot of time for a lot of newbies like me.
There is "Edit" button on the doc pages. You can edit and submit PR with proposed changes @PhilParisot. Tech writers will then help you to polish the wording end details.
Would they allow me to put on a note on there saying this is broken at the moment on Ubuntu 20.04? I haven't tested all distros, and I literally used a script someone else wrote to fix my issue, I don't know what the guidelines are but I'm pretty sure I'd get my PR rejected...
I don't know. distro specific details are easy to get obsolete IMHO.
Adding yet another small repro. The client code is out of our control, it's a callback into our service. Usual caveats about doing this in production -- it's not a production use-case, it's purely a developer use case, but it's blocking our linux-based developers from using HTTPS.
Server:
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
namespace service
{
public class Program
{
public static void Main(string[] args)
{
WebHost.CreateDefaultBuilder(args)
.Configure(applicationBuilder =>
{
applicationBuilder.UseRouting();
applicationBuilder.UseEndpoints(endpoints =>
endpoints.MapGet("/test", async context => await context.Response.WriteAsync("OK")));
})
.Build()
.Run();
}
}
}Client
using System;
using System.Net.Http;
using System.Threading.Tasks;
namespace client
{
class Program
{
static async Task Main(string[] args)
{
var httpClient = new HttpClient();
var response = await httpClient.GetAsync("https://localhost:5001/test");
Console.WriteLine(response.StatusCode);
Console.WriteLine(response.Content);
}
}
}Output from curl which works fine
% curl https://localhost:5001/test --verbose
* Trying 127.0.0.1:5001...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5001 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: O=Developer CA; OU=Developers; CN=Developer self-signed Server Certificate
* start date: Nov 3 02:49:29 2021 GMT
* expire date: Nov 3 02:49:29 2023 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: O=Developer CA; OU=Developers; CN=Developer self-signed Root CA
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55e197e74850)
> GET /test HTTP/2
> Host: localhost:5001
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< date: Wed, 03 Nov 2021 16:05:06 GMT
< server: Kestrel
<
* Connection #0 to host localhost left intact
OK%And client output.
% dotnet run
Unhandled exception. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: PartialChain
at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
at client.Program.Main(String[] args) in /home/michaeljon/src/repro/client/Program.cs:line 12
at client.Program.<Main>(String[] args)Where did you put your O=Developer CA; OU=Developers; CN=Developer self-signed Root CA @michaeljon?
You can iterate through the appropriate X509Store so see if you see it there.
It's impossible to work with .net Core on Linux!
It's impossible to work with .net Core on Linux!
Why not? The claim does not have enough details. While there is no automatic creation and trust for dev certificate, you can use user distro tools to create one - just like you would for production.
I have to concur with @shpsyte , Linux does feel like a second class citizen for .NET and it's frustrating, I still haven't been able to make my certs work on Firefox for Ubuntu
@PhilParisot curious if this works for you? https://dev.to/lmillucci/firefox-installing-self-signed-certificate-on-ubuntu-4f11
It's impossible to work with .net Core on Linux!
No, it is not. I am doing it daily.
But unfortunately, it is not that easy to support Linux, simply because there is no Linux. There are distributions and different distros have different cert stores. Therefore, applications tend to bring their own cert store. If you install the application as a snap package, the application cert store is in a different place, etc....
But if you know your distro and applications, it is possible to setup the certs for your workflow in a few minutes.
Written from Microsoft Edge, running on Arch Linux, with .NET 3.1, NET 5 and .NET 6, Rider, Postman, Azure Data Studio.
@PhilParisot curious if this works for you? https://dev.to/lmillucci/firefox-installing-self-signed-certificate-on-ubuntu-4f11
@DamianEdwards THANK YOU, yes it finally works! You have no idea how long I've searched for a solution!
I'd REALLY appreciate if someone could update the docs here: https://docs.microsoft.com/en-us/aspnet/core/security/enforcing-ssl?view=aspnetcore-6.0&tabs=visual-studio#trust-the-certificate-with-firefox-on-linux with the solution.
Creating the /usr/lib/firefox directory is for some reason denied, I'm guessing that changed when Ubuntu started using the Firefox Snap package instead of the DEB.
It's impossible to work with .net Core on Linux!
Why not? The claim does not have enough details. While there is no automatic creation and trust for dev certificate, you can use user distro tools to create one - just like you would for production.
Very well said, just skip using donet dev-certs. Have a dev cert and avoid headaches. Would be helpful if the documentations that refer to building multi container or docker orchestration applications, kept things simple.
I have managed to create a self-signed certificate that I can trust and use in Ubuntu. In addition the certificate can be imported into the .NET SDK and will then be used by ASP.NET Core as a development certificate without changing the config. You can find the script and more informations about this on my blog https://blog.wille-zone.de/post/aspnetcore-devcert-for-ubuntu
Worked for me on Arch with .NET 6 and ASP.NET Core, ty!
O=Developer CA; OU=Developers; CN=Developer self-signed Root CA
Could you be more specific? I'm facing the same issue.