How to setup the dev certificate when using Docker in development

[Rick-Anderson]

related #3310
Javier is contact:
This needs to go in Enforce HTTPS in an ASP.NET Core
The first time you run dotnet after installing the SDK you get this message
Successfully installed the ASP.NET Core HTTPS Development Certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). For establishing trust on other platforms please refer to the platform specific documentation.
For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.

Copied from #3310
We also need to cover how to setup the dev certificate when using Docker in development:

• Create an application on Visual Studio using the MVC template.
• Run the app to ensure its working.
• Add docker support for the application through the tooling.
• Modify the dockerfile to expose the port 443 with
  EXPOSE 443
• Modify the docker-compose override file to map ports, volumes and environement variables as follows (this will all be unnecessary after docker tooling has support for HTTPS):

    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://localhost;http://localhost
      - ASPNETCORE_HTTPS_PORT=44349
    ports:
    # Replace the values on the left by the values on your launchSettings.json
      - "51217:80"
      - "44349:443"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets/:/root/.microsoft/usersecrets
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https/

• Export the HTTPS certificate into a PFX file using the dev-certs global tool to %APPDATA%/ASP.NET/Https/<>.pfx using a password of your choice (recommended password new-guid on powershell)
• On your project, open user secrets and add the following configuration keys:

{
    "Kestrel":{
        "Certificates":{
            "Default":{
                "Path":     "/root/.aspnet/https/<AppName>>.pfx",
                "Password": "<<Your-Password>>"
            }
        }
    }
}

• Run your application within the container.
• Navigate to the HTTP endpoint on your application
  • You should not see any warning about the HTTPS certificate being invalid.
  • You should be redirected to the HTTPS endpoint automatically.

[Rick-Anderson]

Subject: RE: Docker for ASP.NET Core 2.1 Preview 2

. Provided that your docker file and docker compose look like the ones that VS generates when you add VS support, there are a couple of steps you need to take to enable it manually.
In essence, you need to get the HTTPS development certificate from your machine into the container image as a pfx file in a special location on disk that we recognize and provide the password through user.

• Modify the dockerfile to expose the port 443 with
• EXPOSE 443
• Modify the docker-compose override file to map ports, volumes and environment variables as follows:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=https://+;http://+
- ASPNETCORE_HTTPS_PORT=44349
ports:
# Replace the values on the left by the values on your launchSettings.json
- "51217:80"
- "44349:443"
volumes:
- ${HOME}/.microsoft/usersecrets/:/root/.microsoft/usersecrets
- ${HOME}/.aspnet/https:/root/.aspnet/https/
• Export the HTTPS certificate into a PFX file using the dev-certs global tool to ${HOME}/.aspnet/https/<>.pfx using a password of your choice (dotnet dev-certs https -ep ${HOME}/.aspnet/https/ -p <>)
• Add the password to the user secrets in your project.
• Add the password to the user secrets in your project (dotnet user-secrets set "Kestrel:Certificates:Development:Password" "<>"

per Javier:
For setting up a non development certificate on kestrel the instructions are the same as for the development case (obviously you need to provide the certificate) and you need to set the path to the certificate in a configuration key.
"Kestrel:Certificates:Default:Path"

[csharpfritz]

Is there an update for this now they we are in RTM for 2.1 and preparing for 2.1.1 release?

[robinmanuelthiel]

Can we please avoid things like "Create an application on Visual Studio using the MVC template." in the documentation for a cross-platform framework like .NET Core, that also runs on Linux and Mac devices?

[RaccoonDev]

Hi.

The method works fine if I run the application in development environment. Is there are way to specify what certificate and port should kestrel use in production mode?

[dradovic]

When using a pfx file generated by dotnet dev-certs on my local Windows machine, I ran into a

error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure

So I switched to simply calling dotnet dev-certs https --export-path ./localhost.pfx -p $LOCALHOST_CERTIFICATE_PWD as part of building the Docker image and then referencing that in Startup.cs using KestrelServerOptions:

listenOptions.UseHttps("../../localhost.pfx", Environment.GetEnvironmentVariable("LOCALHOST_CERTIFICATE_PWD"));

[kdcllc]

I had an issue with Docker-Compose.yml and Visual Studio.Net 15.7.5.
The resolution was adding the entry for secret,json:

{
    "Kestrel":{
        "Certificates":{
            "Default":{
                "Path":     "/root/.aspnet/https/<AppName>>.pfx",
                "Password": "<<Your-Password>>"
            }
        }
    }
}

[se-augustus]

Do these same rules apply to MacOS users? How about those of us using Kitematic? I'm wondering if I can extrapolate the notes above to fit into the areas on/in the attached screenshots.

[chrisckc]

In essence, you need to get the HTTPS development certificate from your machine into the container image as a pfx file in a special location on disk that we recognize and provide the password through user.

I have tried this in Docker and unless I specify the certificate location and password in the Startup configuration it crashes with:
System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.

I followed the guide provided by @Rick-Anderson which seems to imply that UseStartup() now checks for the existence of the cert at root/.aspnet/https/<>.pfx or checks the UserSecrets for Kestrel:Certificates:Development:Path and Kestrel:Certificates:Development:Password

The UserSecrets is working and configured correctly as i am using it to get the path and password, it works using this code:

                .UseKestrel(options =>
                {
                    var configuration = (IConfiguration)options.ApplicationServices.GetService(typeof(IConfiguration));
                    var httpsPort = configuration.GetValue("ASPNETCORE_HTTPS_PORT", 443);
                    var certPassword = configuration.GetValue<string>("Kestrel:Certificates:Development:Password");
                    var certPath = configuration.GetValue<string>("Kestrel:Certificates:Development:Path");

                    options.Listen(IPAddress.Any, httpsPort, listenOptions =>
                    {
                        listenOptions.UseHttps(certPath, certPassword);
                    });
                });

[chrisckc]

I have found that if the path to the cert is specified in user secrets or an env var, it breaks.

I was able to get it to work using the default startup by removing the path which i added when trying a bunch of other things:
dotnet user-secrets remove "Kestrel:Certificates:Development:Path"

I can break it again by re-adding the path:
dotnet user-secrets set "Kestrel:Certificates:Development:Path" "/root/.aspnet/https/<<AppName>>.pfx"

The path is correct and the cert file exists as demonstrated in the code above, but for some reason it doesn't like having the path specified even if it is correct. The guide above does only say to add the password, but there should be no reason that adding the correct path, or even any valid path where the cert exists could not be made to work.

This only applies to Development environment, in Production environment it works as per the docs using the 'Default' path:
dotnet user-secrets set "Kestrel:Certificates:Default:Path" "/root/.aspnet/https/<<AppName>>.pfx"

[chrisckc]

I have published a repo which uses this technique which can be used to reproduce this issue:
https://github.com/chrisckc/DotNetCoreAureliaSPA

[Rick-Anderson]

@danroth27 what's the priority on this? We might need some help - at least for the first draft.
@chrisckc could you provide a draft?

[chrisckc]

@Rick-Anderson I should be revisiting that repo soon to build something out so will have a dig around to see what is causing the issue i posted about.

[chrisckc]

Ok, i traced back the error message:

System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.

which occurs when "Kestrel:Certificates:Development:Path" is specified in the configuration from UserSecrets or env vars despite the path being correct and the file existing.

I cloned the KestrelHttpServer repo and traced the message back to the Load() method inside the KestrelConfigurationLoader class (line 216).

The Load() method calls LoadDefaultCert(ConfigurationReader)
LoadDefaultCert checks if the "Kestrel:Certificates:Default" config entry exists, if so then the cert is loaded using the LoadCertificate(CertificateConfig certInfo, string endpointName) method which just builds the path using Path.Combine(env.ContentRootPath, certInfo.Path) , which if certInfo.Path is an absolute path just returns certInfo.Path due to the way Path.Combine works.

However if the "Kestrel:Certificates:Default" config entry does not exist is call FindDeveloperCertificateFile(configReader, logger) which is where things get a little odd.

The following particularly ugly piece of code is used to locate the development certificate:

               if (configReader.Certificates.TryGetValue("Development", out var certificateConfig) &&
                    certificateConfig.Path == null &&
                    certificateConfig.Password != null &&
                    TryGetCertificatePath(out certificatePath) &&
                    File.Exists(certificatePath))
                {
                    var certificate = new X509Certificate2(certificatePath, certificateConfig.Password);
                    return IsDevelopmentCertificate(certificate) ? certificate : null;
                }
                else if (!File.Exists(certificatePath))
                {
                    logger.FailedToLocateDevelopmentCertificateFile(certificatePath);
                }

The TryGetCertificatePath(out certificatePath) method looks for a cert named $"{appName}.pfx" in any of the following paths to cover both Mac OSX and Windows:
$APPDATA/ASP.NET/https, $APPDATA/.aspnet/https, $HOME/ASP.NET/https, $HOME/.aspnet/https

That piece of code is completely broken if it's intention is to either use the certificateConfig.Path or go and find the certificate file from a default location and naming convention, which would be the sensible thing to do.

The result of that 'if' statement is that if a certificate path is supplied in the config it is never used.

It also will report a misleading error because a missing password would result in a log error which says that the certificate could not be located and also an exception inside the Load() method stating that that certificate could not be found.

Experiencing misleading errors has always been an issue when working with Microsoft software in the past, was hoping that the situation has been improving.

I also noticed that the IsDevelopmentCertificate method checks is the certificate.Subject is equal to "CN=localhost", which may not always be the case for every developers development environment. For example if you are developing using docker running on another machine or a VM etc.

The code can be simplified, i don't really see the need to use the 2 separate configuration structures: "Kestrel:Certificates:Default" and "Kestrel:Certificates:Development" and check the subject of the cert. Just one config with a check to try and find the cert from the default location and name if it is not supplied in ":Path"?

[chrisckc]

Looking at the referenced issue on line 332 of KestrelConfigurationLoader.cs

aspnet/Hosting#1294

It looks like the broken code was added as part of that, which seemed to start off with good intentions. Maybe the behaviour is intentional, if so i would like to know the reasoning.

@Rick-Anderson Any documentation written to cover this setup with its current behaviour would look rather odd.

[danroth27]

@javiercn @Tratcher

[javiercn]

That piece of code is completely broken if it's intention is to either use the certificateConfig.Path or go and find the certificate file from a default location and naming convention,

It's not meant to be used with a path, the development certificate key is only there to support tooling scenarios. If you want to specify a path and a password, simply use the Default key.

I also noticed that the IsDevelopmentCertificate method checks is the certificate.Subject is equal to "CN=localhost", which may not always be the case for every developers development environment.

It's only meant to support localhost scenarios (including docker localhost). Other scenarios will have security implications we would have to worry about.

It also will report a misleading error because a missing password would result in a log error which says that the certificate could not be located and also an exception inside the Load() method stating that that certificate could not be found.

There's no way AFAIK to check the validity of the password on the PFX.

[PrefabPanda]

I've been trying to follow this example and I'm tearing my hair out!

I get these exceptions:
AuthenticationException: The remote certificate is invalid according to the validation procedure.
HttpRequestException: The SSL connection could not be established, see inner exception.

I have two containers, one talks to the other by HTTPS and by use of an alias configured in the compose file. The container being talked to doesn't have a public IP on purpose (Gateway Microservice architecture).

I have generated a certificate and am applying it to the service being called.

Any ideas?

[PrefabPanda]

Turns out I needed this: https://blog.roushtech.net/2016/12/20/ignoring-ssl-certificate-errors-net-core-httpclient/

and yes this is in my development environment.

[killnine]

Thanks, @jcoutch this worked for me. It's a little jankey seeing the full assembly name as the project name in my solution but w/e....

[Rick-Anderson]

@scottaddie are you able to do this in the next 6 weeks or should we assign to the PU?

[n10l]

Easiest workaround, works for me (when creating new project with https and docker support with linux container):

1. From visual studio, while creating new project select checkbox to add Docker Support. Visual studio will cry for Docker Desktop is not running. Ignore that.
2. Next, add Orchestration support in your project and a docker compose will be generated automatically.
3. Now run and access your app with docker ip.
   Hopefully, everything works if above steps done in sequence.

Note: I am on a Windows 10 Home VM running on Mac inside Parallels. I stopped and removed all the old docker containers before trying this, where I was facing the errors mentioned here.

[scottaddie]

@Rick-Anderson Please assign to PU

[javiercn]

There are docs for this here https://github.com/dotnet/dotnet-docker/blob/master/samples/aspnetapp/aspnetcore-docker-https.md

[javiercn]

@mkArtakMSFT I've already pointed out to the related docs, now sure what you expect from me here. I think this was already discussed on a thread and addressed, but I might be wrong. @danroth27 can confirm.

[mkArtakMSFT]

I see, hadn't read through all this. Should this be closed then? @Rick-Anderson ?

[danroth27]

The content in https://github.com/dotnet/dotnet-docker/blob/master/samples/aspnetapp/aspnetcore-docker-https.md should be part of our official docs. Someone needs to do the work of turning that content into official doc content.

[Rick-Anderson]

I've done that - see https://docs.microsoft.com/en-us/aspnet/core/security/docker-https?view=aspnetcore-2.2
Not sure why this didn't get closed.

[Rick-Anderson]

#13046

[danroth27]

Ah! Cool, sounds good then 😃.

[jorgeolive]

I know this is closed already.. but, does the suggested configuration work with 3.0? I've followed above steps and I am getting a weird docker compose error "Duplicate mount point: /root/.aspnet/https" using the following images:

FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-buster-slim AS base

FROM mcr.microsoft.com/dotnet/core/sdk:3.0-buster AS build

Any help would be greatly appreciated.

Thanks,
Jorge.

[javiercn]

Hi.

It looks like you are posting on a closed issue!

We're very likely to lose track of your bug/feedback/question unless you:

1. Open a new issue
2. Explain very clearly what you need help with
3. If you think you have found a bug, include detailed repro steps so that we can investigate the problem

[jnpwly]

@jorgeolive -- kia ora. Did you end up opening a new issue regarding this and whether it works on .NET Core 3.x? I am also having difficulty getting a multi-containerised topology up and running, so it is either "user error" on my part, or possibly something you have already fixed. So, I'm just checking :)

[KevinBurton]

I started with the suggestions Rick-Anderson commented on May 22, 2018 but I am running into 'access denied' when exporting the certificate, for not only the .aspnet path as suggested but what seems to be any path. Ideas on how to get over this hurdle?

[javiercn]

Hi.

It looks like you are posting on a closed issue!

We're very likely to lose track of your bug/feedback/question unless you:

1. Open a new issue
2. Explain very clearly what you need help with
3. If you think you have found a bug, include detailed repro steps so that we can investigate the problem

[bilalmalik777]

i updated my application from 2.2 to 3.1 and facing the following error.It was working perfectly in 2.2 with docker
'error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure'

it was working fine with 2.2 but now facing an error in development mode. i run the following command to generate the dev certificate

dotnet dev-certs https -ep %APPDATA%\ASP.NET\Https\TT.Core.Portal.Web.AzureHybrid.pfx -p password
dotnet dev-certs https --trust
dotnet user-secrets -p TT.Core.Portal.Web.AzureHybrid.csproj set "Kestrel:Certificates:Development:Password" "password"

program.cs

            Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseKestrel(options =>
                             {
                                 bool.TryParse(Environment.GetEnvironmentVariable("IsDockerDeployment"), out bool isDockerDeployment);
                                 if (isDockerDeployment)
                                 {
                                     options.Listen(new IPEndPoint(IPAddress.Any, 443), listenOptions =>
                                     {
                                         var configuration = (IConfiguration)options.ApplicationServices.GetService(typeof(IConfiguration));
                                         var certPassword = Environment.GetEnvironmentVariable("ASPNETCORE_Kestrel__Certificates__Development__Password");
                                         var certPath = Environment.GetEnvironmentVariable("ASPNETCORE_Kestrel__Certificates__Development__Path");
                                         Console.WriteLine(certPassword);
                                         Console.WriteLine(certPath);

                                         var certificate = new X509Certificate2(certPath, certPassword);
                                         Console.WriteLine("Certificate provided");
                                         var httpsConnectionAdapterOptions = new HttpsConnectionAdapterOptions()
                                         {
                                             ClientCertificateMode = ClientCertificateMode.NoCertificate,
                                             SslProtocols = System.Security.Authentication.SslProtocols.Tls12,
                                             ServerCertificate = certificate,
                                         };
                                         listenOptions.UseHttps(httpsConnectionAdapterOptions);
                                     });
                                 }
                             });
                    ////webBuilder.UseIIS();
                    webBuilder.UseStartup<Startup>();
                });```

i also declared both environment varibale **ASPNETCORE_Kestrel__Certificates__Development__Password** &&  **ASPNETCORE_Kestrel__Certificates__Development__Path** in the docker compose file.
Please help me to solve this issue

[badrdouah]

one question, can someone please answer,
should i use the certficate created by asp.net to enable https for my container on a production server, or the cert-manager ingress tls certificate is enough,
there is absolutely nothing about this one the internet, can someone please explain this

[ma1f]

Easiest is to simply use mkcert (https://github.com/FiloSottile/mkcert) to generate any self-signed certificates, using a root cert, then place something like cloudflare in front with flexible SSL, this way you maintain SSL all the way through.
Alternatively if your'd rather manage the actual certificates yourself simply set the correct environment variables as detailed in the docker image below and strip out the rootCA/mkcert commands.

Example base docker image

FROM mcr.microsoft.com/dotnet/aspnet:6.0

# update system
RUN apt-get update -y && apt-get upgrade -y

# dotnet specific env vars, default to development environment
ENV ASPNETCORE_ENVIRONMENT=Development
ENV ASPNETCORE_URLS=http://+:80;https://+:443

# dotnet kestrel env vars
ENV Kestrel:Certificates:Default:Path=/etc/ssl/private/cert.pfx
ENV Kestrel:Certificates:Default:Password=changeit
ENV Kestrel:Certificates:Default:AllowInvalid=true
ENV Kestrel:EndPointDefaults:Protocols=Http1AndHttp2

# copy certificate authority certs from local file system
ARG CA_KEY=./devops/Infrastructure.Hosting.HTTP.SSL/rootCA-key.pem
ARG CA_CERT=./devops/Infrastructure.Hosting.HTTP.SSL/rootCA.pem
ARG DOMAINS='localhost 127.0.0.1 ::1'

# default ca cert location (mkcert)
COPY ${CA_KEY} /root/.local/share/mkcert/rootCA-key.pem
COPY ${CA_CERT} /root/.local/share/mkcert/rootCA.pem

# install CA and SSL cert
RUN apt-get install curl -y && \
	curl -L https://github.com/FiloSottile/mkcert/releases/download/v1.4.3/mkcert-v1.4.3-linux-amd64 > /usr/local/bin/mkcert && \
	chmod +x /usr/local/bin/mkcert
RUN mkcert -install
RUN mkcert -p12-file /etc/ssl/private/cert.pfx -pkcs12 $DOMAINS

# Install locale
RUN apt-get install locales -y \
	&& localedef -f UTF-8 -i en_GB en_GB.UTF-8 \
	&& update-locale LANG=en_GB.utf8

ENV LANG=en_GB:en \
	LANGUAGE=en_GB:en \
	LC_ALL=en_GB.UTF-8

WORKDIR /app

EXPOSE 80
EXPOSE 443

Then each application uses this base docker image and overrides the domains if needed

RUN mkcert -p12-file /etc/ssl/private/cert.pfx -pkcs12 localhost ::1 127.0.0.1 mydomain-k8s-alias mydomain.com

As all the correct environment variables are setup, using the default webhost builder is all that is needed.

            Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder => {
                    webBuilder.UseStartup<Startup>();
                });