Dockerfile should trigger the building of the local package cache.
MichaelSimons opened this issue · comments
With the introduction of the offline feature, the dotnet CLI builds a local package cache on first use. The dotnet Dockerfile should trigger this to happen. This takes some time therefore it will be a savings for all users of the image. Additionally having the package cache in the base image will save disc space for scenarios in which base image is used multiple times on the same machine as the base layer will be shared.
Consider whether this will seriously bloat the base image - it may be worth it though.
@friism, yes I acknowledge your concern. This appears to be a trade-off situation.
- This only applies to the SDK image. It does not affect the core (e.g. runtime) images.
- The local package cache is built on first use. This can take some time which of course varies greatly based on your machine configuration. To give you a ballpark, on my machine it is taking ~ 18 seconds.
- The image size impact is significant. For the preview image it went from 423.5 to 859.7 MB (uncompressed). This will affect image download times but it is a wash at minimum on disk usage for the local machine because this is going to get built regardless once the image is used. If the image is used multiple times it is going to be a significant savings on disk space unless users author their Dockerfiles in such a way that the package cache is generated as a layer that can be shared.
Without this users are definitely going to be affected. They can workaround this themselves but they need to be well educated and will have to explicitly handle this in their Dockerfiles. I am of the opinion we should handle this for them at the expense impacting the image download time.
@friism, @glennc, @dleeapho, and others - please provide your thoughts on this.
cc @SteveLasker
We are focusing on two different scenarios. The developer inner loop and build scenario, where code compiling and image building is the priority. And the runtime scenario, where startup time (docker run --> processing requests), density and image size are the priority. That said, we don't want the lower priority items in each scenario to become distractors. For production, as long as startup time is fast, we can get maximum density and image sizes are small for network traffic, it can take a while to actually build the production image, as it happens in a CI system. However, it can't take 30 minutes to optimize. That's just an extreme.
For development, where compilation time is a priority, including startup time, we might pre-load content on the image. However, what's the extreme size limitation? >600mb seems pushing it. 860 definitely seems overly large. We do have some optimizations coming in the dotnet restore api that currently doesn't know whats already on the machine, so we download nugets anyway, and the XML files, and other platforms. This work won't make the June release, so we will have some short term bloat.
I don't know if we've got a prototype of the nuget api leveraging the shared framework to know what the final size will look like for something like a standard website. I'd like to understand that before we bloat our image to the point of a distraction. I'd like people to focus on the startup time and density, and if the image size is a bit larger than Node or Go, Considering these are both ~256mb max, it seems we have some crunching to consider.
@glennc informed me of the NUGET_XMLDOC_MODE environment variable. When set to skip, NuGet restore will not extract the all of the XML files for each package. This yields a nice savings.
With .NET CLI 1.0.0-Preview2-003096 I see the following images sizes on disk (not compressed)
No Cache - 416.7 MB (this is the current SDK image)
Full Cache - 852.9 MB (+436.2)
Cache w/o XML - 548.7 MB (+132)
Given that you will consume this disk space in your container layer in order to build any .NET application anyway I think this is reasonable. Given that the purpose of this image is primarily to build applications inside of it. We don't have this on the image that is intended for running compiled applications.
Agree with @glennc. We should just use NUGET_XMLDOC_MODE for the SDK image. 548 MB is reasonable and the developer would have arrived at 548+ in build usage anyway.
How we trigger the cache building will be affected by https://github.com/dotnet/cli/issues/3533
This looks good. We got to the target size <600mb, and this is still a stop gap until we fix the NuGet restore. So, between the NuGet restore, and the other perf improvements, this looks like a safe place to be for RTM and an opportunity to improve in with newer tool builds.
@MichaelSimons You say "when set to skip." Do you mean literally "SET NUGET_XMLDOC_MODE=skip"
@shanselman - yes but when used within a Dockerfile you should use ENV