Bring back solution level packages
yishaigalatzer opened this issue · comments
This is an initial strawman suggestion for bringing back solution level packages (work in progress, I plan to update this a bit more, capturing some initial notes)
Why
NuGet 3 supports installing packages that previously where moved to solution level based on content into projects. So theoretically this doesn't prevent developers from using these packages.
However this solution doesn't roundtrip to Visual Studio 2013. And bring it back as it was in Visual Studio 2013 will break developers that adopted the new mechanism and installed these packages into a project.
Suggestion
- The name should probably be changed, because solution level packages is a misleading name that also collides with the menu - "Manage Packages for Solution".
- The feature should support explicitly installing packages at the solution level
- The feature should enable round tripping to Visual Studio 2013 (so the pattern is the same), keeping in mind that since it is going to allow new packages to be installed it might not be 100% roundtrippable, but projects starting from Visual Studio 2013 will be able to go back and forth, assuming you don't install a none supported package in 2015.
CC // @adamralph @davidfowl
As a side note, the NuGet.exe "update" command line currently requires a package.config, solution, or project file as an argument. Packages installed using:
nuget install <SomePackage> -o Package -pre -ExcludeVersioncannot be updated using nuget.exe.
@dmccaffery I don't understand how this comment (although correct, and by design) is related to this issue. Can you please explain?
I suppose, after thinking about it, its not truly related.
Its just my way of getting around the fact that solution-level packages don't exist for the moment. One could add a line to a build script that installs the package in a predefined location; but its impossible to then update that package without first deleting it and reinstalling it -- so its 50% of what a solution-level package provides.
KoreBuild / Sake is one such example.
You are right, but you can delete it an install again, which is almost exactly what update would do (except it will rollback on failure, and its a bit more expensive). So that shouldn't block you. And you can always use a packages.config file you author manually and call update on it
Agreed @yishaigalatzer, except that update does a version check to decide whether or not to delete / re-download. Since I am using this as part of a build script, it adds a bit of time to get going that would otherwise be unnecessary. Its not blocking, just... meh.
I guess all that I am asking for, is that any implementation of explicitly installed solution-level packages be supported by the CLI and not just the UI or PoSH console... since the CLI is what I rely on for xplat reasons. DNU is currently limited with respect to package management outside of the scope of a project.json, which makes it less adaptive than its NPM and Bower counterparts.
Thanks for all the hard work; I'm not complaining... just politely nudging. 👍
What's wrong with putting it in packages.config and calling NuGet.exe update on that file?
Nothing except staying out of the way of the developer that is using it
My use case is https://github.com/pulsebridge/condo
I could move their packages.config out of the way (if its there), drop my own, call update, and move theirs back; but that is a delicate process. I don't want to muck with theirs (if it exists), or create a dependency where something they change breaks the build, which is otherwise completely self-contained.
I have my work around, and it works. If CLI support for updating a package without a referencing packages.config file isn't in the cards for the future, I can always work around it. It was only a suggestion.
:)
It could be. But I don't see it happening soon without a community pull request.
Please file a separate issue for this
Another reason for bringing back this feature is semantics.
For many tools packages it is counter-intuitive to add them to a project. E.g. a build tool which has the entire solution as it's target of operation. Yes, you can work around it by installing the tool into "some" project, but this is awkward and counter-intuitive. If the semantics dictate that the tool is required for the entire solution and may even be complete unaware of projects within the solution, this should be embraced and expressed by the tooling.
Delighted to see this come back. I use this heavily in commercial and oss project to deliver tooling such as test runners, build tools (psake), etc. Associating these with a specific project really made no sense.
👍 Please bring back this feature!!
What would be the successor in your mind?
If the feature is being brought back, why does it need a successor?
Hi Adam, If the future brings back the function, then no successor is required 😉
I am with @damianh - We use this heavily to bring in all build tools. GitHub Releases, Nunit, Code Coverage, Nant etc.
We install Nuget.exe using configuration management globally on the system and then just call nuget restore on the solution. Then our next build step calls all tools from inside packages.
This is an incredibly important feature for us. We built our development environment around it, with solution packages for different project types and roles. Right now it's keeping us from updating to VS 2015, as we would have to redefine a lot of our scripts.
It's worth noting that this feature was actually a bit broken in VS 2013. Two major issues come to mind:
- Solution packages couldn't reference other solution packages, despite documentation indicating they could
- When attempting to update from the Nuget Package Manager it would just double install the solution package (two entries appearing the package.config), which then became difficult to uninstall. It worked fine from Package Manager Console, but the GUI was broken.
To prevent changing build tools and scripts every time a new version comes out, we need solution level nuget packages restored without the version folder.
I'd like to add my two-penneth
We distribute our build framework (which is PowerShell scripts, including psake) using nuget. We also package our application code using nuget. Our application code is not .net, it doesn't even get developed in Visual Studio, but we still use nuget because we like it as a mechanism of packaging and distributing a bunch of files as a single artifact.
To aid with installation and updating of those packages we have a .sln file that contains only one thing, a packages.config file. We can successfully install/update packages from Package Manager Console in VS however it seems we cannot do it from nuget.exe as the following screenshot hopefully demonstrates:
This is a problem as we want to write scripts to update all packages across all solutions in one fell swoop.
I have tried to workaround the problem by adding a notionally empty C# project into the solution but I cannot install my nuget packages into that project. I try from Package Manager Console but the project-level packages.config does not get updated.
In short, I'd like a mechanism for using solution-level nuget packages along with the capability of updating them using nuget.exe.
TIA
Please return the feature, we use it in similar to @jamiekt's scenario.
Fwiw, I just manually add packages.config beside the sln and edit it by
hand. Packages are restored via build script. The solutiom packages, and
versions thereof, dont change often so its not that painful.
On 18 Nov 2015 9:34 am, "Nikita Govorov" notifications@github.com wrote:
Please return the feature, we use it in similar to @jamiekt
https://github.com/jamiekt's scenario.—
Reply to this email directly or view it on GitHub
#1521 (comment).
@damianh top pro-tip.
People often just open the solution in Visual Studio and try to build it though, which then fails, causing lots of "why doesn't this build straight from source" questions when they don't know/think to do it via the command-line.
This issue is a blocker for me and unfortunately i cannot wait until the planned changes are complete. Over in #522 Solution-level packages are no longer working in Visual Studio 2015 RC @yishaigalatzer said
"The idea is that you put it in some common project"
Fair enough. Can anyone suggest what a "common project" might look like? Remember that I don't have any .net projects in my solution(s), I'm merely using nuget as a means of packaging up files. I guess what I'm looking for is the most vanilla project possible (if that makes any sense). Is it possible to create a project that contains nothing more than a packages.config file?
Any advice appreciated.
As an example, in one of my repos I have this file which an MSBuild targets file that defines some properties I use in all the .csproj files in the solution.
I then import this into each .csproj file with a single MSBuild <Import>, such as this one.
You could use a similar mechanism to hook-in some MSBuild script to restore the solution level packages for you in Visual Studio manually when the project(s) are compiled to do a NuGet package restore from the command line (like I do here).
Thanks @martincostello , its a nice idea and I could do that but my ultimate aim here is to enable updating of packages from the command-line, that means I don't even want to open Visual Studio. Also there is no notion of compiling my solution (or any projects therein) anyway.
In my world all that still works. Solution packages tend to be other things
such as xunit commandline runner, ilmerge etc that are not used directly by
projects.
I always have a 'how to build' in the readme.md in the repo too.
On 19 Nov 2015 1:18 pm, "Martin Costello" notifications@github.com wrote:
People often just open the solution in Visual Studio and try to build it
though, which then fails, causing lots of "why doesn't this build straight
from source" questions when they don't know/think to do it via the
command-line.—
Reply to this email directly or view it on GitHub
#1521 (comment).
@martincostello this is good idea, we can even imagine create a nuget package that add all the stuff to our projects to run restore of the solution level packages. However we will still be unable to use the UI to list or update our solution level packages.
As lot of others, I can says that we use this really often in my company, and we were glad to see it back.
omg, I don't understand NuGet projects at all o_o
and now I see new incredible things...
libraries of course, but how about additional tools for services of build processes, debugging etc. for all projects or for each...
it's important feature for providing all this above via common solution like nuget. Flexible and powerful distribution of any tools or libraries - isn't it so ?
However, ~"only libraries and only for each projects" is it new roadmap for nuget ? is it also will be for Command Line nuget.exe later ?
now I should think about new solutions for VS2015 and compatible things for old VS2013 -_-
please restore this trivial feature /let use this 'as is'
ok, now I use the non-binary GetNuTool for work via msbuild.exe and restoring of packages for solution-level inside IDE, for example:
#[($([System.Convert]::ToInt32("$(VisualStudioVersion.Replace('.', ''))")) >= 140)
{
#[var msbuild = "$(MSBuildToolsPath)\msbuild.exe"]
#[IO scall(#[var msbuild], "gnt.core /p:ngconfig=\".nuget/packages.config\"", 60)]
}]but I hope to see this feature again <_<
When this feature is implemented, will it still require the solution level packages.config to be under a .nuget folder or will it be able to operate on a packages.config located next to the solution file?
The reason I ask is that David Fowler's .NET project structure gist suggests that the .nuget folder has been deprecated since it shows NuGet.config existing next to the solution rather than in a .nuget folder, which where, historically, it has been located.
Fwiw, I just manually add packages.config beside the sln and edit it by
hand. Packages are restored via build script. The solutiom packages, and
versions thereof, dont change often so its not that painful.
NuGet.exe, by default, only looks for packages.config in the .nuget folder and not next to the solution. I can force it to look at packages.config next to the solution by running nuget restore .\packages.config but this will restore packages from that packages.config only and not from the various packages.configs contained in the projects in the solution, so I'd have to do a second pass nuget restore My.sln to do that. Is this how you are doing it? With two passes?
for project that referenced above on this issue, I was forced to use for this case ~
- if from cmd:
gnt.core /p:ngconfig="slnlevel-config|project1-config|project2-config" - if inside IDE:
#[NuGet gnt.raw("/p:ngconfig=\"slnlevel-config\"")]
so if you also have similar problem and if you don't care about new dependencies you can also try to resolve this temporarily, like above
until we decide... need or not -_-
We’ve had a couple of teams working around this for some time, with the release of StyleCop.Analyzers we’re likely to having to tackle this across the board. I see this issue is listed as priority one and in the 3.4 milestone. Does this mean the expectation is that an methodology will be in the next release?
For our purposes (mostly around tooling), we need a few features.
- Ability to specify a package and version solution wide
- Ability to explore, install, restore and update via a GUI
- Ability to bootstrap restore direct from source control both locally and on a build server
However, VS2013 round tripping is not really an issue for us. Simply having clear migration guidance would be fine.
In terms of how this could work we’re currently using a few methods including targets files directly pulling in packages and doing a restore (requiring manual intervention to update), package config/json files beside the SLN, restored again either by a targets file or a batch file.
Ideally we’d avoid a method that ends up editing individual project files. I could see the possibility of modification to packages config/json schema (by section or attribute) to indicate that package is to be applied across the solution.
I see this issue is listed as priority one and in the 3.4 milestone.
The problem with the 3.4 milestone is that it still has 234 open issues assigned to it. I don't understand this approach. 3.4 already has 95 closed issues, why not release features as they are closed? As it stands, I guess we'll be waiting for quite some time to get our hands on anything which is assigned to 3.4.
@adamralph you are absolutely right. We plan to release 3.4 rather soonish, and we will triage 3.4 items to a new milestone. This one particularly doesn't seem like is going to make it in, i'm moving it to 3.5
@Lexcess this bug does not track the issue of having a common version for a set of projects, it tracks the ability to install a set of packages once per solution that can be used for tooling and similar scenarios.
@yishaigalatzer thanks, that makes more sense.
With most projects I work on, we take a slightly different approach. We prioritise issues, and they are assigned to the milestone representing the next pending release when they are closed. When that release is done, a milestone for the next pending release is created so any issues closed after that simply fall into the next release bucket. This allows us to release at any time.
I understand, the reason we run how we run at the moment is as follows (and if you want more meta discussion, lets remove it from this issue, because it is really unrelated).
Our releases are tied to Visual Studio releases, not 100% true statement, but in many cases we release in coordination with Vs Updates, which means we have to test and validate against not yet released bits, which in turn doesn't allow us to just push because all the other partners haven't yet released we can create situations where new APIs are not solidly tested by our users yet. This is the case because we ship feature heavy releases, that are tied to feature heavy releases of visual studio (see Update I with new support for UWP, and Update II was attempting to bring in fundamental for .NET core), so we can't move as fast as we would typically.
Assigning things to a version is a way to communicate internally our intentions, but not a guarantee, so if in item is not in a milestone it means we don't plan to fix it for that milestone. If it is, it means we will try. 3.4 effort was taken over by .NET Core work, where we are merging in DNU and other features built by other teams in an attempt to make NuGet process more coherent. We haven't yet moved 3.4 issue we won't be able to fix out, which is why things are confusing at the moment. We will move them out.
@yishaigalatzer I thought that 3.4 was an unlikely milestone. Thanks for the update on this.
I'm definitely talking about Solution wide packages, I specifically mention tooling packages as our use case. Part of that should be the ability to set the solution wide package's version (one of our workarounds doesn't do this, which is why I mention it).
The main points I wanted to bring up were if there were any current thoughts on what the implementation of this feature might look like and is VS2013 round tripping a must have requirement? On the latter I am assuming both that the number of users round tripping will be lower this far into the VS2015 cycle and presumably such a requirement will make the new implementation harder (or limit options). In this case perhaps some migration guidance would suffice.
The general design is to have a roundtrippable story, but with an explicit model of installing packages into a single "packages.config" or "project.json" at a solution level.
You can mimic it today (without a UI) by putting entries in packages.config and running restore with nuget.exe on this file.
But we want to come up with an overall better solution, the design is very generic and we haven't nailed down the details.
Ok that's a model we can work with. Our main concern is that the workarounds we build in the interim will be relatively compatible with whatever the final solution is.
Just to drill down on how such a model might work in practice; would there be an expectation that packages in the future will have to be built to be solution wide? Alternatively would the notion be any package could be applied at project or solution level with the latter simply adding the package to all projects within a sln.
Just by way of example this has implications for a package like Stylecop.Analyzers where you have some configuration. It would be useful to be able to place configuration in one location (say solution items). However the current package would expect such configuration to be in each projects root folder, with the only way to share configuration being the 'add file as link' workaround.
with the latter simply adding the package to all projects within a sln.
This is exactly what I'm referring to in my first comment. Solution level packages and common package across the whole solution are two completely different things.
The general idea (and this is not a guarantee, just my current thoughts) is that you could install any package into this global level, and it doesn't have to be specifically designed (of course this aspect doesn't roundtrip backwards to 2013), but it has zero effect on projects. It just restores the packages to a global folder that is it.
For having a common version of a package across all the projects in the solution:
- We do no plan to build a top level configuration feature like this in package.config world, we might add feature that make it easy to stay on one version, but nothing that will auto change all the projects.
- We do plan on a feature in the project.json world. I'm working on summarizing the design at the moment and will post it in the wiki when I manage to find time to make it a coherent document.
To be absolutely clear, I'm not advocating the implementation of adding packages to every project under a solution, nor am I trying to get a common version of packages across a SLN. I was simply trying to understand what the current thinking behind the feature was. I do want global level packages!
It just restores the packages to a global folder that is it.
I think this is the area of confusion/disagreement. While what you describe is useful (especially for MSBuild based tooling). I was also hoping to see a way for tooling packages that operate at the solution level (such as StyleCop.Analyzers) to be added globally rather than added to projects.
As can be read from other posters further up the thread we currently have to do workarounds, such as create empty common project files to pull in such packages. From the sounds of it, the feature as currently envisaged wouldn't help directly with this tooling use case (although it might make the workarounds a little easier).
I was also hoping to see a way for tooling packages that operate at the solution level (such as StyleCop.Analyzers) to be added globally rather than added to projects.
Isn't that more about MSBuild/VS than NuGet? NuGet can restore an analyzer package at the solution level (i.e. just restore it to the package folder). That part is easy. But at build time, what is then is going to use it and apply the analysis to the code contained in all the projects?
Given the given scope of package restore to a global folder, I think you are probably right.
However, if for a second we accept the notion of global packages that work for the use case of solution level tooling (which is where I and I think others started from), then we'd probably need to see some changes in Nuget to make it work properly (i.e. so that files are restored to a useful place).
From the last few replies I think it's apparent that at best that would be a new feature request or at worst something that Nuget as a project just doesn't want support. I would still like to see a better solution than targets files and empty project files, but perhaps that has to start with the IDE/Build teams rather than nuget. I'd still be interested in the Nuget team's view though.
I was also hoping to see a way for tooling packages that operate at the solution level (such as StyleCop.Analyzers) to be added globally rather than added to projects.
I was under the impression that Roslyn analyzers are specifically designed to work against projects, not solutions, and they are hooked into the build process via the project system.
Solution level packages (in VS2013) are typically exe's, such as unit test runners, which don't fit into this workflow. I.e. they are usually invoked via build scripts and sit outside the MSBuild/VS ecosystem.
If you are proposing changing the Roslyn analyser ecosystem to make it aware of solutions rather than just projects then that seems like a much bigger undertaking. I'm not sure the benefit would be worth the added complexity.
@adamralph I am not proposing this.
@Lexcess then I'm afraid I don't really understand what you are proposing 😄
@adamralph No problem, as I said I mostly wanted to understand how this feature would work, especially given the goal of making it better. Thanks to @yishaigalatzer I get the proposed implementation and that will fix most of our gaps.
Beyond that was the use case of having packages be able to make changes at a solution/global level. StyleCop was my example but anything that might want to say add a sln level configuration, runsettings, targets or a file to solution items would be useful. We can do this already with some creative use of targets but it'd be nice to have in a more supported manner. Given the discussion this seems way beyond the scope of the current issue.
To give an example of how we'd like to use it in a large multi-solution project:
All rulesets for different targets(CSharp, Cpp, CSharpTest & CppTest) would be added to this quality packages, combined with runsettings, clonesettings and Resharper settings. In the projects we'll refer to $(solutionDir)RuleSet4CSharp.ruleset.
This will give me the advantage that we can centrally push these things, without having the need for inclusion in solution as solution items. It also cleans up our repos and avoids a maintenance hell.
@alexdegroot a solution level package is one that does not touch any .csproj files when it is installed.
In the projects we'll refer to $(solutionDir)RuleSet4CSharp.ruleset.
This is a characteristic of a project level package.
I've set up projects where all projects share a single common targets file, and things are pulled in that way. In this scenario a global packages.config file is a great way to pull in a single package to use in dozens of projects without the churn and noise of installing it into them all individually.
@martincostello it's great that you did ;) This is not the bug to discuss it. And this is exactly the reason we want to separate the two issues completely.
- Global way to install tooling into a project
- A way to commonize package versions across a solution or reuse package definitions across a solution.
BTW if you move to project.json the churn dies out, all you do is update the project.json files, and your csproj never ever gets touched.
@yishaigalatzer Yay for project.json! ... now if only we can kill off xproj entirely... ;-)
Hi @alpaix. Is solution/tooling level packaging still on track for 3.5 release? Cheers
@JackUkleja We're working on a schedule for 3.5 release. Will update the thread when we get right estimates.
Any news?
I'm now using Cake which makes it easy to install and run all tooling from NuGet without a solution or project.
no plan to bring this back for now. Dotnet tools (available with 15.7 shortly) will bridge this gap.
https://github.com/dotnet/cli/issues/5147#issuecomment-359836475
Hi @rohit21agrawal ; I can't relate your link to Visual Studio solution restore? Can you please examplify how to replace the ability of automatic package restore on solution level in Visual Studio without doing some kind of pre build script that runs on a project level?