NUGET_PACKAGES environment variable not being picked up from the system

Question

NUGET_PACKAGES environment variable not being picked up from the system

opened this issue 5 years ago · comments

We just updated to the latest version of TeamCity (2019.2) and some of the builds started failing, more specifically when running the unit tests. Everything worked correctly before that, when using TeamCity 2019.1.4.

We previously configured NuGet cache folder to be on a separate drive, by setting the System Environment Variable of NUGET_PACKAGES to our new path.
The problem is that when using the dotNet CLI plugin to run the tests, the new path is not picked up, but instead the default one is always being used \BuildAgent\system\dotnet\.nuget.

System Configuration:

System: Windows Server 2016
Net Core: 3.0
TeamCity: 2019.2

We've tried different things:

setting the System Environment variable
setting env.NUGET_PACKAGES=new_path in buildAgent.properties
setting NUGET_PACKAGES=new_path in buildAgent.properties

but none of these actually worked.

The workaround is to use the Command Line plugin and invoke dotnet test directly.

Are we missing something?

Nikolay Pianikov · Answer 1 · Sat Dec 14 2019 01:44:01 GMT+0800 (China Standard Time)

@rungureanu-eq I am not sure I understand how did you specify the environment variable at first? Did you specify it for the Windows user account related to the TeamCity agent service (the TeamCity agent restart is required in this case) or just did something like on my screenshot?

Radu Ungureanu · Answer 2 · Sat Dec 14 2019 17:23:37 GMT+0800 (China Standard Time)

@NikolayPianikov it was specified in Windows > Environment Variables > System variables. Not specific to any user.

Nikolay Pianikov · Answer 3 · Mon Dec 16 2019 19:34:31 GMT+0800 (China Standard Time)

@radu-ungureanu Did you restart the TeamCity agent after adding the environment variable? Could you please add the command set in the command line runner step to print out all environment variables? Could you find NUGET_PACKAGES in that list?

Deleted user · Answer 4 · Tue Dec 17 2019 01:06:47 GMT+0800 (China Standard Time)

@NikolayPianikov, yes, we restarted the agent itself and even the whole server.

Runing set, we can find the variable NUGET_PACKAGES=D:\Cache\NuGet\packages in the list.
This is the expected FilePath that we want.

Deleted user · Answer 5 · Tue Jan 07 2020 19:43:46 GMT+0800 (China Standard Time)

@NikolayPianikov was the information I provided enough? Can I help in any other way?

Nikolay Pianikov · Answer 6 · Sat Jan 11 2020 00:25:30 GMT+0800 (China Standard Time)

@rungureanu-eq Sorry for delay. It looks very strange. Could you please create a build step using the command line runner: dotnet new mstest and check that packages were restored using this cache path after build?

Deleted user · Answer 7 · Wed Jan 15 2020 17:40:10 GMT+0800 (China Standard Time)

Hmm, I'm not sure how to check that packages were restored using this cache path after build.

These are the logs that are displayed in TeamCity during the step execution:

Starting: D:\BuildAgent\temp\agentTmp\custom_script1001565014634932654.cmd
in directory: D:\BuildAgent\work\53a5246bcd95cea2
Getting ready...
The template "Unit Test Project" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on D:\BuildAgent\work\53a5246bcd95cea2\53a5246bcd95cea2.csproj...
Restore completed in 1,67 sec for D:\BuildAgent\work\53a5246bcd95cea2\53a5246bcd95cea2.csproj.

Restore succeeded.

Process exited with code 0

Nikolay Pianikov · Answer 8 · Fri Feb 21 2020 19:13:02 GMT+0800 (China Standard Time)

@rungureanu-eq I cannot reproduce this issue on the latest .net SDK. Which SDK did you use?
Is it possible to share the result of dotnet --info

Could you turn on the verbosity level diagnostics and find in the log lines as
Task "GetRestoreSettingsTask"
(out) OutputPackagesPath '....'

as workaround try using command line args --packages ....

Deleted user · Answer 9 · Mon Feb 24 2020 21:09:15 GMT+0800 (China Standard Time)

We currently have multiple installed SDKs on the build machine, due to the fact that we have multiple apps built there

.NET Core SDK (reflecting any global.json):
Version: 3.1.101
Commit: b377529961

Runtime Environment:
OS Name: Windows
OS Version: 10.0.14393
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\3.1.101\

Host (useful for support):
Version: 3.1.1
Commit: a1388f194c

.NET Core SDKs installed:
2.2.203 [C:\Program Files\dotnet\sdk]
3.0.100 [C:\Program Files\dotnet\sdk]
3.1.101 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.2.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.2.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.2.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Also, while running the test command of the plugin, with the verbosity set to diagnostics, there were 2 matches in the logs, both of them:
(out) OutputPackagesPath 'D:\BuildAgent\system\dotnet\.nuget'

johnendev · Answer 10 · Mon Mar 09 2020 13:28:17 GMT+0800 (China Standard Time)

Having the same issue, the NUGET_PACKAGES environment variable is ignored when running dotnet cli commands.

Maybe caused by this?
https://github.com/JetBrains/teamcity-dotnet-plugin/blob/master/plugin-dotnet-agent/src/main/kotlin/jetbrains/buildServer/dotnet/EnvironmentVariablesImpl.kt#L37

If that environment variable is already set then it should not be overwritten by the plugin.

Radu Ungureanu · Answer 11 · Fri Mar 27 2020 21:24:39 GMT+0800 (China Standard Time)

@johnendev you might be right with that file. I was also looking over it at the beginning. Now I see there's a new commit that changes the logic of the implementation
0863a4e

Maybe this helps solve the issues.

Nikolay Pianikov · Answer 12 · Fri Mar 27 2020 21:58:53 GMT+0800 (China Standard Time)

@radu-ungureanu yes I've fixed this issue recently in master, sorry for inconvenience. But you should be able to override it via build configuration environment variables before this fix like in the screenshot above

Radu Ungureanu · Answer 13 · Fri Mar 27 2020 22:17:09 GMT+0800 (China Standard Time)

No worries. Thanks for looking into it.
As I mentioned, we tried setting the variable as you've shown in the screenshot, but with no luck.

One question I would still have, is how to update the plugin? From what I know, now, it is being shipped together with TeamCity, so there isn't a way to update it separately.
Do we need to wait for a new release of TeamCity?

Nikolay Pianikov · Answer 14 · Fri Mar 27 2020 23:40:50 GMT+0800 (China Standard Time)

@radu-ungureanu you can download this plugin directly from this page. This fix is now in the Dev version (it is fully compatible with TeamCity 2019.2), thus some features might not be tested enough by our QA team or some features and UI might be changed in the Release version. But you could help our QA :)

And of course the next release version will contain this fix.

Alexey Adadurov · Answer 15 · Mon Nov 16 2020 17:55:06 GMT+0800 (China Standard Time)

For me, the workaround 'define environment variable in configuration' solved a similar issue when neither an environment variable defined for an agent in TeamCity, nor environment variable defined by docker-compose file that defined the configuration of the agent running on Linux were picked up by a build.

Jason DiOrio · Answer 16 · Thu Nov 03 2022 23:57:08 GMT+0800 (China Standard Time)

We're having an issue in TeamCity 2022 with the dotnet runner type ignoring our NUGET_XMLDOC_MODE environment variable (set in the build parameters). It's defaulting to skip but we need it to be none because we're using those xml docs. Is there a workaround for this? It appears to be very much related to this linked code which specifies "skip" in its defaultVariables

Jason DiOrio · Answer 17 · Fri Nov 04 2022 06:28:19 GMT+0800 (China Standard Time)

To gather some additional information, we ran a test build configuration using the following MSBuild project.

<Project>
  <Target Name="Build">
    <Exec Command='set' />
  </Target>
</Project>

The build log (redacted version below) indicates that, even though the build shows the environment variable correctly set to none in the parameters tab, the MSBuild execution is overriding this value:

Step 4/5: VS2022 Build Test (.NET)
15:09:29     MSBuild
15:09:29       Starting: "[REDACTED]\MSBuild.exe" [REDACTED]\EnvVars.csproj -t:Build -p:Configuration=Debug [REDACTED]
15:09:29       in directory: [REDACTED]
15:09:29       MSBuild version 17.3.1+2badb37d1 for .NET Framework
15:09:29       [REDACTED]\MSBuild.exe /noconsolelogger -bl:Debug.binlog -p:Configuration=Debug [REDACTED] [REDACTED]\EnvVars.csproj
15:09:30       Build started 11/3/2022 3:09:30 PM.
15:09:30       Project "EnvVars.csproj": Build
15:09:30         Project "[REDACTED]\EnvVars.csproj" on node 1 (Build target(s)).
15:09:30         Build:
15:09:30           set
[REDACTED]
15:09:30           NUGET_XMLDOC_MODE=skip
[REDACTED]

Dmitry Treskunov · Answer 18 · Mon Nov 07 2022 18:43:55 GMT+0800 (China Standard Time)

You are right, this bug was fixed in 2022.10 release https://youtrack.jetbrains.com/issue/TW-76151

Jason DiOrio · Answer 19 · Tue Nov 08 2022 05:58:38 GMT+0800 (China Standard Time)

Great, upgrading fixed us, thanks!