Support yarn v2
oliversalzburg opened this issue · comments
In a repository using yarn 2, vsce package
will fail trying to enumerate dependencies:
ERROR Command failed: yarn list --prod --json
To reproduce, in an extension repository, use:
yarn set version berry
yarn install
vsce package
What does yarn list --prod --json
return?
Can I clone your repo and try it out?
It returns Usage Error: Couldn't find a script named "list".
, as it's not a valid yarn2 command: https://yarnpkg.com/cli/
My reproduction should work in any extension repository, but I created a fresh one that should showcase it perfectly: https://github.com/oliversalzburg/vsce-yarn2-support
Likewise, yarn 2 omits the run command since it's not needed. Yarn 2 is not meant to be installed globally (yet) If there is yarn available in global, then it's probably v1. I don't know if vscode is running an embedded yarn or global yarn, but ordinarily yarn v1 checks if yarn 2 (or any yarn) is installed in the repository and runs it instead.
In my case, running an npm script from the side panel results in this:
The terminal process "C:\Program Files\PowerShell\7\pwsh.exe -Command yarn run dev" terminated with exit code: 1.
Which is incorrect, it should just be yarn dev
I had the same issue, here was my temporary fix, although it can be annoying,
- Optional: Checkout a new temp branch so that package-lock.json and other npm files to carry over
- Run
npm i
- Run
vsce package --no-yarn
just info, as of now yarn list
still not implemented in yarn berry: https://yarnpkg.com/getting-started/migration#not-implemented-yet
Not the cleanest/best solution, but this is how I circumvented this issue:
- Configure your
.yarnrc.yml
with alockfileFilename
different fromyarn.lock
- Run
vsce
with the--yarn
parameter (since it won't detect Yarn using theyarn.lock
anymore)
This will make vsce
skip its whole process of looking at your dependencies and partially including node_modules
in the packaged extension. Skipping this process also skips its internal yarn list
(and because of --yarn
also npm ls
) which prevents this error from happening.
The only issue with this is that vsce
might bundle some items in node_modules
if it isn't in .vscodeignore
or something similar. You'd lose that "bundling" of course (which you would lose anyway if using Yarn PnP), but if your extension doesn't have any dependencies or you bundle it using e.g. Webpack, this is a non-issue.
Of course, having vsce
support Yarn 2 natively is still better.
With the freeze of yarn v1 this should perhaps be re-prioritized.
With the freeze of yarn v1 this should perhaps be re-prioritized.
See #439. I think yarn v2 is supported.
This ticket is already two years old and we still do not have full support for yarn 2.
For example vsce ls
command is fully broken because:
$ npx vsce ls
INFO Detected presence of yarn.lock. Using 'yarn' instead of 'npm' (to override this pass '--no-yarn' on the command line).
ERROR Command failed: yarn list --prod --json
There is no list
in yarn 2 and if you try to pass --no-yarn
to vsce, you endup with the other problems and failures.
I guess that for the moment I will need to replace vsce ls
with zipinfo -1 *.vsix
.
In general, it seems like very poor practice to rely on a specific third party shell application to perform anything of meaning in a NodeJS project, especially when that third party application is a package manager.
Which is why I have given up on vsce a long time ago. Please open a new issue to track the total absence of progress on this issue, if you need to.
Contributions here are more than welcome! Yarn v2 just isn't that big of a deal for us and most of our users (only 8 upvotes here), so this issue never makes it to the front page. Also, and I'm sure you know, we're constantly busy with plenty of other work.
@joaomoreno Thanks for clarifying as not everyone understands what is expected to be fixed by core team and what goes into welcoming fixes from outside. Some might get frustrated waiting for something to happen that nobody is working on.
At least knowing that the core team does not see yarn v2 as a priority, could motivate some to propose fixes. I fixed one yarn bug last week and I may have to fix this one too.
I did check the outcome of vsce package
with yarn v2 and the extension seem to be correctly built, so the incompatibilities are kinda minor, more around on the missing or wrong command line arguments.
I was recently forced to switch to yarn because npm link
-ing was never fixed and apparently they are not going to do it. One extra benefit was that operations seem to be like 5x faster if not even more and managing dependencies become much easier. I really do not want to go back to npm.
And yep, I am closely watching code team release notes each month, I am giving them as a good example on how to deliver fast and also inform users about what changed. Great work!
If you need to keep track of your lack of interest to work on this subject, please create your own internal ticket. I don't want to be bothered by activity on this repository.
@oliversalzburg You already mentioned that you are no longer interested in following it up and you could have just pressed the unsubscribe button. That is an open-source project and it is not nice to undo maintainers work, I even seen people being temporarily banned for not playing nice.
Regarding broken ls
command, the workaround is to use npx vsce ls --no-dependencies --no-yarn
, which seems to be working with yarn2.
@oliversalzburg, please unsubscribe.
@joaomoreno Thanks for clarifying as not everyone understands what is expected to be fixed by core team and what goes into welcoming fixes from outside. Some might get frustrated waiting for something to happen that nobody is working on.
This is why the issue is currently labeled help wanted
, I guess we should somehow make that label clearer. Here's where we inform this.
Did you all see the comment about reacting to the OP? That's how they track heat. I wish that was more obvious.
I also ran into this issue, but resolved it by creating a Yarn plugin that implements the list
command. I have only tested this on our own repository, so I may not have covered all edge cases, but it works for us.
See: https://gist.github.com/arendjr/415747652a8c79f12b005ce3cfb2f808
If there is interest in contributing this to Yarn itself, please let me know.
Hi @arendjr, thanks for sharing your plugin. I think that it's a good idea of having your plugin listed on the yarn
website.
How can I help to make this? I guess that we would need to put the plugin in a dedicated package perhaps.
Hi @joaomoreno, from your point of view, how would you envision this issue moving forward? I might be lacking context but I think that having @arendjr plugin installed on yarn berry
repositories it's good and probably it will evolve to support the previous use cases from yarn list
from yarn v1
, but at the same time, maybe vscode-vsce
shouldn't depend on the users installing the plugin but rather have this functionality embedded.
Maybe another way to handle it could be listing @arendjr's plugin as a prerequisite of vscode-vsce
for supporting yarn berry
.
edit: I'm happy to help, I just wanted to understand what would be the best approach to follow 😄
Thank you so much for working on this @lordrip :). Fyi, it seems that yarn list
has simply be renamed/restructured to yarn info --recursive
in case that helps you with your WIP.
Hi @ivangabriele I haven't worked on this yet, I would like to make sure that this is still needed first. Thanks for the heads-up about the renaming, I gave it a try and it seems that the JSON output is not exactly right:
Using:
yarn info --recursive --json
this is the output, notice how it will need a little bit of adjustment before using it:
{
"value":"@4tw/cypress-drag-drop@npm:2.2.4",
"children":{
"Instances":1,
"Version":"2.2.4"
}
}{
"value":"@adobe/css-tools@npm:4.2.0",
"children":{
"Version":"4.2.0"
}
}{
"value":"@ampproject/remapping@npm:2.2.1",
"children":{
"Version":"2.2.1",
"Dependencies":[
{
"descriptor":"@jridgewell/gen-mapping@npm:^0.3.0",
"locator":"@jridgewell/gen-mapping@npm:0.3.3"
},
{
"descriptor":"@jridgewell/trace-mapping@npm:^0.3.9",
"locator":"@jridgewell/trace-mapping@npm:0.3.18"
}
]
}
}
@lordrip Indeed I didn't check in detail how it's used yet. Normally I think that could be a one-shot fix by updating this function and detect Yarn 2/3 via the .yarn
directory existence.
I can at least confirm I had the same issue today on a brand new extension using Yarn stable (v3.6.1).
In the meantime I used @arendjr script.
I can try to poke a PR if you wish ^^.
Sure, I'm not the maintainer but I'm confident that it will help a lot.
If you happen to submit a pull request I can lend you a hand if needed.
If it helps, I adjusted a bit @arendjr 's plugin here to incorporate the yarn link
workflow as well
Would it help if I moved my script into its own repository?
Hi @arendjr, as I mentioned in a few comments above, I think that there's value in trying to publish your script in the yarn plugin
list.
Plenty of plugin examples exist, for instance, yarn-plugin-env-vars.
That being said, I don't think that both efforts are mutually exclusive, at least how I see it, it would be quicker to have a public plugin, installed through the CLI, like:
yarn add arendjr-plugin
while we wait for a dedicated implementation in this repository, which probably would be the best approach in the long run 😃
EDIT: I took your cool script and adjusted it a little bit in order to support linked packages, the ones that you get when you do a:
yarn link ~/my-package
Alrighty, this should do the trick :) https://github.com/arendjr/yarn-plugin-list
I have incorporated your support for Yarn resolutions as well, @lordrip
Thanks again here! Fyi I will try to open a PR-fix today or tomorrow for this issue.
I was blocked by this issue for a while, but in case it is helpful to others, my workaround was to use esbuild to bundle, then pass --no-dependencies
to my package command. This skips the outdated yarn list
command. Credit for this idea: #421 (comment)
For emphasis:
there is no need to include npm dependencies when package and publish, all things is in your bundle
As an additional aside, it looks like the compatibility issue looks like it was introduced in v2.9.2
: https://github.com/microsoft/vscode-vsce/releases/tag/v2.9.2. If you are OK using an earlier version of vsce
, this might help too.
Closing this as out of scope. As part of our recurring maintenance milestone, we've decided to limit the amount of supported package managers to npm and yarn v1 only.
That being said, I'm glad workarounds exist for anyone who is blocked.
Have you considered using an off-the-shelf solution for resolving Node.js dependencies instead? It seems like the package manager-dependent code is just used for listing which node_modules
should be put into the VSIX package, and I think that could be done by using something like https://github.com/vercel/nft
I think resolving dependencies from the code would lead to better accuracy (since you cannot be so sure all the package maintainers are managing devDependencies
and dependencies
well) and better interoperability overall without juggling over every package manager VSCE may need to support.