`yarn install` ignores `files` from local dependencies
phtrivier opened this issue · comments
What is the current behavior?
It seems that yarn install ignores files from package.json of projects that are locally depended on.
If the current behavior is a bug, please provide the steps to reproduce.
Using two simple projects that depends on each other locally:
○ → tree
.
|____project1
| |____bad.js
| |____good.js
| |____package.json
|____project2
| |____package.json
project1 specifies which files should be copied by dependent projects:
// project1/package.json
{
"name" : "project1",
"version" : "0.0.1",
"files" : ["good.js"]
}
project2 depends on project1 locally:
// project2/package.json
{
"name" : "project2",
"version" : "0.0.1",
"dependencies" : {
"project1" : "file:../project1"
}
}
npm install in project2 only copies the good file:
2017-03-02 12:43:41 ⌚ turus in ~/tmp/yarn-issue/project2
○ → npm install
project2@0.0.1 /home/phtrivier/tmp/yarn-issue/project2
└── project1@0.0.1
npm WARN project2@0.0.1 No description
npm WARN project2@0.0.1 No repository field.
npm WARN project2@0.0.1 No license field.
2017-03-02 12:43:45 ⌚ turus in ~/tmp/yarn-issue/project2
○ → ll node_modules/project1/
total 32
drwxrwxr-x 2 phtrivier phtrivier 4096 mars 2 12:43 .
drwxrwxr-x 3 phtrivier phtrivier 4096 mars 2 12:43 ..
-rw-rw-r-- 1 phtrivier phtrivier 37 mars 2 12:37 good.js
-rw-rw-r-- 1 phtrivier phtrivier 1171 mars 2 12:43 package.json
However, yarn install copies everything, ignoring files:
2017-03-02 12:43:48 ⌚ turus in ~/tmp/yarn-issue/project2
○ → rm -rf node_modules
2017-03-02 12:44:55 ⌚ turus in ~/tmp/yarn-issue/project2
○ → yarn install
yarn install v0.21.3
warning project2@0.0.1: No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 0.26s.
2017-03-02 12:44:58 ⌚ turus in ~/tmp/yarn-issue/project2
○ → ll node_modules/project1/
total 44
drwxrwxr-x 2 phtrivier phtrivier 4096 mars 2 12:44 .
drwxrwxr-x 3 phtrivier phtrivier 4096 mars 2 12:44 ..
-rw-rw-r-- 1 phtrivier phtrivier 57 mars 2 12:37 bad.js
-rw-rw-r-- 1 phtrivier phtrivier 37 mars 2 12:37 good.js
-rw-rw-r-- 1 phtrivier phtrivier 80 mars 2 12:38 package.json
2017-03-02 12:45:03 ⌚ turus in ~/tmp/yarn-issue/project2
○ →
What is the expected behavior?
Same behavior as npm.
Please mention your node.js, yarn and operating system version.
○ → npm --version
3.10.10
○ → yarn --version
0.21.3
○ → uname -a
Linux turus 4.8.0-39-generic #42-Ubuntu SMP Mon Feb 20 11:47:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
I suspect this may be a common error across other resolvers as well.
Normally when you npm publish a package, only the items listed in files are included in the .tgz, so for normal NPM packages, the cli tool (npm or yarn) doesn't have to really worry about it.
(you can verify this by setting up an offline cache directory and yarn add bootstrap, then compare the downloaded .tgz to their GitHub repo. Only the items from files are in the downloaded/cached .tgz, not every file in the Github repo)
For other resolvers, it's going to have to filter on its own. I did a quick search through the yarn source and didn't see anything that would be doing that check (though I could have missed it).
@rally25rs yarn pack does this filtering to avoid bundling them before publish.
Confirming @rally25rs's suspicion: this issue appears to also affect installation from GitHub using the git+https:// protocol. I've seen Yarn install a private GitHub dependency with its src and build folders, despite its package.json:files array containing only the build folder.
I suppose publishing with yarn pack is the workaround, but that's not feasible for all projects. Ultimately it'd be better to honor the files property.
For private git repositories, if npm is able to filter packages by reading the files:[] property, shouldn't yarn be doing the same ?
We face exactly the same problem with a dependency we're pulling directly out of our private corporate git repository. Any idea when this will be fixed?
Has there been any progress made on this?
just out of curiosity, is there any plan to change this behaviour?
I've just run into a situation where I need one specific file from private repo A as an import in private repo B, but the file I need is created as part of repo A's build step, so I don't need the other 5MB of stuff that it contains...
I've created a package.json in repo A with the relevant files key, but as other people in this thread have discovered, npm install limits the files that are retrieved, while yarn add does not. looking through the output from yarn add --help, it seems there are no command flags to control this, so is this the behaviour we're stuck with now?
Not sure if it's related, but I'm experiencing a situation where yarn ignores package.json:files from a workspace package.
You can see the src dir being included when running yarn pack (as well as when installing the dep from npm), even though it's not included in package.json:files.
cat package.json
{
"name": "react-cosmos-test",
"version": "4.0.0-beta.0",
"description": "APIs for testing Cosmos fixtures in headless environments",
"repository": "https://github.com/react-cosmos/react-cosmos/tree/master/packages/react-cosmos-test",
"license": "MIT",
+ "files": [
+ "lib",
+ "enzyme.js",
+ "generic.js"
+ ],
"dependencies": {
"react-cosmos-config": "^4.0.0-beta.0",
"react-cosmos-loader": "^4.0.0-beta.0",
"react-cosmos-shared": "^4.0.0-beta.0",
"traverse": "^0.6.6"
},
"xo": false
}yarn pack
tar -tf react-cosmos-test-v4.0.0-beta.0.tgz
package
package/enzyme.js
package/generic.js
package/lib
package/package.json
+package/src
package/lib/enzyme.js
package/lib/generic.js
+package/src/__tests__
+package/src/enzyme.js
+package/src/generic.js
+package/src/__tests__/enzyme.js
+package/src/__tests__/generic.jsI appreciate any help with this one as it's annoying for react-cosmos users to download unnecessary source files, which among other things can cause Flow errors in the dependent project. Thanks!
Hi everyone, I'm going to fix it.
After the initial research I have two questions.
I plan to modify https://github.com/yarnpkg/yarn/blob/master/src/fetchers/copy-fetcher.js#L10, and then I have several options:
Option one:
- read files field from package.json
- delete all the files that do not match regexp (using the logic from https://github.com/yarnpkg/yarn/blob/master/src/cli/commands/pack.js#L75)
Option two:
- run pack command for this directory
Which one is better?
And the second question – does doing it in the fetcher make sense?
@BYK @kittens I will appreciate your feedback on how to proceed here.
This problem occurs to me when depending on local packages using Yarn workspaces, too.
Could someone please confirm if this is the same issue? Or should I create a new issue for yarn workspaces specifically?
E.g.
workspace/packages/a has files: ['good.js]
workspace/packages/b depends on package a through yarn workspaces
yarn install --production results in package b's node_modules/a having the full contents of package a (even when package b has been set as nohoist), while it should only include good.js
If this is a separate issue please let me know so I can create it.
As I can see the problem in that yarn using git archive
https://github.com/yarnpkg/yarn/blob/master/src/util/git.js#L225
command for fetch packages and its not included '.git' folder and no ignore checks at all here
https://github.com/yarnpkg/yarn/blob/master/src/fetchers/git-fetcher.js
If I added files: ['.git'] (for fetching with submodules also) - its will be another behavior than npm :(
very sad
Have the yarn team any plans for this?
I would love to see this issue solved. if npm had this issue, it's solved already... and this one is marked as high priority since 2017 and nothing untill now... This is blocking my development :/
+1, I am building a component library with workspaces and lerna. Imports only work when I import @scope/package/**dist**/Component. This obviously doesn't look good or translate when when it's build, as the published package will be available at @scope/package/Component.
strange that this is still an issue, it's like the first issues I ran in straight when using lerna and yarn, got me all confused
Actually today (version 1.19.1), at least in my project, even the workaround yarn pack does not work and includes all files.
JFTR, I just came across with this after upgrading yarn from 1.12.3 to 1.15. Surprisingly, it worked with old version w/o any workaround.
This is exceptionally annoying as far as using certain IDEs is concerned e.g. auto imports can be barfed when using typescript
Any word on this? I am running into this when developing a package the specifies the babel transpiled dist directory in files. So when someone who gets it from the repository uses it they will do import something from 'mypackage/somefolder'. But as the yarn workspaces link is to the root of the package not the dist folder, I have to do import something from 'mypackage/dist/somefolder'
experiencing the same issue
any update on this?
I'm having same issue, trying to test a package before publishing and using folder reference yarn gets all the contents including node_modules folder, which creates issues for typescript. Using yarn pack and installing tarbal seems to solve it but then I run into the issue of yarn caching that tarball, so I have to clean yarns cache every time and I run into errors in there, some directories don't exist etc etc... so either way is not working for me.
@merceyz this has been tagged as "fixed-in-v2" but it is not (v2.4.0). When I specify a dep as "file:...", yarn copies the full tree, tatally ignoring the dep package.json files entry. This is particulary anoying for sample / example projects in a subdir of the main project, since it get recursively copied...
This is still an issue on yarn 1.23.0. Tried yarn 2 and it did not appear to be fixed there either as hl037 states.
Any updates on this? As of now I am still encountering this issue. I am trying to install a private package in a project and only need the /dist folder in the package, but also have some other dependencies in the same repo that I am using for the documentation of the package, that I dont want in the project I am using the package in. It's particularly annoying that this feature works fine with npm, but not with yarn.
So... this might be insanely silly but I added a prepare script that did nothing -- just "prepare": "echo something" and the install began respecting the my package.json's files property. Hope this hack works for others!
So... this might be insanely silly but I added a
preparescript that did nothing -- just"prepare": "echo something"and the install began respecting the my package.json'sfilesproperty. Hope this hack works for others!
Awesome that worked! Thank you
So... this might be insanely silly but I added a prepare script that did nothing -- just "prepare": "echo something" and the install began respecting the my package.json's files property. Hope this hack works for others!
@akramer24 In my experience, this only works for git repos. Was that your use case?
No solutions nor workarounds yet for local dependencies...
So... this might be insanely silly but I added a prepare script that did nothing -- just "prepare": "echo something" and the install began respecting the my package.json's files property. Hope this hack works for others!
@akramer24 In my experience, this only works for git repos. Was that your use case?
No solutions nor workarounds yet for local dependencies...
It was a git repo in my case
Looks like there's been a PR open for this issue since last year. Can someone with access get around to merging it please?
This is not a novel use case and should be supported because npm supports it.
Looks like there's been a PR open for this issue since last year. Can someone with access get around to merging it please?
This is not a novel use case and should be supported because npm supports it.
FYI (and no idea why this decision was thought to be good or why they still allow issues to be open here) this repo is no longer maintained and the new version is over here: https://github.com/yarnpkg/berry
Trying to upgrade has been an absolute mess with yarn and we have since moved back to npm and it has fixed all of our issues. Unfortunately I do think yarn has shot itself in the foot by messing up the upgrade path & moving to a different repository but thats the beauty of open source :)