Trouble converting with file:// urls
jneuendorf opened this issue · comments
Issue description:
When running synp --source-file yarn.lock after a fresh yarn install I get this error:
The "url" argument must be of type string. Received type undefined
I couldn't find the message's text anywhere in this repo so it must come from some dependency.
Therefore, I have no idea where to look or what the error means exactly. 😞
Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:
Are you reporting a conversion error? Please also include a link to your package.json:
Here part of the file (I can't publicly post it because it's a project at work):
{
"name": "projectX
",
"version": "0.0.1",
"private": true,
"scripts": {
"test": "jest"
},
"dependencies": {
"babel-polyfill": "^6.26.0",
"bulma": "^0.7.1",
"change-case": "^3.0.2",
"classnames": "^2.2.6",
"d3": "^5.7.0",
"emmapi": "file:../emmapi",
"fastclick": "^1.0.6",
"fetch-everywhere": "^1.0.5",
"fetch-headers": "^2.0.0",
"immutable": "^3.8.2",
"jquery": "^3.3.1",
"js_utils": "file:../js_utils",
"jsc-android": "^236355.0.0",
"lodash.groupby": "^4.6.0",
"lodash.memoize": "^4.1.2",
"lodash.uniqueid": "^4.0.1",
"moment": "^2.22.2",
"prop-types": "^15.6.2",
"react": "16.3.1",
"react-day-picker": "^7.2.4",
"react-decoration": "^2.0.0",
"react-dom": "^16.3.2",
"react-is": "^16.4.1",
"react-native": "0.55.4",
"react-native-communications": "^2.2.1",
"react-native-device-info": "0.24.0",
"react-native-fetch-blob": "^0.10.8",
"react-native-keyboard-aware-scroll-view": "^0.7.4",
"react-native-navigation": "^1.1.478",
"react-native-pdf": "^3.0.16",
"react-native-push-notification": "^3.0.2",
"react-native-splash-screen": "^3.1.1",
"react-native-vector-icons": "^4.6.0",
"react-native-wkwebview-reborn": "^2.0.0",
"react-redux": "^5.0.7",
"react-spring": "^6.1.9",
"react-with-gesture": "^2.0.4",
"redux": "^4.0.0",
"redux-logger": "^3.0.6",
"redux-promise": "^0.6.0",
"redux-thunk": "^2.2.0",
"smoothscroll-polyfill": "^0.4.3",
"url-polyfill": "^1.0.13"
},
"devDependencies": {
"babel-cli": "^6.26.0",
"babel-eslint": "^8.2.3",
"babel-jest": "22.4.4",
"babel-loader": "^7.1.4",
"babel-plugin-module-resolver": "^3.1.1",
"babel-plugin-preval": "^3.0.1",
"babel-plugin-transform-decorators-legacy": "^1.3.4",
"babel-preset-env": "^1.7.0",
"babel-preset-react": "^6.24.1",
"babel-preset-react-native": "4.0.0",
"babel-preset-stage-0": "^6.24.1",
"cross-env": "^5.2.0",
"css-loader": "^0.28.9",
"eslint": "^4.19.1",
"eslint-plugin-import": "^2.12.0",
"eslint-plugin-react": "^7.8.2",
"extract-loader": "^2.0.1",
"extract-text-webpack-plugin": "^4.0.0-beta.0",
"fast-sass-loader": "^1.4.5",
"file-loader": "^1.1.11",
"globby": "^8.0.1",
"html-inline": "^1.2.0",
"jest": "22.4.4",
"jsdom": "^11.11.0",
"ncp": "^2.0.0",
"node-sass": "^4.9.0",
"postcss-loader": "^2.1.4",
"react-test-renderer": "16.3.1",
"resolve": "^1.7.1",
"resolve-url-loader": "^2.3.0",
"sass-loader": "^7.0.3",
"svg-sprite-loader": "^4.1.3",
"url-loader": "^1.1.2",
"webpack": "4.5.0",
"webpack-cli": "^2.0.14",
"webpack-shell-plugin": "git+https://github.com/cdeutsch/webpack-shell-plugin.git#bee537d"
},
"jest": {
"preset": "react-native"
}
}
Hey @jneuendorf - in order to make this work, I had to delete the emmapi and js_utils packages that lead to files on your HD. I didn't have them, so couldn't run the install.
I removed them, ran the install and then the conversion worked for me.
May I suggest you see if this works for you? If it does, you can easily add them later after the conversion and then npm install. Your locked versions will not be changed.
If that is the case and you'd like to submit a PR to fix this issue with files, it would be greatly appreciated. :)
I'm getting the same issue. I think it might be b/c I have local dependencies ("file": "../pkg"). If I go into the yarn.lock file and remove all of the dependencies like pkg@file:../pkg then the conversion works fine. Maybe the tool expects each package to be some real URL and not potentially a path.
I was guessing this as well. I am currently looking around in the code and see if I find a nice solution (=> PR?! 😉).
The easiest approach I can think of is like @imsnif said: Remove the file entries, do the conversion, and then tell the user to run npm install manually. But maybe I find out which tool actually throws the error and get the conversion work out of the box and without any workarounds.
I'll keep you updated. 😄
That's great @jneuendorf - let me know if you have any questions or if I can help in any way!
@imsnif I actually already have a question 😉 In entry.js the object representing an entry in the according lock file is created. I.e. entry.requires is set to a value derived from dependencies:
Lines 67 to 71 in c6f4e38
In an NPM lock file there also is a
dependencies key in (almost?) each entry, but that key is never set - what I mean is, I didn't find entry.dependencies = ....
Is that on purpose? Can this always be omitted or could this information be relevant in the case of file dependencies?
Hey @jneuendorf - the dependencies value is a special case. It needs to include all the child entries of a package.
Synp traverses the dependency tree "top to bottom" (or "left to right"), meaning it starts with the package itself, continues with its children, moves on to its "grandchildren" etc. For this reason, when it calculates an entry, it doesn't yet have the entry information of its children - which is what we need for the dependencies field.
For this reason, we do it the other way around. An entry is added to its parent's dependencies after it is created. It's done here: https://github.com/imsnif/synp/blob/master/lib/tree.js#L52-L54
I hope I managed to explain this clearly enough?
Hey @imsnif thanks for your quick and detailed answer. It makes things a lot clearer.
And it means that I don't have to worry about the dependencies in my changes because your code already takes care of that, right?
@jneuendorf - that sounds right, but I'll have to look at the code to be sure. :)
What I suggest you do is:
- Make sure the existing tests still pass.
- Add new tests for your feature that would make sure you get the desired result - you can get inspiration from the existing tests if you like.
Makes sense?
Hey @imsnif, I just created my test case. 🎉
But I noticed that lots of tests are only done for either NPM -> Yarn or Yarn -> NPM (bot not both).
Is this on intent?
Is it due to the fact that one would need 2 fixtures (one for each test) so the existing and generated files do not conflict?
And would it be ok for you if I committed an .editorconfig file? It's nice because it keeps the code style consistent across contributors and editors (i.e. I usually use 4 spaces as indentation in Atom).
And is it ok to commit .yarn-integrity (in my fixture's node_modules)?
Hey @jneuendorf - sorry for the delayed response!
Relevant test cases (such in this case) should have test cases both ways. Since synp is not 100% accurate (due to the caveats stated in the README), there should be one fixture for each direction.
I don't mind an .editorconfig file (that passes standard :) ), but would rather have it in a separate PR to keep things organized.
Hey @jneuendorf - looks good! Nice work. Might have some small comments, but nothing major. Want me to review it as is, or would you like to add the test first?
Hey @imsnif, sorry for the delayed answer. I've been incredibly busy with work and college.
But I just pushed the missing piece: the 2nd test case and according code changes. 🎉 How you can have a look and let me know what you think. 😉
@jneuendorf - I merged your PR and published a new version. Looks great - thanks so much for your contribution!