Final Boss test failure on Windows (line endings)
XaserAcheron opened this issue · comments
A peculiarity I noticed on Windows: the final-boss test fails due to line ending shenanigans:
The final-boss.output.js fixture uses \r\n everywhere, but the output generated by ember-modules-codemod is using \ns on those three lines for some reason. For grins, I hex edited the output.js fixture to change those 3 lines to ns, and the test passes, but this it obviously non-ideal. ;)
I say this is Windows-specific since Travis seems to be pretty chill with the current state of things, but of course that means it's probably on me to try and come up with a solution given how rare we ember-windows folk seem to be. Welp.
The last step in the codemod before saving to disk, you could do:
const EOL = require('os').EOL;
fs.writeFileSync('output.js', str.replace(/\n/g, EOL));like we do here. This would be a noop for Unix.
Do I understand correct that currently ember-modules-codemod outputs different line endings for linux vs windows? If so then running a codemod on windows will violate ember-cli's
editorconfig from the app blueprint.
Or is it just a test setup issue?
The codemod itself doesn't touch lines it doesn't care about, fortunately, even in Windows-land.
Thinking on it a bit, the likely culprit means there's something inaccurate in my opening post -- I'll bet the source doesn't use \r\n at all; just \n... because Git is configured on my system to convert the line endings to Windows' format. Oy vey -- I really don't know why that's considered the "recommended" approach with Git; you'd think we'd have this problem licked in 2017. ;)
[EDIT] Yup, there it is. ember-modules-codemod itself uses \n consistently. The \r\ns are Git conversion artifacts on my end.
Unfortunately this does also mean that the codemod always inserts \ns even if the file in question is using \r\ns, so I may write a quick patch that will check .editorconfig if it's present and use require('os').EOL otherwise. Hopefully that will knock out the non-edge cases.
I'm sure everyone's rolling their eyes at yet-another-one-of-these-problems surfacing, but at least it isn't spaces/tabs. ;)
@XaserAcheron maybe we can force autocrlf=false on repository level somehow? If it's possible does it sound like a sufficient solution?
upd: for me it seems cheaper than editorconfig stuff.
thanks @kellyselden
That'd do it for the unit tests (and is probably a good call for everyone's sanity), but there's a distinct-but-related issue: when actually running the codemod, it's going to insert \ns, resulting in mixed line endings in files that already use \r\n.
...but thinking on it, that's probably best left for another time, if at all. That really is a lot of effort spent fixing something that's (IMO) super-minor. The important bit is that the tests work, so +1 for @ro0gr's PR (#45) and calling this done-until-further-notice.
I think this is fixed by #45 now, can you confirm and close?
Huh -- now I'm getting the literal opposite problem: every test is failing, with every line flagged as different except the ones in the final boss test that were problematic before. :P
The .gitattrbutes patch did what it was supposed to -- my local files on a freshly cloned repo have \ns, but the codemod is still spitting out \r\ns. I did some searching and found this line in recast; looks like it's unconditionally using the OS-defined line endings if nothing else is specified, and I'm not seeing anything in the codemod that's overriding it.
Insert audible sigh here, I suppose.
the same issue. I've just tried #47 to pass option to recast and seems like it fixes the issue
I think it may make sense to just do the inverse of #47 and convert all hardcoded \ns with require('os').EOL -- that'll keep things consistent with recast at least, and probably resolve most of the issues for folks (since "convert to CRLF on checkout" is the Git default on windows).
It's a step forward at least.
I think it may make sense to just do the inverse of #47 and convert all hardcoded \ns with require('os').EOL -- that'll keep things consistent with recast
isn't it the same as it was before #47. We did't specify lineTerminator and that was the reason why all the tests failed on windows. I think it's still important to make sure that we have consistent line endings in the fixtures in order to make testing deterministic.
What if we try to detect an actual source file line ending before the transform? I think detection is not a big deal but seems like there are few caveats:
- it looks like recast supports single
lineTerminatorper invocation so it would be impossible to handle mixed line endings in the same file. Don't think it's a real issue - it complicates testing a little bit
isn't it the same as it was before #47.
Not quite. The "before scenario" on Windows goes something like:
- Git (assuming recommended settings) uses the OS default (
\r\n) recastuses the OS default (\r\n)ember-modules-codemodalways uses\n(whoops! inconsistency happens)
I'm suggesting making the codemod use the OS default (\r\n on Windows) for starters, since that's a step toward keeping things in sync (leaving the user's Git settings as the wildcard). Doing some file detection and/or .editorconfig checking would close the gap even further, so that's a great next step. 👍
When running ember new appname git settings are not taken into account.
I've just checked freshly generated app and all the files have lf line endings.
So I think if we follow recast default we may introduce inconsistency rather than fix it on a real apps.
My point is that we need to consider git part only for fixtures consistency purpose.
In the transformation itself we should not take git into account and detect an actual source file line endings(even not editorconfig). That's how I understand the situation at least..
@XaserAcheron I'm going to make one more pr(with auto-detection) if you don't have strong objections.
I hope this way we can validate an approach at least
When running ember new appname git settings are not taken into account.
I've just checked freshly generated app and all the files have lf line endings.
It turns to \r\n when the file gets modified by Git (e.g. checking out a different version when switching branches and whatnot), though that indeed means a fresh, never-modified file will retain its LFs as long as Git (or some other editor) never messes with it. I suppose that means all bets are off for original authors of an app/addon. It's edge cases all the way down. :P
Auto-detection ought to make this all moot anyway. Go for it!
never-modified file will retain its LFs as long as Git (or some other editor) never messes with it
of course you are right
Confirmed that #48 fixes all my nutty Windows gripes. ;)