gulp-ng-annotate appears to be destroying previous sourcemaps
OverZealous opened this issue · comments
I can tell you've already had a bunch of fun with sourcemaps, so I hate to bring this up. However, I have a script that, among other things, passes the files through gulp-6to5, then through gulp-ng-annotate.
If I leave ng-annotate in-place, I only get sourcemaps showing the changes from ng-annotate, as if the original source was the output from 6to5. If I remove ng-annotate, I clearly see the source map from 6to5. For some reason, ng-annotate is clobbering the sourcemap.
I've removed all extra steps to verify the issue occurs with 6to5. Obviously, since 6to5 is an ES6 compiler, and ng-annotate does not understand ES6 code, I cannot reverse the order to see if there is still an error.
This might be fixed if gulp-ng-annotate updated its dependency on ng-annotate to v0.14.0
Quoting from:
https://github.com/olov/ng-annotate/blob/master/CHANGES.md
v0.14.0 2014-11-27
- support sourcemap combination and better map granularity
I've tried using npm shrinkwrap to force gulp-ng-annotate to use v0.14.0 of ng-annotate.
It seemed to work, so updating the dependency is all that should be needed to fix this in gulp-ng-annotate.
(In my gulpfile I have gulp-ng-annotate taking the output, including sourcemaps, of browserify, and of gulp-strip-ng-log.)
I tested the output with: http://sokra.github.io/source-map-visualization/#custom
For anyone interested my npm-shrinkwrap.json looks like:
{
"name": "my-project-name",
"version": "0.0.1",
"dependencies": {
"gulp-ng-annotate": {
"version": "0.3.6",
"dependencies": {
"gulp-util": {
"version": "2.2.20"
},
"merge": {
"version": "1.2.0"
},
"ng-annotate": {
"version": "0.14.0"
},
"through2": {
"version": "0.4.2"
},
"vinyl-sourcemaps-apply": {
"version": "0.1.4"
}
}
}
}
}note, I had to include all dependencies of gulp-ng-annotate otherwise it would leave out any missing dependencies.
@ntdaley, thanks for your investigation. I've updated ng-annotate version and also applied some fixes since sourcemap option of ng-annotate now is an object not a boolean.
I've published gulp-ng-annotate 0.4.0 to npm. @OverZealous, @ntdaley, would you like to test?
I'm seeing some differences between using gulp-ng-annotate 0.4.0 and 0.3.6 with npm-shrinkwrap.
I've tried a few variations of the configuration, but...
0.4.0 is adding an (unwanted) extra file to the sources and sourcesContent of the sourcemap.
My task for building scripts looks like:
return browserify({
//require.gen.js is generated by a previous task that uses gulp-require-angular
entries : ['./.tmp/require.gen.js'],
debug : true
})
.bundle()
.pipe(source('app.js'))
.pipe(buffer())
.pipe(sourcemaps.init({loadMaps : true}))
.pipe(stripNgLog())
.pipe(ngAnnotate({
sourceMap : true,
add : true
}))
.pipe(sourcemaps.write('.'))
.pipe(gulp.dest('build'));Again I've been using this site to analyse the resulting sourcemaps:
http://sokra.github.io/source-map-visualization/#custom-choose
When I use gulp-ng-annotate 0.3.6 with ng-annotate 0.14.0, then the resulting source map has the original files in sources and sourcesContext.
When I use gulp-ng-annotate 0.4.0, then the resulting source map has an extra file 'app.js' with the file content as it was passed into gulp-ng-annotate. This extra source mostly has no mappings using it, and shouldn't be there because it is an intermediate file.
I don't know, but I suspect this is happening because of these lines added in index.js (line 30-32) 76be559
if (file.path) {
opts.sourcemap.inFile = file.relative;
}
This is working perfectly for me, I'm now seeing the original sources before 6to5 or ng-annotate in Firefox.
Thanks!
@ntdaley You need to pass sourcemap options object to ng-annotate (not sourceMap), see Library API.
Though this shouldn't make any sense since you are using gulp-sourcemaps and ng-annotate's sourcemap option will be automatically enabled.
Could you please simplify your build configuration and provide minimal example reproducing the problem? Also it would be helpful to provide more debug info using gulp-tap plugin for example (the most interesting info is file.sourceMap value at different steps of the build).
inFile option is needed for ng-annotate in order to correctly generate the mappings so we can combine the sourcemaps via vinyl-sourcemaps-apply. (See here for more details.) Otherwise the default source.js value will be used. We use ng-annotate via the API and there is no other way to say it the name of the input file.
There is possibility that something goes wrong because of the unusual build/tools settings since the sourcemap support in tools for now is very unstable.
Hey, I'm the author of the ng-annotate sourcemaps feature. The extra intermediate source file being added to the sourcemaps is an unfortunate side effect of sourcemap combination, and fixing it is not completely trivial (sorry @ntdaley, your hypothesis was a bit too optimistic!)
Technical details: how sourcemap combination works is, each mapping from the NEW sourcemap is traced backwards through the OLD sourcemap to see where it came from. (This is backwards from how I intuitively understood sourcemaps, but that's how the algorithm works, so I've learned to Deal With It). So, if there are any mappings in the new sourcemap that do not have a corresponding mapping in the input sourcemap (due to a lower granularity of mapping in the input sourcemap), then you end up with these orphaned mappings that can only map against the intermediate file. One possible way to fix that would be to purge these mappings as a post-processing step. I have written a 100% unverified-in-production transform to do that. It feels like a hack though, so I hope there is a better way to do it. Suggestions welcome.
I seem to be having the same problem as described by @OverZealous with this code:
return gulp.src([appRoot + '**/*.ts'])
.pipe(sourcemaps.init())
.pipe(typescript())
.pipe(ngAnnotate())
.pipe(sourcemaps.write("./", {addComment: true, includeContent: false, sourceRoot: '/../source'}))
.pipe(gulp.dest(deployRoot + 'scripts'));
The resulting js.map file seems to be referring incorrectly:
{"version":3,"sources":["services/contacts-service.js"]...
instead of
{"version":3,"sources":["/source/contacts-service.ts"]
Regardless, even if I do rename the path in the map file, I can see in the visualizer mentioned above, that the sourcemap is incorrect.
I'm using version 0.4.0 of ngAnnotate, so the fix described by @ntdaley should be in, it seems. I can try to supply a full malfunctioning github repository sample, if helps.
I can try to supply a full malfunctioning github repository sample, if helps.
Yes, I think it will help. It's good to have full-featured example of the problem.
Added here: https://github.com/vitalybe/typescript-annotate-bug
To reproduce:
- npm install
- gulp
You can see that the bug reproduces for .ts files that are nested inside a directory, in the reproduction there are 3 files:
- app\greeter3.ts
- app\controllers\greeter.ts
- app\controllers2\greeter2.ts
For the first one the source map, in deploy folder is created correctly, for greeter3.ts, for the other 2, however, it is as I wrote above - The sourcemap seems to be referring to the .js file, for some reason, e.g: {"version":3,"sources":["controllers2/greeter2.js"]...
I've also added a task gulp concat. That puts everything in a single file, the results are the same, however, greeter3.js is correct, the others are not.
I think I found it, currently still debugging, but this line seems to suspiciously broken:
var sourceMap = JSON.parse(res.map);
sourceMap.file = file.relative.replace(/\\/g, '');
This causes the file:
It turns file.relative value of "controllers\greeter.js" to: controllersgreeter.js
Hm, yes, it might cause problems on Windows. It was added in the #8 pull request.
Could you please change it to file.relative.replace(/\\/g, '/'); and check again?
Didn't seem to do anything :( Same result. Will have to continue debugging.
Do you think it is a Windows problem? Does it reproduce for you?
Yes, I can reproduce it on Linux.
So the problem seems to be because of the different formats of file and sources sourcemaps properties in gulp-typescript and gulp-ng-annotate (see here for details).
gulp-typescript sets only basename of the path to file property but other gulp plugins tends to use file.relative value instead (e.g. gulp-coffee and gulp-ng-annotate).
So gulp-typescript sets file: 'greeter.js' (but sources: ['controllers/greeter.js']), gulp-ng-annotate sets file: 'controllers/greeter.js' sources: ['controllers/greeter.js'] for intermediate sourcemap and finally vinyl-sourcemaps-apply is executed but it can't properly merge it.
With this patch for index.js of gulp-ng-annotate:
diff --git a/index.js b/index.js
index 1cecba8..e67fe81 100644
--- a/index.js
+++ b/index.js
@@ -1,5 +1,6 @@
"use strict";
+var path = require("path");
var gutil = require("gulp-util");
var through = require("through2");
var ngAnnotate = require("ng-annotate");
@@ -28,7 +29,7 @@ module.exports = function (options) {
opts.sourcemap = {};
}
if (file.path) {
- opts.sourcemap.inFile = file.relative;
+ opts.sourcemap.inFile = path.basename(file.path);
}
}
@@ -45,7 +46,7 @@ module.exports = function (options) {
if (opts.sourcemap && file.sourceMap) {
var sourceMap = JSON.parse(res.map);
- sourceMap.file = file.relative.replace(/\\/g, '');
+ sourceMap.file = path.basename(file.path);
applySourceMap(file, sourceMap);
}or this:
diff --git a/index.js b/index.js
index 1cecba8..d4a9f97 100644
--- a/index.js
+++ b/index.js
@@ -45,7 +45,8 @@ module.exports = function (options) {
if (opts.sourcemap && file.sourceMap) {
var sourceMap = JSON.parse(res.map);
- sourceMap.file = file.relative.replace(/\\/g, '');
+ file.sourceMap.file = file.relative;
+ sourceMap.file = file.relative;
applySourceMap(file, sourceMap);
}your example seems to be working. But I'm not sure is it correct and will it break interaction of gulp-ng-annotate with other plugins. Maybe it's gulp-typescript should set file.relative to the file property. Needs additional investigation.
Though I applied the second fix with a warning and published as 0.4.1. @vitalybe, could you test it, please?
There is still another problem which @ntdaley and @smrq was talked about: if I replace gulp-typescript with gulp-coffee in the typescript-annotate-bug example repo then I get duplicate files in the sourcemap sources option.
@Kagami Regarding the duplicates - I think I experienced them, yes. However, they seemed a rather minor issue - They clutter the sources view in chrome, but don't produce any errors by themselves - Unless explicitly opened.
Thank you for the fix, I will try it tomorrow and will let you know. I am not sure about compatibility with other plugins - While debugging I was comparing ng-gulp-annotate with ng-gulp-uglify for reference, since it works with vinyl-sourcemaps-apply and if I use it with the aforementioned sample, it works fine. Not sure it helps, but it seems to be that the logic should be the same for both.
Tested now, still doesn't work 😞
The resulting source maps of greeter2.js and greeter.js still refer to the js file in the sourcemap, e.g:
{"version":3,"sources":["controllers2/greeter2.js"],
NOW it seems like a Windows thing :) Fixing
It would be nice if we could mute these messages - http://www.screencast.com/t/p0WzZKMl
@amcdnl yeap, good catch. I've just published gulp-ng-annotate 0.5.1 with gulpWarnings option. If you set it to false this warning won't be shown anymore.
Thanks!
Also it was a minor bug with relative path comparision on Windows caused by #14. So in your case warnings shouldn't be shown anymore at all.
I don't know if the problem I'm seeing is related to this issue, but when I add gulp-ng-annotate to my gulp build after gulp-typescript, the resulting map includes filenames with both .ts and .js extensions, effectively duplicating every source file. E.g., {"version":3,"sources":["app.ts","app.js"]... Without gulp-ng-annotate, it contains only .ts extensions, as it should.
It looks like @vitalybe's sample repo exhibits the same behavior. If it's not related to this issue, should I open a new one, or is this just down to something I'm doing wrong?
btw I'm on a Mac, so it's not a Windows path thing.
@wilsonjackson please open a new issue. I should have done that previous time (because @vitalybe issue was irrelevent to this one, to be honest), but forgot about it.