joeljeske / karma-parallel

A Karma JS Framework to support sharding tests to run in parallel across multiple browsers

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Aggregated Log Output

eloisetaylor5693 opened this issue · comments

I'm submitting a ...

  • bug report
  • feature request

What is the current behavior?
When using the brief reporter, none of the output is aggregated. It showed the output for 1 of the 6 exectuters, not all

I tried the option: aggregatedReporterTest: [(reporter)=>regex=/coverage|brief|istanbul|junit/i]

And other permutations of the reporter name

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

What is the expected behavior?
All output should be aggregated, so that I can see the end result for ALL instances

Please tell us about your environment:

  • version:
    "karma-parallel": "^0.3.1",
    "karma-brief-reporter": "0.0.7",

  • Browser: [ Chrome XX ]

  • Language: [TypeScript X.X | ES6/7 ]
    Node v10.15.3
    NPM v6.4.1

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

Karma Config I was using

module.exports = function (config) {
    config.set({
        basePath: "",
        frameworks: ["parallel", "jasmine", "@angular-devkit/build-angular"],
        plugins: [
            require("karma-jasmine"),
            require("karma-chrome-launcher"),
            require("karma-brief-reporter"),
            require("@angular-devkit/build-angular/plugins/karma"),
            require("karma-parallel")
        ],
        reporters: ["brief"],
        port: 1234,
        colors: true,
        logLevel: config.LOG_WARN,
        autoWatch: true,
        browsers: ["ChromeHeadless"],
        singleRun: false,

        parallelOptions: {
            executors: (Math.ceil(require('os').cpus().length / 2)),
            shardStrategy: 'round-robin',
            aggregatedReporterTest: [(reporter)=>regex=/coverage|brief|istanbul|junit/i]
        },

        briefReporter: {
            suppressBrowserLogs: true
        }
    });
};

You are setting the option incorrectly. It should be a function or regex. Try this

aggregatedReporterTest: /coverage|brief|istanbul|junit/i

That doesn't work either. I get this error:

  NaN pending  10 05 2019 14:13:31.289:ERROR [karma-server]: TypeError: Cannot read property 'toString' of undefined
    at padNumber (C:\git\angularWebApp\node_modules\karma-brief-reporter\lib\util\printers.js:7:21)
    at printStats (C:\git\angularWebApp\node_modules\karma-brief-reporter\lib\util\printers.js:13:19)
    at Object.exports.printProgress (C:\git\angularWebApp\node_modules\karma-brief-reporter\lib\util\printers.js:28:3)
    at Brief.onSpecComplete (C:\git\angularWebApp\node_modules\karma-brief-reporter\lib\brief.js:45:16)
    at reporters.forEach (C:\git\angularWebApp\node_modules\karma-parallel\lib\reporter.js:61:26)
    at Array.forEach (<anonymous>)
    at AggregatedCoverageReporter._.rest (C:\git\angularWebApp\node_modules\karma-parallel\lib\reporter.js:58:15)
    at apply (C:\git\angularWebApp\node_modules\lodash\lodash.js:475:27)
    at C:\git\angularWebApp\node_modules\lodash\lodash.js:6569:16
    at AggregatedCoverageReporter.onSpecComplete (C:\git\angularWebApp\node_modules\karma-parallel\lib\reporter.js:121:7)
    at Server.<anonymous> (C:\git\angularWebApp\node_modules\karma\lib\events.js:40:26)
    at Server.emit (events.js:194:15)
    at Browser.onResult (C:\git\angularWebApp\node_modules\karma\lib\browser.js:164:20)
    at Socket.socket.on (C:\git\angularWebApp\node_modules\karma\lib\browser.js:214:42)
    at Socket.emit (events.js:189:13)
    at C:\git\angularWebApp\node_modules\socket.io\lib\socket.js:528:12
Cannot read property 'success' of undefined
TypeError: Cannot read property 'success' of undefined
    at TestCommand.runSingleTarget (C:\git\angularWebApp\node_modules\@angular\cli\models\packages\angular\cli\models\architect-command.ts:242:21)
    at process._tickCallback (internal/process/next_tick.js:68:7)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! now@0.0.0 test-fast: `ng test --sourceMap=false --karmaConfig=src/karma.conf.fast.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the now@0.0.0 test-fast script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Any other suggestions on how to resolve?

Hi Joel,
I am also looking for help to get the code coverage (aggregated one) using karma-parallel. Kindly share sample karma.conf.js. Thanks in advance!!

commented

We had similar issues using istanbul reporter. I got it working by adjusting lib/reporter.js:

--- a/lib/reporter.js
+++ b/lib/reporter.js
@@ -106,11 +106,8 @@ const AggregatedCoverageReporter = function(injector, logger, config, baseReport
 
     delete aggregate.alias.lastResult._realLastResults[browser.id];
     aggregate.real[browser.id] = UNSTARTED;
-
-    // Call through on the very first browser start
-    if (getStartedBrowserCount(aggregate) === 1) {
-      callThrough('onBrowserStart', aggregate.alias);
-    }
+     
+    callThrough('onBrowserStart', aggregate.alias);
   };
    this.onSpecComplete = function (browser, result) {

would be nice if anybody could explain me the reason for the if-condition.

I ended up adding this getters to the AggegatedBrowserLastResult. No need to comment out the if below. I wonder how best to proceed with the changes.

  get success() {
    return _.sum(
      _.map(this._realLastResults, 'success')
    )
  }
  get skipped() {
    return _.sum(
      _.map(this._realLastResults, 'skipped')
    )
  }

@jo-soft How did you override the lib inside of your project ?

commented

@MikeDabrowski basically creating a local fork and adjusting it (after testing it by adjusting local node_modules). but it look like you figured out something