joeljeske / karma-parallel

A Karma JS Framework to support sharding tests to run in parallel across multiple browsers

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

karma-parallel do not return exit code 1 anymore on too low code coverage

petrce opened this issue · comments

  • I'm submitting a

    • bug report
  • What is the current behavior?
    when I run Istanbul code coverage using:
    ng test --source-map --code-coverage
    tests do not return exit code 1 on code coverage below threshold anymore, so it is hard to detect coverage errors in CI

  • If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

  1. set the threshold for code coverage using Istanbul in karma.conf
    coverageIstanbulReporter: {
      dir: require('path').join(__dirname, '../coverage'),
      reports: ['html', 'lcov', 'text', 'text-summary', 'cobertura'],
      fixWebpackSourcePaths: true,
      thresholds: {
        statements: 100,
        lines: 100,
        branches: 100,
        functions: 100
      }
    },
  1. run ng test --source-map --code-coverage
  • What is the expected behavior?
    ng test --source-map --code-coverage
    tests should return exit code 1 on code coverage below threshold

  • Please tell us about your environment:

  • version: 0.3.1
  • Browser: Chrome Version 72.0.3626.121 (Official Build) (64-bit)
  • Language: TypeScript 3.2.4, Angular 7

workaround in poweshell:

$allOutput = & ng test --source-map --code-coverage 2>&1
echo $allOutput

if ($allOutput -cmatch 'ERROR')
{
    echo $matches
    Write-Error "(FAILED)"
}

What version of karma are you using?

Having the same issue. Using karma 4.0.1 and karma-coverage-istanbul-reporter 2.0.5. Also using the same TypeScript and Angular 7

I have not yet tested with karma v4. Can you try downgrading to v3.x to see if that fixes it?

I’ll try to support v4, just seeing if that’s related.

Just to update, I'm seeing this issue still as well.

angular 8.2.4
angular/cli 8.3.2
karma 4.2.0
karma-parallel 0.3.1
karma-coverage-istanbul-reporter 2.0.6

Could you create a reproduction repo? I don’t use angular nor Istanbul in my normal processes.