Cannot focus specs properly

Question

Cannot focus specs properly

kkempster94 opened this issue 7 years ago · comments

kkempster94 commented 7 years ago

I'm submitting a ...
- bug report
- feature request
- support request => Please do not submit support request here, see note at the top of this template.
Do you want to request a feature or report a bug?
report a bug
What is the current behavior?
focused specs are not respected in every browser instance
Steps to reproduce

focus a "describe" or "it" block in any test

What is the expected behavior?
All running browsers respect the focused spec (probably should just let the one browser with the focused spec run its tests)
What is the motivation / use case for changing the behavior?
Without this functionality, we cannot test subsets of our tests. (there are over 1400 in my particular case)
Please tell us about your environment:

version: 0.1.1
Browser: Chrome 63 (Headless)

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

The image below shows that there are 4 tests focused in one of the browsers running the tests, but the other browsers continue to run their subset of the tests unaffected.

Joel Jeske · Answer 1 · Sat Jan 20 2018 13:22:25 GMT+0800 (China Standard Time)

Great find! This should be fixed in the latest release.

kkempster94 · Answer 2 · Mon Jan 22 2018 23:18:12 GMT+0800 (China Standard Time)

Please tag a release when you get the chance, good sir.

Joel Jeske · Answer 3 · Mon Jan 22 2018 23:35:04 GMT+0800 (China Standard Time)

Oh yes, sorry. Done

kkempster94 · Answer 4 · Sat Jan 27 2018 05:59:25 GMT+0800 (China Standard Time)

looks like I'm running into some odd errors when I try to focus tests:

"message": "Uncaught TypeError: Cannot read property 'parent' of undefined
at c:/someDirectory/node_modules/mocha/mocha.js:737:14

TypeError: Cannot read property 'parent' of undefined
at Object.only (c:/someDirectory/node_modules/mocha/mocha.js:737:14)
at context.it.only (c:/someDirectory/node_modules/mocha/mocha.js:585:26)
at c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:88:12
at Suite. (c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:132:47)
at Suite. (c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:65:13)
at Object.create (c:/someDirectory/node_modules/mocha/mocha.js:717:19)
at context.describe.context.context (c:/someDirectory/node_modules/mocha/mocha.js:532:27)
at c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:88:12
at Object.beforeStartup (c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:131:13)
at ContextKarma.karma.start (c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:16:23)",
"str": "Uncaught TypeError: Cannot read property 'parent' of undefined
at c:/someDirectory/node_modules/mocha/mocha.js:737:14

TypeError: Cannot read property 'parent' of undefined
at Object.only (c:/someDirectory/node_modules/mocha/mocha.js:737:14)
at context.it.only (c:/someDirectory/node_modules/mocha/mocha.js:585:26)
at c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:88:12
at Suite. (c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:132:47)
at Suite. (c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:65:13)
at Object.create (c:/someDirectory/node_modules/mocha/mocha.js:717:19)
at context.describe.context.context (c:/someDirectory/node_modules/mocha/mocha.js:532:27)
at c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:88:12
at Object.beforeStartup (c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:131:13)
at ContextKarma.karma.start `(c:/someDirectory/node_modules/karma-parallel/lib/karma-parallelizer.js:16:23)"

It looks like the number of these errors directly corresponds with n - 1 where n is the number of browser instances. I would assume that is because the browser instance running the non-fake test which is focused is not erroring, while all the other instances with fake tests are.

kkempster94 · Answer 5 · Sat Feb 10 2018 04:57:17 GMT+0800 (China Standard Time)

bumping this issue in case you didn't notice it. Also, the code throwing this error is: https://github.com/mochajs/mocha/blob/f01f66ec19249d90280c81142b87b01a29f5b80b/lib/interfaces/common.js#L132.

Joel Jeske · Answer 6 · Sat Feb 10 2018 07:51:52 GMT+0800 (China Standard Time)

Thanks, yea I didn’t see it. I’ll be able to look at it on Tuesday or so.

James Fox · Answer 7 · Sun Feb 11 2018 07:04:51 GMT+0800 (China Standard Time)

I believe my tests are failing with the same issue. We call this.timeout in many tests, and the this context is incorrect in the n+1 cases (1 executor works fine)

Joel Jeske · Answer 8 · Sun Feb 11 2018 08:43:20 GMT+0800 (China Standard Time)

Interesting, thanks for the tip. Are you calling that in a before() or it() block? Or maybe inside the describe directly?

Could you send a small example of a failure?

James Fox · Answer 9 · Mon Feb 12 2018 03:34:09 GMT+0800 (China Standard Time)

We commonly call it inside the beginning of the root describe in a test file:

describe('some_module/some_component', function() {
  this.timeout(10000);

  // Setup and run some tests
  ...
});

Here is an example output:

➜  myapp git:(karma_parallel) grunt test:myapp:unit
Running "test:myapp:unit" (test) task

Running "karma:myapp:unit" (karma) task

START:
Hash: e0bdc8cc97632b01d813
Version: webpack 2.4.1
Time: 44ms
webpack: Compiled successfully.
webpack: Compiling...
Hash: e0fab60a728ee52c6d84
Version: webpack 2.4.1
Time: 21798ms
                     Asset     Size  Chunks                    Chunk Names
               0.bundle.js   273 kB       0  [emitted]  [big]  extras_bundle
               1.bundle.js   110 kB       1  [emitted]         stub_data_bundle
             firstparty.js  6.97 MB       2  [emitted]  [big]  firstparty
src/js/tests/myapp/main.js  1.79 MB       3  [emitted]  [big]  src/js/tests/myapp/main.js
                 vendor.js  8.22 MB       4  [emitted]  [big]  vendor

webpack: Compiled with warnings.
Chrome 63.0.3239 (Mac OS X 10.12.6) ERROR
  Uncaught TypeError: this.timeout is not a function
  at src/js/tests/myapp/main.js:287
Chrome 63.0.3239 (Mac OS X 10.12.6) ERROR
  Uncaught TypeError: this.timeout is not a function
  at src/js/tests/myapp/main.js:356

Finished in 2.413 secs / 0 secs

SUMMARY:
✔ 0 tests completed
Warning: Task "karma:myapp:unit" failed. Use --force to continue.

Aborted due to warnings.

And the relevant portion of the karma config:

  frameworks: ['parallel', 'mocha-debug', 'mocha'],

    parallelOptions: {
      executors: 2, // Defaults to cpu-count - 1
      shardStrategy: 'round-robin',
    },

Also I realized I was wrong about which ones fail. Not just n+1 fails, they all do.

Hope this helps!

Joel Jeske · Answer 10 · Wed Feb 14 2018 09:38:26 GMT+0800 (China Standard Time)

@kkempster94 Sorry for the delay, I have tracked down your issue and it should be fixed in the latest release, 0.2.3. It was a mocha-specific issue and I did not catch it before. I finally took the time it implement a test setup for this project that tests various configurations including mocha so that mocha should remain stable. Thanks for the patience.

Joel Jeske · Answer 11 · Wed Feb 14 2018 09:39:13 GMT+0800 (China Standard Time)

@jamesopti Your issue is not the same as @kkempster94 and will require a some thought and a different approach. Would you mind opening a new issue?