publiclab / PublicLab.Editor

A general purpose, JS/Bootstrap UI framework for rich text posting. An author-friendly, minimal, mobile/desktop interface for creating blog-like content, designed for PublicLab.org

Home Page:https://publiclab.github.io/PublicLab.Editor/examples/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Intermittent test failures in both Jasmine and Jest

jywarren opened this issue · comments

This could be due to failing to wait long enough for async tests...?

In #669, all of these three tests failed, but then all passed on re-running:

>> Expected 2 to be 20. Error: Expected 2 to be 20.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/history_spec.js:94:40)
>>     at <Jasmine>

Then:

>> Expected undefined not to be undefined. Error: Expected undefined not to be undefined.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/editor_spec.js:14:39)
>>     at <Jasmine>
     Expected <textarea class="ple-textarea form-control">...</textarea> to be undefined. (2)
>> Expected <textarea class="ple-textarea form-control">...</textarea> to be undefined. Error: Expected <textarea class="ple-textarea form-control">...</textarea> to be undefined.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/editor_spec.js:17:37)
>>     at <Jasmine>

And in Jest:

FAIL test/ui-testing/CustomInsert.test.js (5.961 s)
  Custom Insert text
    ✕ Add Custom Insert text in rich text mode (4 ms)

  ● Custom Insert text › Add Custom Insert text in rich text mode

    Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.

      at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)

  ● Custom Insert text › Add Custom Insert text in rich text mode

    Evaluation failed: ReferenceError: $ is not defined

      at __puppeteer_evaluation_script__:1:37
      at ExecutionContext._evaluateInternal (node_modules/puppeteer/lib/ExecutionContext.js:122:13)
      at ExecutionContext.evaluate (node_modules/puppeteer/lib/ExecutionContext.js:48:12)
      at Object.<anonymous> (test/ui-testing/CustomInsert.test.js:10:9)
        -- ASYNC --
      at ExecutionContext.<anonymous> (node_modules/puppeteer/lib/helper.js:111:15)
      at DOMWorld.evaluate (node_modules/puppeteer/lib/DOMWorld.js:112:20)
        -- ASYNC --
      at Frame.<anonymous> (node_modules/puppeteer/lib/helper.js:111:15)
      at Page.evaluate (node_modules/puppeteer/lib/Page.js:860:43)
      at Page.evaluate (node_modules/puppeteer/lib/helper.js:112:23)
      at Object.<anonymous> (test/ui-testing/CustomInsert.test.js:10:20)

(node:2652) UnhandledPromiseRejectionWarning: Error: Navigation failed because browser has disconnected!

OK, now in #672, we see:

log: Failed to load resource: net::ERR_FILE_NOT_FOUND
   ...adds value to 'tags' key of editor.data, instead of overwriting it...X
     Expected '' to be 'cool,rad'. (1)
>> Expected '' to be 'cool,rad'. Error: Expected '' to be 'cool,rad'.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/tags_module_spec.js:32:28)
>>     at <Jasmine>
     Expected 'first,' to be 'first,cool,rad'. (2)
>> Expected 'first,' to be 'first,cool,rad'. Error: Expected 'first,' to be 'first,cool,rad'.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/tags_module_spec.js:36:28)
>>     at <Jasmine>
   - reports key, value, valid......X
     Expected 'first,' to be ''. (1)
>> Expected 'first,' to be ''. Error: Expected 'first,' to be ''.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/tags_module_spec.js:20:28)
>>     at <Jasmine>
     Expected 'first,cool,rad' to be 'cool,rad'. (2)
>> Expected 'first,cool,rad' to be 'cool,rad'. Error: Expected 'first,cool,rad' to be 'cool,rad'.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/tags_module_spec.js:22:28)
>>     at <Jasmine>

   - exists, and has a textarea......X
     Expected undefined not to be undefined. (1)
>> Expected undefined not to be undefined. Error: Expected undefined not to be undefined.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/editor_spec.js:14:39)
>>     at <Jasmine>
     Expected <textarea class="ple-textarea form-control">...</textarea> to be undefined. (2)
>> Expected <textarea class="ple-textarea form-control">...</textarea> to be undefined. Error: Expected <textarea class="ple-textarea form-control">...</textarea> to be undefined.
>>     at <Jasmine>
>>     at UserContext.<anonymous> (file:///home/runner/work/PublicLab.Editor/PublicLab.Editor/spec/javascripts/editor_spec.js:17:37)
>>     at <Jasmine>

#672 passed on re-run. Now #665 has the same errors as #672!

Re-running that too.

My guess is we should extend the async timeout!

await page.setDefaultNavigationTimeout(60000);

This is resolved by #721 now!