Error when running all snapshots

Question

Error when running all snapshots

tomitrescak opened this issue 5 years ago · comments

Hi, thanks for the amazing plugin!!!

There is a possible bug, as when I run all tests in the cypress browser I receive following error:

CypressError: cy.task('cypress-plugin-snapshot:matchText') failed with the following error:

> Error: Cannot load snapshot file. File "__snapshots__/__all.snap does not contain valid JSON or javascript

Meinaart van Straalen · Answer 1 · Sun Dec 30 2018 08:56:55 GMT+0800 (China Standard Time)

This was quite hard to fix but I managed it. Probably need to contact Cypress about why Cypress.spec is not correct when running all tests. Or why they don't populate the "file" field in Mocha.

Meinaart van Straalen · Answer 2 · Wed Jan 02 2019 18:10:23 GMT+0800 (China Standard Time)

@tomitrescak can you verify it works?

Meinaart van Straalen · Answer 3 · Fri Jan 11 2019 22:39:20 GMT+0800 (China Standard Time)

I will remove the fix, it was causing other issues and the real reason is a bug in Cypress:
cypress-io/cypress#3090

Ray Farias · Answer 4 · Thu May 02 2019 06:07:38 GMT+0800 (China Standard Time)

IMO this issue should be mentioned on the README as it's a huge blocker for me, I only found out after implementing and writing tests (using .only while writing tests) to find the error in the GUI test runner after I ran all my tests which made all my work previously useless.

Please consider adding an eye-catching warning or 'caveat' section which mentions this issue so that other users of this package will not share the same experience I had or at the very least, know of this important bug.

Hope Cypress gets the fix in soon, :fingers

(added a PR as a suggestion for a new section in the README file: #37)

Gleb Bahmutov · Answer 5 · Wed Jul 24 2019 10:58:05 GMT+0800 (China Standard Time)

So the workaround I can offer for now is to fix Cypress.spec object by looking at the __filename in each spec file. Here is the code to place into cypress/support/index.js. It is a utility function that forms Cypress.spec from passed filename

// cypress/support/index.js
export const fixCypressSpec = filename => () => {
  const path = require('path')
  const relative = filename.substr(1) // removes leading "/"
  const projectRoot = Cypress.config('projectRoot')
  const absolute = path.join(projectRoot, relative)
  Cypress.spec = {
    absolute,
    name: path.basename(filename),
    relative
  }
}

From each spec file (that wants to use snapshots)

import { fixCypressSpec } from '../support'

beforeEach(fixCypressSpec(__filename))

I have been playing with this in https://github.com/bahmutov/objection-example using this approach to run all specs with snapshots

Christian Schulz · Answer 6 · Fri Aug 16 2019 02:28:18 GMT+0800 (China Standard Time)

Seems not to be working if you are using TypeScript.

Meinaart van Straalen · Answer 7 · Fri Aug 30 2019 21:10:27 GMT+0800 (China Standard Time)

Investigate if this would work as a possible fix.

Cypress.mocha.getRunner().suite.ctx.currentTest.title

Scott Paradis · Answer 8 · Wed Jan 29 2020 04:09:39 GMT+0800 (China Standard Time)

Seems not to be working if you are using TypeScript.

+1 @CSchulz

When using typescript, the workaround mentioned by @bahmutov does not work, as the __filename resolves to index.js, which is a result of the fact that typescript is being transpiled to js, so (depending on your setup), it will reference the file in which the ts -> js transformation took place (plugins/index.js, for example)

Georg Malahov · Answer 9 · Sun Feb 02 2020 02:46:34 GMT+0800 (China Standard Time)

Typescript users, just pass the exact spec file path to beforeEach(fixCypressSpec('/cypress/integration/spec.ts')). 👍

FYI @CSchulz @meinaart

Nathan Brown · Answer 10 · Sat Mar 14 2020 00:20:14 GMT+0800 (China Standard Time)

Any light at the end of this tunnel? The suggested fixes don't seem to work for me :(

Dennis Ploeger · Answer 11 · Wed Apr 08 2020 05:54:40 GMT+0800 (China Standard Time)

Especially for Typescript users, it's easier to use a custom command for the workaround like this:

Cypress.Commands.add('fixCypressSpec', (filename) => {
    const path = require('path')
    const relative = filename.substr(1) // removes leading "/"
    const projectRoot = Cypress.config('projectRoot')
    const absolute = path.join(projectRoot, relative)
    Cypress.spec = {
        absolute,
        name: path.basename(filename),
        relative
    }
})

add it to the index.d.ts like this:

/// <reference types="cypress" />

declare namespace Cypress {
  interface Chainable {
    fixCypressSpec(filename: string): void
  }
}

and then in the spec in a beforeEach:

describe('my test', () => {
  beforeEach(() => {
    cy.fixCypressSpec('/cypress/integration/mytest_spec.ts')
(...)

Ivan Skachkov · Answer 12 · Tue Jul 21 2020 20:17:12 GMT+0800 (China Standard Time)

+ if using cypress-cucumber-preprocessor, then one could write a "Given" step like so:

Given('I use snapshots for {string} feature', (fileName) => {
	cy.fixCypressSpec(fileName);
});

And in specs/features that use the snapshotting specify that:
Given I use snapshots for "/cypress/integration/mytest.feature" feature

But still would be nice to avoid all these workarounds ;)

Blaine · Answer 13 · Wed Jul 22 2020 00:24:57 GMT+0800 (China Standard Time)

I put the fix suggested by @bahmutov and it seems to work. But it looks like an error is thrown from getSpec.js,
if (Cypress.spec.absolute === '__all') { throw new Error('cypress-plugin-snapshots does not work when running all tests, this will be fixed once this bug is resolved: https://github.com/cypress-io/cypress/issues/3090'); }

If I comment this out the test works fine. The question is how do you handle this for others on the team? Tell them to npm install then go into node_modules and comment out the offending line? Is anyone else doing something more streamlined?

Winggo Tse · Answer 14 · Thu Aug 13 2020 04:14:46 GMT+0800 (China Standard Time)

I put the fix suggested by @bahmutov and it seems to work. But it looks like an error is thrown from getSpec.js,
if (Cypress.spec.absolute === '__all') { throw new Error('cypress-plugin-snapshots does not work when running all tests, this will be fixed once this bug is resolved: https://github.com/cypress-io/cypress/issues/3090'); }

If I comment this out the test works fine. The question is how do you handle this for others on the team? Tell them to npm install then go into node_modules and comment out the offending line? Is anyone else doing something more streamlined?

I had this problem too. Solved the issue by forking the plugin, modifying the fork by commenting out those lines that throw the error, and using that forked module in package.json.

Kris Raven · Answer 15 · Tue Oct 20 2020 09:51:16 GMT+0800 (China Standard Time)

When I apply @bahmutov's workaround - it works well, thanks for this 😄 however, after running the all the test specs, a folder called "ypress" gets created (ie.
ypress/integration/__snapshots__/myTestSpecName.spec.js.snap). I was wondering if this is correct or is a side effect of the workaround, as it has not been mentioned here.

Tim Dilenyan · Answer 16 · Fri Oct 30 2020 02:15:13 GMT+0800 (China Standard Time)

There is no reason to manually specify the path: cy.fixCypressSpec(__filename);

Niels Boecker · Answer 17 · Mon Nov 16 2020 22:19:02 GMT+0800 (China Standard Time)

Thanks @dploeger and @441N143G, that's really helpful.

Btw, TypeScript users will also want to install @types/node and add it to their tsconfig in order for __filename to work:

{
    ...,
    "types": ["cypress", ..., "@types/node"],
}

Also, @krisraven, I have had the same. I believe the issue here is that you may have provided cypress/integration/mytest_spec.ts instead of /cypress/integration/mytest_spec.ts (no leading slash). This is also the case when using __filename. With this argument, the one line in the workaround which truncates the first character is not required anymore:

  const relative = filename.substr(1); // removes leading "/"

Franck · Answer 18 · Wed Mar 31 2021 18:23:53 GMT+0800 (China Standard Time)

When I apply @bahmutov's workaround - it works well, thanks for this 😄 however, after running the all the test specs, a folder called "ypress" gets created (ie.
ypress/integration/__snapshots__/myTestSpecName.spec.js.snap). I was wondering if this is correct or is a side effect of the workaround, as it has not been mentioned here.

I experienced the same thing, to not have "ypress" folder, you juste have to remove substr(1) ;)

Reed Dadoune · Answer 19 · Fri Jul 30 2021 05:24:18 GMT+0800 (China Standard Time)

This solution works for me universally with TypeScript:

support/snapshotPatch.ts:

import { basename } from 'path';

declare global {
  // eslint-disable-next-line @typescript-eslint/no-namespace
  namespace Cypress {
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    interface Chainable<Subject> {
      /**
       * Patch cypress-plugin-snapshots
       *
       * @returns {void}
       */
      fixCypressSpec(): void;
    }
  }
}

Cypress.Commands.add('fixCypressSpec', function () {
  const { absoluteFile, relativeFile } = this.test.invocationDetails;
  Cypress.spec = {
    ...Cypress.spec,
    absolute: absoluteFile,
    name: basename(absoluteFile),
    relative: relativeFile,
  };
});

support/index.ts:

import './support/snapshotPatch.ts';

beforeEach(() => {
  cy.fixCypressSpec();
});

Sergei Shneider · Answer 20 · Sun Aug 15 2021 22:50:28 GMT+0800 (China Standard Time)

This solution works for me universally with TypeScript:

support/snapshotPatch.ts:

import { basename } from 'path';

declare global {
  // eslint-disable-next-line @typescript-eslint/no-namespace
  namespace Cypress {
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    interface Chainable<Subject> {
      /**
       * Patch cypress-plugin-snapshots
       *
       * @returns {void}
       */
      fixCypressSpec(): void;
    }
  }
}

Cypress.Commands.add('fixCypressSpec', function () {
  const { absoluteFile, relativeFile } = this.test.invocationDetails;
  Cypress.spec = {
    ...Cypress.spec,
    absolute: absoluteFile,
    name: basename(absoluteFile),
    relative: relativeFile,
  };
});

support/index.ts:

import './support/snapshotPatch.ts';

beforeEach(() => {
  cy.fixCypressSpec();
});

if i use it globally it saves all the snapshots as index.ts.snap, but works if i call it as a command within tests, which i feel is preferable anyway

Reed Dadoune · Answer 21 · Wed Aug 18 2021 08:27:07 GMT+0800 (China Standard Time)

@SSDlm Thanks for pointing that issue out! That's a bug I discovered shortly after I posted the above solution. I meant to come back here and update but never did 😖

Michel EDIGHOFFER · Answer 22 · Thu Aug 19 2021 00:26:17 GMT+0800 (China Standard Time)

The maintenance of this package is freezed.. I was added to the project but I didn't have any right for merge PR or update project settings (CICD is broken)..

Ilya Radchenko · Answer 23 · Wed Jan 12 2022 03:33:41 GMT+0800 (China Standard Time)

Could this plugin run the fix (workaround) internally?

Chris Braddock · Answer 24 · Tue Dec 13 2022 03:25:05 GMT+0800 (China Standard Time)

if i use it globally it saves all the snapshots as index.ts.snap, but works if i call it as a command within tests, which i feel is preferable anyway

If you change:

const { absoluteFile, relativeFile } = this.test.invocationDetails;

to:

const { absoluteFile, relativeFile } = this.currentTest.invocationDetails;

it works in a global beforeEach.