Combination of component and e2e test seems random. vite-plugin-istanbul
hamidmayeli opened this issue · comments
Logs and screenshots
I won't be able to add logs as it's over GitHub limits
Versions
- What is this plugin's version?
^3.11.0
- What is Cypress version?
^12.17.2
- What is your operating system? Windows 10 x64 / Linux Ubuntu
- What is the shell?
- What is the Node version?
20.3.1
- What is the NPM version?
9.7.2
- How do you instrument your application? Using
vite-plugin-istanbul
verionn^5.0.0
. - When running tests, if you open the web application in regular browser, and open DevTools, do you see
window.__coverage__
object? Yes - Is there
.nyc_output
folder? Is there.nyc_output/out.json
file. Is it empty? It's not empty and has +31K lines content. - Do you have any custom NYC settings in
package.json
(nyc
object) or in other [NYC config files] Yesistanbul({ include: 'src/*', exclude: ['node_modules'], extension: [ '.js', '.ts', '.jsx', '.tsx', ], requireEnv: !process.env.CYPRESS, checkProd: true, }),
- Do you run Cypress tests in a Docker container? Yes
Describe the bug
I am trying to combine the unit test coverage and e2e. But the coverage is different in different environments. I have tested this in 3 different environments.
- My Windows (Win 10 x64) machine, which normally shows 5 uncovered branches. FYI, I delete
.nyc
andcoverage
folder before running the commands. - A docker file which I use to test with. This one show 3 uncovered branches.
- The pipeline (GitHub Actions) which build the same docker file. This one show 8 uncovered branches.
I have RUN npm run components:ci && npm run e2e:ci && npm run build
in my docker file and run the same commands in the same order in windows too. However, if I run the e2e
before components
I will end up with 32 uncovered branches.
I am testing a page using component test which results in 100% code coverage. When I run the e2e (to test the routing to this page), it will be reduced to just over 80%.
Link to the repo
Sadly, I cannot make the project public, and if I remove the codes/tests the issue with go away. But the setup can be find in this template. But, I will be able to give access to a contributor.
package.json
{
"name": "my-vite-app",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "concurrently --kill-others \"npx json-server -w json-server/data.cjs -r json-server/routing.json -m json-server/middleware.cjs --host 127.0.0.1 -p 4000\" \"vite\"",
"build": "tsc && vite build",
"lint": "eslint src --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
"preview": "vite preview",
"integration": "start-server-and-test dev \"http://localhost:5173|http://127.0.0.1:4000\" \"cypress open --e2e\"",
"integration:ci": "start-server-and-test dev \"http://localhost:5173|http://127.0.0.1:4000\" \"cypress run --e2e\"",
"unit": "cypress open --component",
"unit:ci": "cypress run --component"
},
"dependencies": {
"jalaali-js": "^1.2.6",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-easy-crop": "^5.0.0",
"react-router-dom": "^6.14.2"
},
"devDependencies": {
"@cypress/code-coverage": "^3.11.0",
"@faker-js/faker": "^8.0.2",
"@types/jalaali-js": "^1.1.1",
"@types/react": "^18.2.16",
"@types/react-dom": "^18.2.7",
"@typescript-eslint/eslint-plugin": "^6.2.0",
"@typescript-eslint/parser": "^6.2.0",
"@vitejs/plugin-react": "^4.0.3",
"autoprefixer": "^10.4.14",
"concurrently": "^8.2.0",
"cypress": "^12.17.2",
"cypress-file-upload": "^5.0.8",
"eslint": "^8.45.0",
"eslint-plugin-react-hooks": "^4.6.0",
"eslint-plugin-react-refresh": "^0.4.1",
"json-server": "^0.17.3",
"postcss": "^8.4.27",
"start-server-and-test": "^2.0.0",
"tailwindcss": "^3.3.3",
"typescript": "^5.0.2",
"vite": "^4.4.7",
"vite-plugin-istanbul": "^5.0.0",
"vite-plugin-pwa": "^0.16.4",
"vite-tsconfig-paths": "^4.2.0"
}
}
vite.ts
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react';
import tsconfigPaths from 'vite-tsconfig-paths';
import istanbul from 'vite-plugin-istanbul';
import { VitePWA } from 'vite-plugin-pwa'
// https://vitejs.dev/config/
export default defineConfig({
build: {
sourcemap: true,
},
plugins: [
react(),
tsconfigPaths(),
istanbul({
include: 'src/*',
exclude: ['node_modules'],
extension: [ '.js', '.ts', '.jsx', '.tsx', ],
requireEnv: !process.env.CYPRESS,
checkProd: true,
}),
VitePWA({
registerType: 'autoUpdate',
injectRegister: 'auto',
includeAssets: ['favicon.ico', 'apple-touch-icon.png', 'mask-icon.svg'],
manifestFilename: "manifest.json",
manifest: {
name: '*****',
short_name: '****',
description: '*****',
theme_color: '#ffffff',
icons: [
{
src: 'logo192.png',
sizes: '192x192',
type: 'image/png'
},
{
src: 'logo512.png',
sizes: '512x512',
type: 'image/png'
}
]
}
}),
],
})
cypress.config.ts
import { defineConfig } from "cypress";
import registerCodeCoverageTasks from "@cypress/code-coverage/task";
export default defineConfig({
e2e: {
screenshotOnRunFailure: false,
video: false,
retries: {
runMode: 2,
},
baseUrl: "http://localhost:5173",
setupNodeEvents(on, config) {
registerCodeCoverageTasks(on, config);
// implement node event listeners here
return config;
},
},
component: {
screenshotOnRunFailure: false,
retries: {
runMode: 2,
},
video: false,
devServer: {
framework: "react",
bundler: "vite",
},
setupNodeEvents(on, config) {
registerCodeCoverageTasks(on, config);
// implement node event listeners here
return config;
},
},
});
Dockerfile
FROM node:20-slim as build
RUN apt-get update && apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb jq -y
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run unit:ci && npm run integration:ci && npm run build
RUN echo "Current branch coverage is $(cat ./coverage/coverage-summary.json | jq ".total.branches.total - .total.branches.covered")."
RUN [ $(cat ./coverage/coverage-summary.json | jq ".total.branches.total - .total.branches.covered") -le 4 ] \
|| touch ./coverage/failed.flag
FROM scratch as test-result
COPY --from=build /app/coverage /
FROM scratch as deployable
COPY --from=build /app/dist /
FROM nginx:1.25-alpine-slim as deployment
COPY ./nginx.conf /etc/nginx/conf.d/default.conf
CMD ["/bin/sh", "-c", "nginx -g 'daemon off;'"]
COPY --from=build /app/dist /usr/share/nginx/html
Hi @hamidmayeli issues in the repo are reserved for bugs and feature requests. Please check out the Cypress Discord Community , it can be helpful for debugging or answering questions on how to use Cypress and Cypress plugins.