Issue with sass-embedded when the system runs out of resources
VioletFlare opened this issue · comments
Bug report
Prior to this bug report, I've opened an issue on the sass-embedded board and it was suggested that it might be related to how sass-loader handles sass-embedded.
sass/embedded-host-node#296 (comment)
I am using sass-embedded alongside sass-loader in my webpack config.
I've noticed, that when running out of memory and physical memory the dart processes throws errors and stop.
Actual Behavior
Webpack doesn't crash and exit, but there are dart errors printed out and the compilation stops.
webpack --config webpack.configs.dev.js --watch
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
../../runtime/bin/eventhandler_win.cc: 318: error: Failed to start read file thread 22
Expected Behavior
It was suggested that it might be due to sass-loader spawning many instances of sass-embedded, maybe sass-loader should be more conservative when using resources?
How Do We Reproduce?
This is my sass-loader config:
{
loader: 'sass-loader',
options: {
sassOptions: {
sourceMap: true,
includePaths: [path.resolve(__dirname, 'Develop', 'Styles'), path.resolve(__dirname, 'WebComponents')],
},
implementation: require('sass-embedded')
}
}
I run two sass-loader instances in parallel to compile two sections of a pretty big codebase of sass.
To reproduce this problem it is necessary to exceed the system memory, in my case I went above 6-8 GB out of 16 GB. It is also necessary to not have much physical memory available. I had something near 500-250 MB.
Please paste the results of npx webpack-cli info here, and mention other relevant information
I am running windows and using:
webpack 5.91.0
sass-loader 14.2.1
sass-embedded 1.77.0
npx webpack-cli info:
System:
OS: Windows 10 10.0.19045
CPU: (4) x64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
Memory: 7.08 GB / 15.88 GB
Binaries:
Node: 20.11.1 - C:\_MYSPACE\nodejs\node.EXE
npm: 10.2.4 - C:\_MYSPACE\nodejs\npm.CMD
Browsers:
Edge: Chromium (124.0.2478.67)
Internet Explorer: 11.0.19041.3636
Packages:
assets-webpack-plugin: ^7.1.1 => 7.1.1
css-loader: ^6.10.0 => 6.11.0
esbuild-loader: ^4.1.0 => 4.1.0
fork-ts-checker-webpack-plugin: ^9.0.2 => 9.0.2
postcss-loader: ^8.1.1 => 8.1.1
resolve-url-loader: ^5.0.0 => 5.0.0
sass-loader: ^14.2.1 => 14.2.1
terser-webpack-plugin: ^5.3.10 => 5.3.10
ts-loader: ^9.5.1 => 9.5.1
webpack: ^5.90.3 => 5.91.0
webpack-cli: ^5.1.4 => 5.1.4
webpack-merge: ^5.10.0 => 5.10.0
webpack-remove-empty-scripts: ^1.0.4 => 1.0.4
Should be fixed by #1199, please update sass-loader to the latest stable versiob, i.e. 14.2.1, if it will not fix your problem need to wait an answer from sass-embedded, so feel free to feedback
According to my webpack-cli info, I believe I have 14.2.1 installed.
That aside, I have a question,
Looking at the commit there's a comment that says it should handle multi-threading more gracefully.
Does multi-threading mean the way of using webpack when the module exports multiple configurations?
If that's it it's how I am currently using webpack for running multiple configurations.
hm, do you want to share the one instance between processes?
hm, do you want to share the one instance between processes?
I believe it is beneficial for the speed of the build to have some parallel processes running, unless it uses too many resources. I was curious about what does it mean to run sass-loader in a multithreading way. In my previous comment, I said that I am running multiple configurations in parallel, these configurations are exported and then passed to webpack as described in webpack documentation.
https://webpack.js.org/configuration/configuration-types/#exporting-multiple-configurations
Each of these configurations have sass-loader enabled. Does it mean I am already running it in a multithreaded way?
hm, I am afraid it was not easy, we need to close sass compiler, so for each webpack compier we need to create one instance, I can improve the api option (make it a function, so you can manually create one compiler and share it), also perhaps it is worth raising the issue of why a lot of resources will be needed there in sass-embedded repo