Windows: Watching multiple builds

Question

Windows: Watching multiple builds

benmerckx opened this issue 4 years ago · comments

The docs say multiple builds in one file should be fine (under "Building multiple products at once"). While building works without issues, running --watch on such a build file currently streams a bunch of errors (whereas a single build watches fine):

Wrote dist/server.js (0B, 66.42ms)
Watching files for changes...
Unhandled promise rejection: Error: ENOENT: no such file or directory, open 'dist\.esbuild.ox0ayt1u7797gfpoxjymrn227jid09s.meta.json'
    at Object.openSync (fs.js:462:3)
    at Object.readFileSync (fs.js:364:35)
    at Be ...\node_modules\estrella\<estrella>\util.js:129:16)
    at y ...\node_modules\estrella\<estrella>\estrella.js:754:23)
    at i ...\node_modules\estrella\<estrella>\watch\watch.ts:52:25)
    at Object.nl ...\node_modules\estrella\<estrella>\watch\watch.ts:99:3)
    at rn ...\node_modules\estrella\<estrella>\estrella.js:768:23)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
PS ...> events.js:292    
      throw er; // Unhandled 'error' event
      ^

Error: EPIPE: broken pipe, write
    at Socket._write (internal/net.js:54:25)
    at doWrite (_stream_writable.js:403:12)
    at writeOrBuffer (_stream_writable.js:387:5)
    at Socket.Writable.write (_stream_writable.js:318:11)
    at Object.write ...\node_modules\typescript\lib\tsc.js:3471:36)
    at Object.onWatchStatusChange ...\node_modules\typescript\lib\tsc.js:88582:24)
    at reportWatchDiagnostic ...\node_modules\typescript\lib\tsc.js:89177:22)
    at Object.createWatchProgram ...\node_modules\typescript\lib\tsc.js:88937:9)
    at createWatchOfConfigFile ...\node_modules\typescript\lib\tsc.js:91126:19)
    at executeCommandLineWorker ...\node_modules\typescript\lib\tsc.js:90939:24)
Emitted 'error' event on Socket instance at:
    at errorOrDestroy (internal/streams/destroy.js:108:12)
    at onwriteError (_stream_writable.js:418:5)
    at onwrite (_stream_writable.js:445:5)
    at Socket._write (internal/net.js:58:14)
    at doWrite (_stream_writable.js:403:12)
    [... lines matching original stack trace ...]
    at reportWatchDiagnostic ...\node_modules\typescript\lib\tsc.js:89177:22) {
  errno: 'EPIPE',
  syscall: 'write',
  code: 'EPIPE'
}
events.js:292
      throw er; // Unhandled 'error' event
      ^

Error: EPIPE: broken pipe, write
    at Socket._write (internal/net.js:54:25)
    at doWrite (_stream_writable.js:403:12)
    at writeOrBuffer (_stream_writable.js:387:5)
    at Socket.Writable.write (_stream_writable.js:318:11)
    at Object.write ...\node_modules\typescript\lib\tsc.js:3471:36)
    at Object.onWatchStatusChange ...\node_modules\typescript\lib\tsc.js:88582:24)
    at reportWatchDiagnostic ...\node_modules\typescript\lib\tsc.js:89177:22)
    at Object.createWatchProgram ...\node_modules\typescript\lib\tsc.js:88937:9)
    at createWatchOfConfigFile ...\node_modules\typescript\lib\tsc.js:91126:19)
    at executeCommandLineWorker ...\node_modules\typescript\lib\tsc.js:90939:24)
Emitted 'error' event on Socket instance at:
    at errorOrDestroy (internal/streams/destroy.js:108:12)
    at onwriteError (_stream_writable.js:418:5)
    at onwrite (_stream_writable.js:445:5)
    at Socket._write (internal/net.js:58:14)
    at doWrite (_stream_writable.js:403:12)
    [... lines matching original stack trace ...]
    at reportWatchDiagnostic ...\node_modules\typescript\lib\tsc.js:89177:22) {
  errno: 'EPIPE',
  syscall: 'write',

Rasmus · Answer 1 · Thu Oct 01 2020 01:07:39 GMT+0800 (China Standard Time)

What version of estrella is this? (try node build.js -estrella-version)
There was an issue just like this that was fixed in 1.2.6 (specific fix c83ba71)

You can also try enabling debug mode of estrella to get some more detailed output by adding -estrella-debug on the command line.

Ben · Answer 2 · Thu Oct 01 2020 16:13:09 GMT+0800 (China Standard Time)

Version is 1.2.8 (latest).

Here's the debug output

C:\projects\foo> node build.js --watch -estrella-debug
[DEBUG] Parsed initial CLI arguments: {
  options: {
    watch: true,
    w: true,
    'estrella-debug': true
  },
  args: []
[DEBUG] invoking esbuild.build() in C:\projects\foo with options: {
  minify: true,
  sourcemap: false,
  color: true,
  outfile: 'public/bundle.js',
  bundle: true,
  entryPoints: [ 'src/index.tsx' ],
  define: { DEBUG: 'false' },
  metafile: 'public\\.esbuild.ypqr4q1n4izf1buwocf1aw2la41437mts.meta.json'
}
[DEBUG] starting tslint with options {
  colors: true,
  quiet: false,
  mode: 'auto',
  watch: true,
  cwd: 'C:\\projects\\foo\\packages\\app',
  clearScreen: undefined,
  srcdir: 'src',
  tsconfigFile: 'C:\\projects\\foo\\packages\\app\\tsconfig.json',
  onRestart: [Function: onRestart]
}
[DEBUG] spawning process C:\projects\foo\node_modules\.bin\tsc.cmd [
  "--noEmit",
  "--pretty",
  "--watch",
  "--project",
  "C:\\projects\\foo\\packages\\app\\tsconfig.json"
]
[DEBUG] awaiting esbuild
[DEBUG] input config {
[DEBUG] invoking esbuild.build() in C:\projects\foo with options: {
  minify: true,
  sourcemap: false,
  color: true,
  outfile: 'dist/server.js',
  platform: 'node',
  target: 'es2016',
  entryPoints: [ 'src/server.ts' ],
  define: { DEBUG: 'false' },
  metafile: 'dist\\.esbuild.ox0ayt1u7797gfpoxjymrn227jid09s.meta.json'
}
[DEBUG] starting tslint with options {
  colors: true,
  quiet: false,
  mode: 'auto',
  watch: true,
  cwd: 'C:\\projects\\foo\\packages\\server',
  clearScreen: undefined,
  srcdir: 'src',
  tsconfigFile: 'C:\\projects\\foo\\packages\\server\\tsconfig.json',
  onRestart: [Function: onRestart]
}
[DEBUG] spawning process C:\projects\foo\node_modules\.bin\tsc.cmd [
  "--noEmit",
  "--pretty",
  "--watch",
  "--project",
  "C:\\projects\\foo\\packages\\server\\tsconfig.json"
]
[DEBUG] awaiting esbuild
Wrote dist/server.js (0B, 87.93ms)
Watching files for changes...
[DEBUG] loading watch.js module
[DEBUG] fswatch started for project#ox0ayt1u7797gfpoxjymrn227jid09s
[DEBUG] loading debug.js module
[DEBUG] loading debug.js module
Unhandled promise rejection: Error: ENOENT: no such file or directory, open 'dist\.esbuild.ox0ayt1u7797gfpoxjymrn227jid09s.meta.json'
    at Object.openSync (fs.js:466:3)
    at Object.readFileSync (fs.js:369:35)
    at Be (C:\projects\foo\node_modules\estrella\<estrella>\util.js:129:16)
    at y (C:\projects\foo\node_modules\estrella\<estrella>\estrella.js:754:23)
    at i (C:\projects\foo\node_modules\estrella\<estrella>\watch\watch.ts:52:25)
    at Object.nl (C:\projects\foo\node_modules\estrella\<estrella>\watch\watch.ts:99:3)
    at rn (C:\projects\foo\node_modules\estrella\<estrella>\estrella.js:768:23)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Both of the meta files exist on disk. I found it's actually the cwd that is not taken into account when loading the meta file, that's why I get the read errors. By the way, in another expirement I've also seen the cwd not being accounted for in the run option.

Rasmus · Answer 3 · Sat Oct 03 2020 08:52:32 GMT+0800 (China Standard Time)

Can you try https://github.com/rsms/estrella/tree/1af2aff987261ec3a36b015820b64699bf0dac95/dist (or git master) ?

Ben · Answer 4 · Mon Oct 05 2020 16:23:30 GMT+0800 (China Standard Time)

That works, thanks! Run is still executed in the original directory though. Not sure if that is intended? In a scenario like this I'd sort of expect run to be scoped to cwd too:

build({
  platform: 'node',
  cwd: 'packages/server',
  entry: 'src/server.ts',
  outfile: 'dist/server.js',
  run: 'node dist/server.js'
})

Ben · Answer 5 · Mon Oct 05 2020 17:45:47 GMT+0800 (China Standard Time)

I spoke too soon. While I can invoke the build with --watch without it erroring out and building just fine, it does not rebuild on file change. As far as I can tell FSWatcher.basedir is only used in the debug logs through _relname, it is not actually passed to chokidar. The files passed are however relative to cwd.

Ben · Answer 6 · Mon Oct 05 2020 17:57:58 GMT+0800 (China Standard Time)

An example setup of what I'm trying to achieve below. Where you might run node build.js in the root to build both packages, but you can easily build a single package in its own directory as well. Each package has its own src and dist folders.

// build.js
require('./packages/a/build.js')
require('./packages/b/build.js')

// packages/a/build.js
const { build } = require('estrella')
build({
  cwd: __dirname,
  entry: 'src/a.ts',
  outfile: 'dist/a.js',
  bundle: true
})

// packages/b/build.js
const { build } = require('estrella')
build({
  cwd: __dirname,
  entry: 'src/b.ts',
  outfile: 'dist/b.js',
  bundle: true
})

Rasmus · Answer 7 · Sun Dec 20 2020 05:27:56 GMT+0800 (China Standard Time)

I'd be happy to accept a PR along with an investigation report and a test. I'm not a native Windows developer (I just fire it up in a VM when I need to test things.)

Rasmus · Answer 8 · Sun Dec 20 2020 07:04:15 GMT+0800 (China Standard Time)

Some of the improvements have been published with v1.3.0

Ben · Answer 9 · Wed Dec 23 2020 00:11:31 GMT+0800 (China Standard Time)

The only actionable problem that was left here was cwd not being passed to chokidar (hence no rebuild on file change).
Edit: forgot to mention: cwd is also not respected in run option

As far as Windows specific issues I did not run into any. I moved my setup to one where I didn't need the cwd option and everything built fine. However some race conditions came up when making fast file edits (read: ctrl+s, ctrl+s, ctrl+s) - presumably not Windows related either.

As mentioned in my previous PR, sadly the existence of the 'aux.ts' file makes further Windows development too much of a pain (can't checkout the repo or build) and even makes git installs via yarn/npm fail. So for the time being I went with a watchexec + esbuild + tsc -w combination. Feel free to close this for now, I hope to revisit eventually :)