[v0.6.3]windows d2.exe -w is crashed.

Question

[v0.6.3]windows d2.exe -w is crashed.

bo-ku-ra opened this issue 6 months ago · comments

d2.exe -w "sandbox.d2" "sandbox.svg"

'detected change' occurs frequently.
i did not save 'sandbox.d2'. (only the first time)
'd2.exe' sometimes crashed.

�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 103.344ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 125.8546ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 95.0012ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 107.1851ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 102.4472ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 147.4784ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 88.3592ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 153.6542ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 87.7296ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
.
.
.
.
.
.
panic: runtime error: index out of range [0] with length 0

goroutine 13 [running]:
oss.terrastruct.com/d2/d2cli.(*watcher).watchLoop(0xc00007a1e0, {0x282ff88, 0xc000707c70})
        oss.terrastruct.com/d2/d2cli/watch.go:297 +0xb7e
oss.terrastruct.com/d2/d2cli.(*watcher).goFunc.func1()
        oss.terrastruct.com/d2/d2cli/watch.go:203 +0x76
created by oss.terrastruct.com/d2/d2cli.(*watcher).goFunc
        oss.terrastruct.com/d2/d2cli/watch.go:199 +0x8d

bokura · Answer 1 · Fri Dec 08 2023 19:27:02 GMT+0800 (China Standard Time)

�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 507.888ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
�[32msuccess�[0m: successfully compiled sandbox.d2 to sandbox.svg in 582.9534ms
�[34minfo�[0m: broadcasting update to 1 client
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
panic: runtime error: index out of range [0] with length 0

goroutine 7 [running]:
oss.terrastruct.com/d2/d2cli.(*watcher).watchLoop(0xc0005d8000, {0x1e3ff88, 0xc0000d2050})
        oss.terrastruct.com/d2/d2cli/watch.go:297 +0xb7e
oss.terrastruct.com/d2/d2cli.(*watcher).goFunc.func1()
        oss.terrastruct.com/d2/d2cli/watch.go:203 +0x76
created by oss.terrastruct.com/d2/d2cli.(*watcher).goFunc
        oss.terrastruct.com/d2/d2cli/watch.go:199 +0x8d

Alexander Wang · Answer 2 · Sat Dec 09 2023 03:00:48 GMT+0800 (China Standard Time)

Strange. Can't reproduce but I've made an attempted fix on inspection of logic. We'll followup with 0.6.3 shortly.

Are you sure the file wasn't auto-saving on keystrokes or something? E.g. if you type into the file, close it, and then re-open it, those aren't saved?

Alexander Wang · Answer 3 · Sat Dec 09 2023 03:05:52 GMT+0800 (China Standard Time)

reopening until confirmed fix

bokura · Answer 4 · Sat Dec 09 2023 07:46:36 GMT+0800 (China Standard Time)

i do not encounter v0.6.1.
v0.6.2 occurs on both win 10 and 11. so i can not use v0.6.2.

>editor.exe "sandbox.d2"

>editor.exe (exit)
the editor saves "sandbox.d2". (1 byte + CRLF = a\n)

>d2.exe -w "sandbox.d2"

(Wait for a while) <- happens!

bokura · Answer 5 · Sun Dec 17 2023 08:44:12 GMT+0800 (China Standard Time)

d2 v0.6.3 (windows)

�[34minfo�[0m: broadcasting update to 0 clients
�[34minfo�[0m: detected change in sandbox.svg: recompiling...
panic: runtime error: index out of range [0] with length 0

goroutine 12 [running]:
oss.terrastruct.com/d2/d2cli.(*watcher).watchLoop(0xc0007ca1e0, {0x1d34068, 0xc0006b7a40})
        oss.terrastruct.com/d2/d2cli/watch.go:298 +0xb38
oss.terrastruct.com/d2/d2cli.(*watcher).goFunc.func1()
        oss.terrastruct.com/d2/d2cli/watch.go:203 +0x76
created by oss.terrastruct.com/d2/d2cli.(*watcher).goFunc
        oss.terrastruct.com/d2/d2cli/watch.go:199 +0x8d

bokura · Answer 6 · Sun Dec 17 2023 15:22:03 GMT+0800 (China Standard Time)

it was changed watch.go when '--target' was added in v0.6.2.
is it relevant?

b0cbd66#diff-fc70cf71bc274a18a06914bfd17d043d0cdc7795896c3f1abf9597d0340c0ef5

artiga033 · Answer 7 · Wed Dec 27 2023 14:05:41 GMT+0800 (China Standard Time)

I've also encountered this in v0.6.2 & 0.6.3 while v0.6.1 is fine. I did not even open any editor and just ran d2 -w myfile.d2 in the command line. Several seconds later it will say 'detected change' and recompile endlessly.

Melandel · Answer 8 · Wed Jan 31 2024 02:57:43 GMT+0800 (China Standard Time)

Hi, chiming in.

I can confirm I also have this behavior In version 0.6.3 and 0.6.2, but it works in 0.6.1

Logs:

debug: using theme Neutral default (ID: 0)
success: listening on http://127.0.0.1:61566
info: compiling sandbox.d2...
debug: using layout plugin dagre (bundled)
success: successfully compiled sandbox.d2 to sandbox.svg in 48.6529ms
info: broadcasting update to 0 clients
success: GET / 200 446B 0s
success: GET /static/watch.css 200 206B 30.7101ms
success: GET /static/watch.js 200 2,661B 44.1008ms
success: GET /static/favicon.ico 200 1,150B 0s
success: GET /watch 101 0B 0s

# At this point I'm going to do one action: saving the file (without changing it beforehand)

debug: received file system event REMOVE        "some\path\sandbox.d2"
debug: received file system event WRITE         "some\path\sandbox.d2"
info: detected change in sandbox.d2: recompiling...
debug: using layout plugin dagre (bundled)
success: successfully compiled sandbox.d2 to sandbox.svg in 51.1195ms
info: broadcasting update to 1 client
debug: using layout plugin dagre (bundled)
debug: received file system event WRITE         "some\path\sandbox.svg"
success: successfully compiled sandbox.d2 to sandbox.svg in 45.6776ms
debug: received file system event WRITE         "some\path\sandbox.svg"
info: broadcasting update to 1 client
debug: received file system event WRITE         "some\path\sandbox.svg"
info: detected change in sandbox.svg: recompiling...
debug: using layout plugin dagre (bundled)
debug: received file system event WRITE         "some\path\sandbox.svg"
success: successfully compiled sandbox.d2 to sandbox.svg in 43.8889ms
debug: received file system event WRITE         "some\path\sandbox.svg"
info: broadcasting update to 1 client
info: detected change in sandbox.svg: recompiling...
debug: using layout plugin dagre (bundled)
debug: received file system event WRITE         "some\path\sandbox.svg"
success: successfully compiled sandbox.d2 to sandbox.svg in 45.5243ms
debug: received file system event WRITE         "some\path\sandbox.svg"
info: broadcasting update to 1 client
info: detected change in sandbox.svg: recompiling...
debug: using layout plugin dagre (bundled)
debug: received file system event WRITE         "some\path\sandbox.svg"
success: successfully compiled sandbox.d2 to sandbox.svg in 43.6988ms
debug: received file system event WRITE         "some\path\sandbox.svg"
info: broadcasting update to 1 client
info: detected change in sandbox.svg: recompiling...
debug: using layout plugin dagre (bundled)
debug: received file system event WRITE         "some\path\sandbox.svg"
success: successfully compiled sandbox.d2 to sandbox.svg in 43.8911ms
debug: received file system event WRITE         "some\path\sandbox.svg"
info: broadcasting update to 1 client
info: detected change in sandbox.svg: recompiling...

garymevs · Answer 9 · Thu Feb 01 2024 07:03:47 GMT+0800 (China Standard Time)

Could be a red herring, but the fsnotify watcher WatchList() function appears to be returning a different path depending on platform:

Windows - debug: watch list: [C:\Users\garymevs\Documents\tempgit\d2]
Debian - debug: watch list: [/home/debian/personal/d2/test.d2]

(Both results are from throwing a debug print in the pollTicker case of watch loop and running with "go run . -w test.d2 -d")

Looks like at that point d2 is essentially watching for any change in the parent folder. Instead of the specified watch file.
The d2 instance is then starting a new compile on all WRITE events even if it's an output .svg.

Update:
The above is reproducible in a new project with just fsnotify:

package main

import (
	"fmt"

	"github.com/fsnotify/fsnotify"
)

func main() {
	w, _ := fsnotify.NewWatcher()
	w.Add("C:/path/to/file.go")
	fmt.Printf("watch list: %v", w.WatchList())
}

Prints: watch list: [C:\path\to]

Martin Lercher · Answer 10 · Wed Feb 21 2024 22:13:35 GMT+0800 (China Standard Time)

Had a look into the fsnotify code for Windows. IMO the WatchList function is not implemented correctly according to its documentation as it indeed only lists the directories. However, the watcher associates a map of the filenames Add-ed with the directory, and when an event needs to be handled the map watch.names is checked at https://github.com/fsnotify/fsnotify/blob/v1.7.0/backend_windows.go#L573 w.sendEvent(filepath.Join(watch.path, name), watch.names[name]&sysFSIGNORED). I didn't try it, so, this finding is only based of my understanding of the code.

In conclusion, I guess, the "The above is reproducible in a new project with just fsnotify" code needs a bit more elaboration: i.e. change file.go and change a different file and see what events are received. Currently I don't see fsnotify as the reason for unintentionally monitoring changes in the output file on Windows.