[BUG] Fix docker logging on WSL

[Ben Elgar]

The problem is that docker logs don't appear (and probably never have) when using exo in WSL.

I spent some time debugging this with John and my understanding of the problem is as follows:
for logging we have three components that need to communicate: the docker daemon, the exo daemon and the syslog daemon. In practice, the exo daemon and the syslog daemon always reside on the same network interface so we can think of them together as "exo".

On Linux they both run natively on the host. Simple. On MacOS exo runs natively and the docker daemon runs in a VM. In WSL the situation is reversed and the docker daemon runs on the host and exo runs in the WSL VM.

The problem appears to be that UDP packets aren't forwarded from the Windows host, where the docker daemon is running, to the WSL VM in which exo is running. I can't find any documentation about this but I can find multiple people on the internet complaining that this is a limitation of WSL[0].

I can't really see a solution that doesn't involve either swapping away from UDP or giving up on WSL support.

[0] Complaining people:
https://old.reddit.com/r/bashonubuntuonwindows/comments/klxbzv/wsl_win10_udp_data_transfer/ghdvj7z/?context=3
https://old.reddit.com/r/bashonubuntuonwindows/comments/fzauf2/win_ubuntu_wsl2_udp_recieve_issue/
MicrosoftDocs/WSL#950
microsoft/WSL#4825
https://stackoverflow.com/questions/66296434/pod-udp-hostport-not-working-in-docker-desktop

[Brandon Bloom]

Switching from UDP to TCP is problematic because the Docker daemon fails if connections can't be established, and then does not reconnect if the connection is severed. This means that upgrades or other interruptions of the syslog server will cause all logging to stop working. Though, upgrades already run the risk of losing messages with UDP, it would be nice to use something more reliable.

One option is that we can do our own log forwarding process, just like we do for unix processes. Let Docker talk TCP or some other protocol to that, then we forward it along UDP to our syslog server. If we basically just do the simplest possible Syslog forwarder service, we can probably get away with it, but I don't love the idea of extra processes to manage/consider.

There is also some kind of log buffering "dual logging" system in the more recent docker engine. I think we can query/replay recently buffered logs, so we could also create a forwarding system that works that way, instead of overriding the logging driver.

Alternatively, we can explore proper Windows support, abandoning WSL, though I think that might actually be a step backwards for most Windows users, as far as I can understand.

This thread does seem to suggest UDP should work: https://old.reddit.com/r/bashonubuntuonwindows/comments/fzauf2/win_ubuntu_wsl2_udp_recieve_issue/

@BenElgar What do you think?