Consider putting node_modules directories in named Docker volumes
toolness opened this issue · comments
I was just thinking about #1041 and realized that there may be another solution to its woes, which is to move our node_modules directories into separate Docker volumes, rather than keeping them on the host filesystem.
We actually do this in CALC, but not to work around this bug: we do it because devs sometimes want to give IDEs on their host system access to their node_modules, so that e.g. eslint can show them warnings in their IDE. But this causes problems on OSX/Windows where the host system is darwin/win64 and the docker container is running on Linux, meaning that any native modules in node_modules aren't compiled for the host OS. So we put them in separate Docker volumes to ensure that any native modules compiled for linux stay hidden from the docker host.
In any case, I think the solution would help us work around the symlink problem too, because I think docker volumes on darwin/win64 are kept in the linux VM where the containers live, which would eliminate the possibility of symlinks exploding (they are exploding because of complications in the way that docker maps the windows filesystem to linux).
Update: Now that I think about it, I also wonder if this could improve performance on darwin/win64, since there wouldn't have to be any translation between the host OS filesystem and the docker filesystem. Dunno!
I'm generally in favor, but see two drawbacks, mostly around expectations.
- There'd be a mismatch between what the user sees in their file system and what is happening in the container. We could try addressing that by stubbing the library dirs in the host file system (though that'd remove the IDE benefit).
- We can't (easily) poke at the downloaded libraries. This is a handy debugging feature we'd lose.
Ah yeah, good points. I guess maybe we could document how to do those in the README, so folks have better expectations?
Also, speaking of poking at downloaded libraries, I realized that there's not a well-documented way to do this for our python libraries. I think that the venv is in some directory that starts with a . but it might be nice to document that too, so folks know where to look.