Path mapping for remote tcp pyls

[Yaroslav Klyuyev]

Usually I prefer to run my python projects in a docker environment, and there I usually have paths that differ from my local paths (sublime is running locally), e.g.: /apps/project1/repo1 <-> /home/USERNAME/work/project1/repo1, and also all the dependencies are not installed locally, and are only available in docker.

I know I can:

1. start pyls in the docker manually
2. use port-mapping to expose server on localhost
3. use tcp_host, tcp_port in sublime-LSP settings,
   ...
4. but paths are still not correct.

Similar workflow can relatively easy be handled in SublimeAnaconda:

{  //project settings
    "folders": [
        {...}
    ],
    "settings": {
        "python_interpreter": "tcp://localhost:19362?pathmap=/home/USERNAME/work/project1/meta-repo/volumes/pipenv/,/apps/other-repo/.venv/&pathmap=/home/USERNAME/work/project1/,/apps/&extra=/apps&extra=/apps/tika/src&interpreter=/apps/other-repo/.venv/bin/python3.6m"
    }
}

Under the hood SublimeAnaconda server running in docker simply replaces matching parts of the filepaths when it receives request from sublime client one way, and then when sends anything with file path back to client, then it replaces parts of path the other way.

There is no problem with "synchronization" because I'm using my local folders as volumes in docker

I'd like to implement something similar for LSP plugin, but I might need some advises:

1. Is it at all viable idea to try implementing this?
2. "Dumb" approach might be:
   • find all the calls to plugins.core.url.filename_to_uri, and add some extra argument to it (e.g. settings),
   • in function filename_to_uri itself: do some transformations if some setting is set.
   • But I'm not sure what is the best way to get current settings (something like plugin.core.configurations.get_scope_client_config(current_view, ...HOW_TOGET_CONFIGS, WHAT_IS_A_POINT))
   • I don't know where should "backward" transformation be done, and not sure that such place exists at all.
3. Is it better to do on the client (this repo), or maybe it's better to explore server-side changes? So that a pyls server makes all the work.

@tomv564, can you please point me in some direction? And please excuse me for such a verbose text that might not be even relevant to this repo. I was just surprised that modern tool capable of working over TCP either requires to be running on same file-system, or needs to have exactly the same paths. I'm aware that the simplest way would be to place volumes in docker using the same paths I've locally, but that feels kind like a cheating (and also I like to have /apps/ as a main folder in docker regardless of local path)

[Raoul Wols]

I've been thinking about this for a while now. I think both the server (python-language-server) as well as the client (we) must implement this kind of thing: https://github.com/sourcegraph/language-server-protocol/blob/master/extension-files.md, though that seems to be just a proposal.

[Tom van Ommeren]

The files extension @rwols is a clean-ish way of handling it, but every language server would have to commit to using it.

The editor is in a unique position to solve it for all languages, so we could absolutely try it in LSP.

The downside is that filename_to_uri and uri_to_filename can currently be imported and used wherever. Your change would have to introduce the mapping at every call, and it can expect to be vary for every window / project. I think it's a significant effort!

[satlan]

I run ST on windows and clangd on unix. Even without a full extension to the LSP (like suggested above), it would be tremendously useful for the client (the plugin) to implement path rewriting. It would cover a lot of use cases (like mine) when the same filesystem is accessible from the server and the client but with a different root.
For example, I map a drive letter (on windows) to my home directory (on unix) in url.py a re.sub() calls in filename_to_uri and uri_to_filename.

It's straightforward and works correctly across the board. It would be nice to make it a generic setting.

[Raoul Wols]

Note that clangd now supports path-mapping. I still do think an editor is the better place to handle this, but it shows that language servers can also fix this themselves.

[Vincent]

In case someone else finds this and it helps them, the following change to url.py will get WSL2 language servers working:

def filename_to_uri(path: str) -> str:
    path = path.replace('C:', 'mnt/c', 1)
    return urljoin('file:', pathname2url(path))


def uri_to_filename(uri: str) -> str:
    uri = uri.replace('mnt/c', 'C:', 1)
    if os.name == 'nt':
        # url2pathname does not understand %3A (VS Code's encoding forced on all servers :/)
        return url2pathname(urlparse(uri).path).strip('\\')
    else:
        return url2pathname(urlparse(uri).path)

It'll let you edit projects that are on your C: drive but not in WSL, while using a language server running in WSL.

Obviously this also breaks compatibility with any other language server setup so hopefully there'll be an official solution one day.

[Raoul Wols]

With respect to Docker, can't you just map the volume and open the mapped volume in ST?

[Raoul Wols]

There's also plans for virtual file systems now, so this may just get auto-solved in the protocol anyway microsoft/language-server-protocol#1264

Of course pylsp would have to support that as well.

[Raoul Wols]

• local server, local project: works
• local server, remote project: cannot work with ST
• remote server, local project: needs work
• remote server, remote project: cannot work with ST

I think we could get it to work by going with the virtual file system idea. I'll close this now as I don't think we'll implement path mapping.