pappasam / coc-jedi

tl;dr: --system-site-packages option needs to be removed when creating the virtualenv in the createJlsVenvPosix() function.

OS: Ubuntu 20.04
Python3: 3.8.10

I had a few issues trying to get the extension working. This is how I managed to fix it. This issue may contain behaviours caused by multiple root causes.

What I first did was to CocInstall coc-jedi in Vim. This completed without any issues. Then I closed and opened Vim again, navigated to a Python file, and was presented with the error message described in #1

[coc.nvim]: UnhandledRejection: Launching server "jedi" using command jedi-language-server failed.

I thought that I had to install jedi-language-server manually on the system-wide Python path, so I ran pip3 install jedi-language-server. Reopening Vim, and opening a Python file gave me an error when coc-jedi tried to detect jedi-language-server was not installed, tried to create a virtualenv and install the package in the venv (

coc-jedi/src/jlsExecutable.ts

Lines 45 to 49 in d58e031

    
           execSync( 
        
             'python3 -m venv ' + 
        
               `--system-site-packages --clear ${pathVenv} && ` + 
        
               `${pathPip} install -U pip ${JLS_NAME}==${JLS_VERSION}` 
        
           )

) . Investigating this I found that pip was not in the virtualenv .venv created in the coc-jedi extension ($HOME/.config/coc/node_modules/coc-jedi/.venv/bin/). When trying the command manually, Python3 gave me the following output. Apparently, python3-venv is needed in Debian systems for this to work.

 python3 -m venv --system-site-packages --clear /tmp/venvtest                                          
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.8-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/tmp/venvtest/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

So after successfully running sudo apt install python3-venv I see that jedi-language-server was successfully installed, but there's another error when the binary was invoked. Checking $HOME/.config/coc/node_modules/coc-jedi/.venv/bin/ I saw that jedi-language-server has been installed in system-wide site_packages path.

So I removed the .venv directory created by the plugin manually, created a virtualenv without the --system-site-packages option, sourced it, and pip installed JLS which got the binary in the place the plugin is looking for. It's working now.

I'll open a PR with this fix soon, but I'm not entirely sure if it's an issue with my setup, which isn't that specific, or something that should actually be fixed.

Opened PR #49 related to this.

Just noticed that this may be a regression caused by the latest fixes on #47

@chamilad thanks for opening this issue! Hopefully we can get it resolved and document any necessary work-arounds more clearly.

Would you please provide the error message you get after installing python3-venv? Based on my understanding, all --system-site-packages should do is add a line to pyvenv.cfg at the root of your virtualenv: include-system-site-packages = true. This is necessary because it gives users the ability to access site-packages they've installed in their main Python site-packages (eg, outside of their virtualenv).

My guess is that something more subtle is happening, and I'd appreciate your help in figuring out what that is!

For venv's relevant source code, see: https://github.com/python/cpython/blob/612e59b53f0c730ce1b881f7c08dc6d49f02c123/Lib/venv/__init__.py#L170-L180

I think this issue is related to this: pypa/pip#1408

Hi @pappasam , sorry I lost my terminal history that day so I couldn't post the output after installing python3-venv. However I remember the issue was because of the following line of code.

coc-jedi/src/jlsExecutable.ts

Line 30 in b8bb4e6

const pathJls = path.join(pathVenv, 'bin', JLS_NAME)

With --system-site-packages in place, and because I installed jedi-language-server on the system wide Python path, a pip3 install jedi-language-server inside the virtual environment just completes without installing, since it's available on the system wide Python packages. With that, when we go for <venv_root>/bin/jedi-language-server a file not found is thrown.

May be the approach here should be to check first if something like which jedi-language-server resolves or not, and only install and assume <venv_root>/bin as the prefix if it returns a non-zero value.

I'd be glad to help here, and I'll try to open a PR on my suggested fix. Probably this weekend, if no urgent issues are opened because of this.

@chamilad I believe this issue is resolved? Also, we already provide an initialization option for users to configure an existing jedi-language-server installation (which is how I test the language server in development): https://github.com/pappasam/coc-jedi#jediexecutablecommand

Also, I resolved the original issue this this already-released commit: 95a2f45. Now, executables won't be placed outside of virtual environments for coc-jedi-managed installations, but we still benefit from access to system packages.

I see. I forgot about the external JLS exec command option. All good.

	execSync(
	'python3 -m venv ' +
	`--system-site-packages --clear ${pathVenv} && ` +
	`${pathPip} install -U pip ${JLS_NAME}==${JLS_VERSION}`
	)

Issues when opening Python files for the first time