Getting Cannot execute /home/user/.poetry/bin/poetry error when configuring the poetry environment

Question

Getting Cannot execute /home/user/.poetry/bin/poetry error when configuring the poetry environment

michaelwiles opened this issue 3 years ago · comments

Describe the bug
When I setup my poetry environment it correctly auto detects the poetry executable but it says it is not executable. The file is there and it is "executable".

To Reproduce
Steps to reproduce the behavior:

Add environment
Select poetry environment
set correct poetry executable
See error

Expected behavior
That the poetry executable gets correctly set.

Environments (please complete the following information):

IDE: PyCharm Community 2021.1 EAP (same issue with 2020.3)
OS: ubuntu 20.10
Poetry Version [e.g. 1.0.8 ]
Plugin version 1.1.5

Additional context
This is the relevant method AFAICT

fun validatePoetryExecutable(poetryExecutable: @SystemDependent String?): ValidationInfo? {
    val message = if (poetryExecutable.isNullOrBlank()) {
        "Poetry executable is not found"
    }
    else {
        val file = File(poetryExecutable)
        when {
            !file.exists() -> "File ${file.absolutePath} is not found"
            !file.canExecute() || !file.isFile -> "Cannot execute ${file.absolutePath}"
            else -> null
        }
    }

    return message?.let { ValidationInfo(it) }
}

Koudai Aono · Answer 1 · Thu Mar 11 2021 23:11:33 GMT+0800 (China Standard Time)

@michaelwiles
Thank you for creating this issue.
I will investigate the problem.

Michael Wiles · Answer 2 · Fri Mar 12 2021 17:12:40 GMT+0800 (China Standard Time)

@koxudaxi I know its a little off topic but...

When I run with ./gradlew clean runIde it works. I can use the plugin successfully.

But when I do a ./gradlew clean buildPlugin after having commented out the code that causes this problem and then go to pycharm and manually install from file system the "old code" is still present - install from build/distribution.

the other question is how do I log so that I can do trouble shooting?

Currently I've removed the rogue code and am running pycharm and running with the first option so I can work. Thank you for this plugin.

Koudai Aono · Answer 3 · Sat Mar 13 2021 00:48:48 GMT+0800 (China Standard Time)

@michaelwiles

But when I do a ./gradlew clean buildPlugin after having commented out the code that causes this problem and then go to pycharm and manually install from file system the "old code" is still present - install from build/distribution.

I don't know why the behavior. I often run ./grealdew clean without buildPlugin. And delete the build directory 😅

How did you install poetry?
And, Would you please tell me the permission on the executable?

Koudai Aono · Answer 4 · Sat Mar 13 2021 01:02:39 GMT+0800 (China Standard Time)

I tested the plugin on macOS and Windows10.
I may test it on Ubuntu or other distros.

Also, I guess that the condition is valid.
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/File.html#canExecute()
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/File.html#isFile()

Koudai Aono · Answer 5 · Sat Mar 13 2021 01:24:32 GMT+0800 (China Standard Time)

@michaelwiles
Also, This code came from the pipenv integration of PyCharm.
https://github.com/JetBrains/intellij-community/blob/1e7fb15e147145e9ac0526f0bfdb7f028c11fe96/python/src/com/jetbrains/python/sdk/pipenv/pipenv.kt#L124-L135

fun validatePipEnvExecutable(pipEnvExecutable: @SystemDependent String?): ValidationInfo? {
  val message = if (pipEnvExecutable.isNullOrBlank()) {
    PyBundle.message("python.sdk.pipenv.executable.not.found")
  }
  else {
    val file = File(pipEnvExecutable)
    when {
      !file.exists() -> PyBundle.message("python.sdk.file.not.found", file.absolutePath)
      !file.canExecute() || !file.isFile -> PyBundle.message("python.sdk.cannot.execute", file.absolutePath)
      else -> null
    }
  }

Ricardo Amendoeira · Answer 6 · Wed Mar 17 2021 09:54:28 GMT+0800 (China Standard Time)

I'm also getting this error.

Ubuntu 20.10
Poetry 1.1.5, installed poetry with curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - as recommended in the official docs
pyenv global is 3.9.0
PyCharm is version 2020.3.3, installed via snap with classic confinement

I can run /home/ricardo/.poetry/bin/poetry just fine from a shell:

Koudai Aono · Answer 7 · Fri Mar 19 2021 01:47:09 GMT+0800 (China Standard Time)

Thank you for giving me the detail.
I'm installing Ubuntu on my Virtualbox.
But, I can't reproduce the problem.
I will continue working on resolving the problem tomorrow.

Koudai Aono · Answer 8 · Fri Mar 19 2021 01:59:28 GMT+0800 (China Standard Time)

I might have better ignore the check that the file is executable on Linux 🤔

Erol Suleyman · Answer 9 · Mon Mar 22 2021 01:22:03 GMT+0800 (China Standard Time)

Hi, same problem here:

```
Ubuntu 20.04
```

Poetry 1.1.5, installed poetry with curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - as recommended in the official docs

```
pyenv local is 3.9.2, system is 3.8.5
```

PyCharm professional is version 2020.3.4, installed via snap with classic confinement

Koudai Aono · Answer 10 · Mon Mar 22 2021 01:27:24 GMT+0800 (China Standard Time)

@erols
Thank you for the version information.
I'm installing Ubuntu 20.10 into VirtualBox. (I installed Ubuntu 20.04 last week)
I don't use snap to install PyCharm. I will try it.

Erol Suleyman · Answer 11 · Mon Mar 22 2021 01:42:57 GMT+0800 (China Standard Time)

Corrected Pycharm version to 2020.3.4

Koudai Aono · Answer 12 · Mon Mar 22 2021 02:21:23 GMT+0800 (China Standard Time)

Great!! I just reproduced the problem in Ubuntu 20.10.
The problem doesn't happen in Ubuntu 20.04 🤔
I will check the reason tomorrow.

Koudai Aono · Answer 13 · Mon Mar 22 2021 10:33:13 GMT+0800 (China Standard Time)

I just understood why the plugin show Cannot execute .../.poetry/bin/poetry
The plugin executes poetry --version to check executable poetry.
But, The execution is used the system python interpreter. it means pyenv is not used.

Also, I can't run poetry with a system Python interpreter.

Would you please test this command?

$ /usr/bin/python3 `which poetry`

Koudai Aono · Answer 14 · Mon Mar 22 2021 18:20:41 GMT+0800 (China Standard Time)

I guess Poetry should be called by an interpreter which is what you used when installing poetry.
Otherwise, the interpreter may not execute poetry.
Unfortunately, the system interpreter of Ubuntu 20.10 can't execute poetry without additional installing.

pyenv is set in shell settings like .bashrc
But, the plugin creates a process to execute poetry without shell settings.
I'm thinking about how do we execute the poetry. 🤔

Erol Suleyman · Answer 15 · Tue Mar 23 2021 06:43:05 GMT+0800 (China Standard Time)

I just understood why the plugin show Cannot execute .../.poetry/bin/poetry
The plugin executes poetry --version to check executable poetry.
But, The execution is used the system python interpreter. it means pyenv is not used.

Also, I can't run poetry with a system Python interpreter.

Would you please test this command?
$ /usr/bin/python3 `which poetry`

Hi,

I ran this command and got this error:

rols@erols-tux:~$ /usr/bin/python3 `which poetry`
Traceback (most recent call last):
  File "/home/erols/.poetry/bin/poetry", line 17, in <module>
    from poetry.console import main
  File "/home/erols/.poetry/lib/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "/home/erols/.poetry/lib/poetry/console/application.py", line 7, in <module>
    from .commands.about import AboutCommand
  File "/home/erols/.poetry/lib/poetry/console/commands/__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "/home/erols/.poetry/lib/poetry/console/commands/check.py", line 2, in <module>
    from poetry.factory import Factory
  File "/home/erols/.poetry/lib/poetry/factory.py", line 18, in <module>
    from .repositories.pypi_repository import PyPiRepository
  File "/home/erols/.poetry/lib/poetry/repositories/pypi_repository.py", line 33, in <module>
    from ..inspection.info import PackageInfo
  File "/home/erols/.poetry/lib/poetry/inspection/info.py", line 25, in <module>
    from poetry.utils.env import EnvCommandError
  File "/home/erols/.poetry/lib/poetry/utils/env.py", line 20, in <module>
    import packaging.tags
  File "/home/erols/.poetry/lib/poetry/_vendor/py3.8/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
    import distutils.util
ModuleNotFoundError: No module named 'distutils.util'
erols@erols-tux:~$

Erol Suleyman · Answer 16 · Tue Mar 23 2021 06:47:59 GMT+0800 (China Standard Time)

I guess Poetry should be called by an interpreter which is what you used when installing poetry.
Otherwise, the interpreter may not execute poetry.
Unfortunately, the system interpreter of Ubuntu 20.10 can't execute poetry without additional installing.

pyenv is set in shell settings like .bashrc
But, the plugin creates a process to execute poetry without shell settings.
I'm thinking about how do we execute the poetry. thinking

Actually I can't run poetry with system python on Ubuntu 20.04. I get this error:

pyenv: python: command not found

The `python' command exists in these Python versions:
  3.9.2

Note: See 'pyenv help global' for tips on allowing both
      python2 and python3 to be found.

Koudai Aono · Answer 17 · Wed Mar 24 2021 01:24:44 GMT+0800 (China Standard Time)

@erols
Thank you for your report!!

Use System Python to execute Poetry

If you don't use Python2.7 then you can make a symbolic link for python
I found a good package https://packages.ubuntu.com/en/groovy/python-is-python3
Also, you need a pip package to execute poetry with the system's Python3.
https://packages.ubuntu.com/en/groovy/python3-pip

Use pyenv to execute Poetry

I don't know the best way.
The plugin doesn't know what shell is used by a user.
In the first place, the plugin can't know users use pyenv mainly. :(

Erol Suleyman · Answer 18 · Wed Mar 24 2021 04:16:58 GMT+0800 (China Standard Time)

Hi! I did the following:

added a symbolic link from python to python3
disabled pyenv

Executing python correctly runs the system python 3.8.5
However, executing poetry still fails in PyCharm and in bash

Erol Suleyman · Answer 19 · Wed Mar 24 2021 05:32:04 GMT+0800 (China Standard Time)

Hi. I got it working finally. Steps from fresh ubuntu 20.04:

If installed, disable pyenv by commenting out pyenv init in .bashrc
Start new shell
install pip3 sudo apt install python3-pip
pip3 install --upgrade keyrings.alt -> This is the fix
Now install poetry curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 -
edit first line of .poetry/bin/poetry. Change shebang to reference python3 #!/usr/bin/env python3
Now poetry works in the command line
Re-enable pyenv
Check in Pycharm. Using poetry in Add Interpreter now working

Michael Wiles · Answer 20 · Wed Mar 24 2021 13:58:29 GMT+0800 (China Standard Time)

Thank you @koxudaxi - very helpful.

So in a nutshell poetry must be executable as a "system" command - without the local shell being fully hydrated - for this to happen, python must be executable as a "system" command.

For me this means adjusting my poetry script to actively reference the system python shell - which is python3 in my case. I want to always control the local python executable so I don't want python to resolve on the system. Or if poetry does not run via the system python this location can be adjusted to point to the python executable location that does have poetry correctly installed (could be a pyenv location).

And this can be done by creating a new poetry executable crafted for pycharm with the relevant python shell specified on the first line. So my first line of this new poetry executable is #!/usr/bin/env python3

I think what might be helpful is to differentiate between the poetry file not being executable and the poetry file executing but failing to return successfully from 'poetry --version'. In this regard if we can log the output of this call we could send the user to the log messages in pycharm maybe for them to troubleshoot.

You want poetry to run in a small a shell as possible and requiring it to have a virtual environment causes a bit of a chicken and egg problem as poetry is a virtual environment tool...

So in other words, I don't think the plugin can fix this issue. All I think it needs to do is provide better feedback to the user to indicate that although it found the poetry executable it failed to execute it and it needs a working poetry executable in order to run.

Erol Suleyman · Answer 21 · Wed Mar 24 2021 17:31:40 GMT+0800 (China Standard Time)

Hi Michael,

I missed that I had to edit .poetry/bin/poetry. I have updated my steps.

I agree that this cannot be fixed in the Pycharm plugin. Unfortunately the poetry team should make this work out of the box on latest ubuntu flavours, or Ubuntu should come with a python executable and pip installed.

Koudai Aono · Answer 22 · Thu Mar 25 2021 09:46:14 GMT+0800 (China Standard Time)

@erols @michaelwiles
Thank you for your comments.
I agree.

So in other words, I don't think the plugin can fix this issue. All I think it needs to do is provide better feedback to the user to indicate that although it found the poetry executable it failed to execute it and it needs a working poetry executable in order to run.

Users use flavors of different Python interpreters for each user.
The plugin can show the error message of the poetry command.
But, the message may not indicate the problem's direct reason to the user.
I could know the reason by your feedbacks.

Hi, @abn
Would you tell me what your advice on this problem?
We have problems poetry doesn't run with an interpreter of Ubuntu20.10. But, some users usually use pyenv for poetry.
PyCharm plugin can't detect whether users usually use pyenv for poetry. Because the plugin doesn't read shell settings.
Unfortunately, Poetry can't run on the system's python interpreter of Ubuntu20.10. We need to install few packages into a system .#196 (comment)

How should we solve the problem?
Thank you.