Extra quotes appear in commands
iakoff opened this issue · comments
Description
When '' or '~' or other special symbols in command, Universum wrap them with quotes.
Pay attention in example, that equal commend with bash -c work ok, so LIC files have to exist in root repository folder.
Steps to reproduce
config:
| from universum.configuration_support import Variations, get_project_root
| configs = Variations([
| dict(name='1 ', command=['mkdir','-p','./tmp']),
| dict(name='2 ', command=['cp','./LIC*','./tmp']),
| dict(name='3 ', command=['cp','./LIC*','~/tmp']),
| dict(name='4 ', command=['bash','-c', 'cp ./LIC* ./tmp']),
| ])
Command:
$ python3.7 -u -m universum --p4-client ${P4CLIENT} --p4-force-clean --vcs-type p4
Expected result
- Executing build steps
| 5.1. [ 1/4 ] 1
| | $ /bin/mkdir -p ./tmp
| └ [Success]
|
| 5.2. [ 2/4 ] 2
| $ /bin/cp ./LIC* ./tmp
| └ [Success]
|
| 5.3. [ 3/4 ] 3
| | $ /bin/cp ./LIC* ~/tmp
| └ [Success]
|
| 5.4. [ 4/4 ] 4
| | $ /bin/bash -c 'cp ./LIC* ./tmp'
| └ [Success]
|
└ [Success]
Actual result
- Executing build steps
| 5.1. [ 1/4 ] 1
| | $ /bin/mkdir -p ./tmp
| └ [Success]
|
| 5.2. [ 2/4 ] 2
| | | stderr: /bin/cp: cannot stat './LIC*': No such file or directory
| $ /bin/cp './LIC*' ./tmp
| | Error: Module sh got exit code 1
| └ [Failed]
|
| 5.3. [ 3/4 ] 3
| | $ /bin/cp './LIC*' '~/tmp'
| | stderr: /bin/cp: cannot stat './LIC*': No such file or directory
| | Error: Module sh got exit code 1
| └ [Failed]
|
| 5.4. [ 4/4 ] 4
| | $ /bin/bash -c 'cp ./LIC* ./tmp'
| └ [Success]
|
└ [Success]
Environment
- OS: Ubuntu 16.04.5
- Python 3.7
- Universum 0.18.7
It looks like the issue is that the Universum doesn't process special characters, which normally are processed by the shell.
@iakoff, please confirm it.
If that is the case, we could discuss whether this feature is needed.
No. Universum tries to process something that does not need to be processed. Universum encloses in quotation marks those parts of the command that contain an asterisk or tilde.
Here Universum add additional quotes around ./LIC* :
dict(name='2 ', command=['cp','./LIC*','./tmp']), -----> $ /bin/cp './LIC*' ./tmp
Here Universum add additional quotes around ./LIC* and ~/tmp:
dict(name='3 ', command=['cp','./LIC*','~/tmp']), -----> $ /bin/cp './LIC*' '~/tmp'
As a result, this additional quotes cause error: cannot stat './LIC*': No such file or directory
Last command work correct, just to prove, that files ./LIC* exists, and issue exactly in additional quotes.
dict(name='4 ', command=['bash','-c', 'cp ./LIC* ./tmp']),
Dear @iakoff
Even though Universum itself does not add any quotes to launched commands, the cited quotes were indeed added to log (but only to log).
This behaviour is a feature of a library used for external command execution, that is used to notify user of filename expansion not applied to any arguments, passed directly.
For the reference please see https://github.com/python/cpython/blob/409ce4a09e4f96ca9b251c19f5819205aae9ae34/Lib/shlex.py#L325
When passed through bash, these arguments are processed by shell and therefore expanded before cp command execution, and that is the reason why step '4' in your example succeeds.
Dear @k-dovgan. I observe, that Universum add quotes and this cause failure.
I appreciate you for explanation of internal Universum behavior, but the issue still present.
Please provide solution (examples, or step by step description) how I can resolve this problem.
Thanks in advance.
Well, the only solution sutable for you is to use bash -c 'cp ./LIC* ./tmp' instead of cp ./LIC* ./tmp.