Package Selection Broken
dehrk opened this issue · comments
Describe the bug
I am getting an array to string conversion error when attempting to build a specific package.
To Reproduce
Specify a package for the build command.
Outcome
Loading config file /home/dkinne000/.composer/auth.json
Checked CA file /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem: valid
Executing command (/var/www/cdxoc/html/satis): git branch --no-color --no-abbrev -v
Executing command (/var/www/cdxoc/html/satis): git describe --exact-match --tags
Executing command (/var/www/cdxoc/html/satis): git log --pretty="%H" -n1 HEAD
Executing command (/var/www/cdxoc/html/satis): hg branch
Executing command (/var/www/cdxoc/html/satis): fossil branch list
Executing command (/var/www/cdxoc/html/satis): fossil tag list
Executing command (/var/www/cdxoc/html/satis): svn info --xml
Failed to initialize global composer: Composer could not find the config file: /home/dkinne000/.composer/composer.json
To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
Reading /var/www/cdxoc/html/satis/vendor/composer/installed.json
Scanning packages
In PackageSelection.php line 182:
[ErrorException]
Array to string conversion
Exception trace:
at /var/www/cdxoc/html/satis/src/PackageSelection/PackageSelection.php:182
Composer\Util\ErrorHandler::handle() at n/a:n/a
sprintf() at /var/www/cdxoc/html/satis/src/PackageSelection/PackageSelection.php:182
Composer\Satis\PackageSelection\PackageSelection->select() at /var/www/cdxoc/html/satis/src/Console/Command/BuildCommand.php:183
Composer\Satis\Console\Command\BuildCommand->execute() at /var/www/cdxoc/html/satis/vendor/symfony/console/Command/Command.php:255
Symfony\Component\Console\Command\Command->run() at /var/www/cdxoc/html/satis/vendor/symfony/console/Application.php:934
Symfony\Component\Console\Application->doRunCommand() at /var/www/cdxoc/html/satis/vendor/symfony/console/Application.php:273
Symfony\Component\Console\Application->doRun() at /var/www/cdxoc/html/satis/src/Console/Application.php:49
Composer\Satis\Console\Application->doRun() at /var/www/cdxoc/html/satis/vendor/symfony/console/Application.php:149
Symfony\Component\Console\Application->run() at /var/www/cdxoc/html/satis/bin/satis:26
build [--repository-url [REPOSITORY-URL]] [--repository-strict] [--no-html-output] [--skip-errors] [--stats] [--] [<file> [<output-dir> [<packages>...]]]
Expected behavior
I expect Satis to reprocess the specified package.
Additional context
According to git bisect, the issue started with a July 10th commit:
commit 969905e4ef3217cf51319a24beca5e56d607c649
Author: Pierre <pierre.basile@gmail.com>
Date: Wed Jul 10 16:37:42 2019 +0200
Update only one package / Add using documentation
:040000 040000 e628dba5982aa296d8b6fdf6110e5840f18518f4 5db09b717cc3644f5fda268eaef9dd97f874373c M docs
:040000 040000 c71d67350e3fa975eaf9db22e2cddb5344a541b3 8b54082f022873c58f7176a7efc8c50510ff707a M src
What version of Satis are you running?
Because your trace does not match up with current master.
I just did a git pull and a composer update in my testing environment, and I'm still getting the same error.
$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: composer.lock
no changes added to commit (use "git add" and/or "git commit -a")
$ git remote -v
composer https://github.com/composer/satis.git (fetch)
composer https://github.com/composer/satis.git (push)
origin https://github.com/composer/satis.git (fetch)
origin git@github.com:composer/satis.git (push)
$ cd /var/www/cdxoc/html/satis && php bin/satis build -vvvn /var/www/cdxoc/html/packages/satis.json /var/www/cdxoc/html/packages cdoss/coremvc
Loading config file /home/dkinne000/.composer/config.json
Loading config file /home/dkinne000/.composer/auth.json
Executing command (/app/www/cdxoc_dev/html/satis): git branch --no-color --no-abbrev -v
Reading /home/dkinne000/.composer/composer.json
Loading config file /home/dkinne000/.composer/config.json
Loading config file /home/dkinne000/.composer/auth.json
Loading config file /home/dkinne000/.composer/composer.json
Loading config file /home/dkinne000/.composer/auth.json
Reading /home/dkinne000/.composer/auth.json
Reading /app/www/cdxoc_dev/html/satis/vendor/composer/installed.json
Reading /home/dkinne000/.composer/vendor/composer/installed.json
Scanning packages
In PackageSelection.php line 166:
[ErrorException]
Array to string conversion
Exception trace:
at /app/www/cdxoc_dev/html/satis/src/PackageSelection/PackageSelection.php:166
Composer\Util\ErrorHandler::handle() at n/a:n/a
sprintf() at /app/www/cdxoc_dev/html/satis/src/PackageSelection/PackageSelection.php:166
Composer\Satis\PackageSelection\PackageSelection->select() at /app/www/cdxoc_dev/html/satis/src/Console/Command/BuildCommand.php:184
Composer\Satis\Console\Command\BuildCommand->execute() at /app/www/cdxoc_dev/html/satis/vendor/symfony/console/Command/Command.php:255
Symfony\Component\Console\Command\Command->run() at /app/www/cdxoc_dev/html/satis/vendor/symfony/console/Application.php:934
Symfony\Component\Console\Application->doRunCommand() at /app/www/cdxoc_dev/html/satis/vendor/symfony/console/Application.php:273
Symfony\Component\Console\Application->doRun() at /app/www/cdxoc_dev/html/satis/src/Console/Application.php:49
Composer\Satis\Console\Application->doRun() at /app/www/cdxoc_dev/html/satis/vendor/symfony/console/Application.php:149
Symfony\Component\Console\Application->run() at /app/www/cdxoc_dev/html/satis/bin/satis:26
build [--repository-url [REPOSITORY-URL]] [--repository-strict] [--no-html-output] [--skip-errors] [--stats] [--] [<file> [<output-dir> [<packages>...]]]
Different line though, something I can work with
Oh, can you include your satis.json ?
I just pushed a commit that might fix your issue. If you update your clone of satis, can you give it another shot?
Package selection seems similarly broken for me. In an older version of the code, I could execute a command like:
... build satis.json /web foo/barThis command would only build the foo/bar package rather than everything. However, now it looks like using this parameter now requires that foo/bar be set as a name key on a VCS repository definition.
I'm not entirely familiar with the Satis codebase, so I'm fumbling my way through it to figure this out. My theory is that the logic in PackageSelection.php::filterPackages() and PackageSelection.php::select() has become incorrect after recent changes in this class.
It looks like the set of repositories have repository filters applied first, and then package filters are applied, and then whatever is left is built. The filterPackages() method is looking for a match between the <packages> command line parameter and repository name keys. So if you don't have any name keys defined on VCS repositories, no repository will every match anything you list on the command-line. The resulting empty list of repositories causes the "Could not find any package(s) matching: foo/bar" error.
I believe this has fixed the problem for me, in PackageSelection.php:
private function filterPackages(array $repositories): array
{
$packages = $this->packagesFilter;
return array_filter(
$repositories,
function ($repository) use ($packages) {
if (!($repository instanceof ConfigurableRepositoryInterface)) {
return false;
}
$config = $repository->getRepoConfig();
+ foreach ($packages as $package) {
+ if (count($repository->findPackages($package))) {
+ return true;
+ }
+ }
+
if (!isset($config['name']) || !in_array($config['name'], $packages)) {
return false;
}
return true;
}
);
}However, I'm not sure if there are any unwanted side effects of this change or if there might be a more appropriate solution.
In my case satis is local mirror for packagist packages and partial update completely broken:
+ php -d memory_limit=768m -d allow_url_fopen=true vendor/bin/satis build satis.json var/web/ psr/log
Scanning packages
In PackageSelection.php line 168:
Could not find any package(s) matching: psr/log
build [--repository-url [REPOSITORY-URL]] [--repository-strict] [--no-html-output] [--skip-errors] [--stats] [--] [<file> [<output-dir> [<packages>...]]]
Cannot reproduce without a satis.json configuration.
Here is minimal satis.json for reproduce
{
"name": "my composer",
"homepage": "http://localhost",
"repositories": [
{
"type": "composer",
"url": "https://packagist.org"
}
],
"require": {
"psr/log": ">=1.0.0",
"myclabs/php-enum": "^1.4 || ^1.5 || ^1.6"
},
"require-all": false,
"require-dependencies": true,
"require-dev-dependencies": false,
"archive": {
"directory": "dist",
"format": "tar",
"prefix-url": "http://localhost",
"skip-dev": true
}
}
I cannot reproduce this. Are you running the latest version?
yes, on master branch (688aa41)
i guess that problem is in PackageSelection::filterPackages method -- it filter repository by config parameter 'name', but packgist repository has no this parameter
Like I said, I cannot reproduce this. With your satis.json example I can just run the build command and it works fine.
yes, build all packages work fine, but run build with package selection will fail
[adbrvn@adbrvn ~/src/my-satis]$ php -d allow_url_fopen=true vendor/bin/satis build satis.json var/web/ psr/log -vvv
Loading config file /home/adbrvn/.composer/config.json
Loading config file /home/adbrvn/.composer/auth.json
Checked CA file /etc/ssl/certs/ca-certificates.crt: valid
Deprecation warning: Your package name my composer is invalid, it should have a vendor name, a forward slash, and a package name. The vendor and package name can be words separated by -, . or _. The complete name should match "[a-z0-9]([_.-]?[a-z0-9]+)*/[a-z0-9]([_.-]?[a-z0-9]+)*". Make sure you fix this as Composer 2.0 will error.
Executing command (/home/adbrvn/src/my-satis): git branch --no-color --no-abbrev -v
Failed to initialize global composer: Composer could not find the config file: /home/adbrvn/.composer/composer.json
To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
Reading /home/adbrvn/src/my-satis/vendor/composer/installed.json
Scanning packages
In PackageSelection.php line 168:
[InvalidArgumentException]
Could not find any package(s) matching: psr/log
Exception trace:
at /home/adbrvn/src/my-satis/vendor/composer/satis/src/PackageSelection/PackageSelection.php:168
Composer\Satis\PackageSelection\PackageSelection->select() at /home/adbrvn/src/my-satis/vendor/composer/satis/src/Console/Command/BuildCommand.php:184
Composer\Satis\Console\Command\BuildCommand->execute() at /home/adbrvn/src/my-satis/vendor/symfony/console/Command/Command.php:255
Symfony\Component\Console\Command\Command->run() at /home/adbrvn/src/my-satis/vendor/symfony/console/Application.php:912
Symfony\Component\Console\Application->doRunCommand() at /home/adbrvn/src/my-satis/vendor/symfony/console/Application.php:264
Symfony\Component\Console\Application->doRun() at /home/adbrvn/src/my-satis/vendor/composer/satis/src/Console/Application.php:49
Composer\Satis\Console\Application->doRun() at /home/adbrvn/src/my-satis/vendor/symfony/console/Application.php:140
Symfony\Component\Console\Application->run() at /home/adbrvn/src/my-satis/vendor/composer/satis/bin/satis:26
build [--repository-url [REPOSITORY-URL]] [--repository-strict] [--no-html-output] [--skip-errors] [--stats] [--] [<file> [<output-dir> [<packages>...]]]
may be it bad command for run build?
Oh, I missed that, my bad.
I came to this thread for the same issue, but discovered another bug along with package selection.
So yeah the initial problem is that you need to specify the name parameter to your repository declarations for the filtering to work by default (I ran into this issue too) - I suppose the fix mentioned in the comments above means this is not needed, I do not think everyone goes adding a name manually when it could be fetched from the repositories themselves.
Like so for the filtering to work:
{ "name": "aws/aws-sdk-php", "type": "git", "url": "https://github.com/aws/aws-sdk-php.git" },
However, the logic at the moment means you cannot specify more than 1 package name for partial updates cause it will match > 1 on line 174 of PackageSelection.
This is expected if you have more than 1 package specified that it will be > 1, seems like this was to solve some issue where you have specified more than one vcs entry with the same name but was applied more broadly. Currently this means you can not do partial updates on more than 1 package, seems like a bug cause the documentation says you should be able to.
The current check of the following appears to be incorrect if you have multiple packages you want to specify to do partial updates for:
https://github.com/composer/satis/blob/master/src/PackageSelection/PackageSelection.php#L174
if (count($repos) > 1) {
throw new \InvalidArgumentException(sprintf(
'Found more than one package matching: %s',
implode(', ', $this->packagesFilter)
));
}
There should be a check on each repo to ensure they have a unique name - but is this really a problem I wonder? You may genuinely have different repositories that have the same name - guess you can get around this limitation by just having a unique name.