guzzle / guzzle

Guzzle, an extensible PHP HTTP client

Home Page:https://docs.guzzlephp.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

endless yield problem

xuxujie opened this issue · comments

I am want to write a pool process by guzzle. code like this:

$requests = function($list) {
while(true){
$some = Redis::bLpop($list, 10);
if(!$some) continue;
[$key, $payload] = $some;
$array = json_decode($payload, true);
// likes [$url, $headers, $body] = $array;
yield new Request(...$array);
}
};
$pool = new Pool($client, $requests(['high','low']), ['concurrency' => 5, .......]);
$pool->promise()->wait();

when I push 10 requests to redis list 'high', but only got 6 responses.
And I push another 10 requests again, then I got 16 responses.
how can i fix this problem?

This is not an issue, but here I give you an idea:

<?php

use GuzzleHttp\ClientInterface;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request;
use Psr\Http\Message\ResponseInterface;

class GeneratePool
{

    private ClientInterface $httpClient;
    private array $list;


    public function execute()
    {
        // populate $this->list ...
        $poolConfig = [
            'fulfilled' => function (ResponseInterface $response, $index): void {
                $this->fulfilled($response, $index);
            },
            'rejected' => function (Exception $reason, $index): void {
                $this->rejected($reason, $index);
            },
            'concurrency' => 50,
            'options' => [
                'connect_timeout' => 15,
                'exceptions' => true,
                'timeout' => 30,
            ],
        ];

        $pool = new Pool($this->httpClient, $this->generateRequests($this->list), $poolConfig);
        $promise = $pool->promise();
        $promise->wait();
    }

    private function generateRequests(): \Generator
    {
        foreach ($this->list as $index => $item) {
            // create the request and set a known index
            yield $index => new Request('GET', 'url', []);
        }
    }

    /**
     * @param ResponseInterface $response a successful response
     * @param $index the index set at the yield
     * @return void
     */
    private function fulfilled(ResponseInterface $response, $index): void
    {
        // this is a successful response from the Request yield a generateRequests
        // you have the response and do whatever you need to do with the item

        $item = $this->list[$index];
    }

    /**
     * @param Exception $reason this is an error response
     * @param $index the index set at the yield
     * @return void
     */
    private function rejected(\Exception $reason, $index): void
    {
        // $reason you might want to check reason->hasResponse() depends on the version of Guzzle you are working on
        // you have the response and do whatever you need to do with the item

        $item = $this->list[$index];
    }
}