endless yield problem
xuxujie opened this issue · comments
xuxujie commented
I am want to write a pool process by guzzle. code like this:
$requests = function($list) {
while(true){
$some = Redis::bLpop($list, 10);
if(!$some) continue;
[$key, $payload] = $some;
$array = json_decode($payload, true);
// likes [$url, $headers, $body] = $array;
yield new Request(...$array);
}
};
$pool = new Pool($client, $requests(['high','low']), ['concurrency' => 5, .......]);
$pool->promise()->wait();
when I push 10 requests to redis list 'high', but only got 6 responses.
And I push another 10 requests again, then I got 16 responses.
how can i fix this problem?
Maikel Salazar commented
This is not an issue, but here I give you an idea:
<?php
use GuzzleHttp\ClientInterface;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request;
use Psr\Http\Message\ResponseInterface;
class GeneratePool
{
private ClientInterface $httpClient;
private array $list;
public function execute()
{
// populate $this->list ...
$poolConfig = [
'fulfilled' => function (ResponseInterface $response, $index): void {
$this->fulfilled($response, $index);
},
'rejected' => function (Exception $reason, $index): void {
$this->rejected($reason, $index);
},
'concurrency' => 50,
'options' => [
'connect_timeout' => 15,
'exceptions' => true,
'timeout' => 30,
],
];
$pool = new Pool($this->httpClient, $this->generateRequests($this->list), $poolConfig);
$promise = $pool->promise();
$promise->wait();
}
private function generateRequests(): \Generator
{
foreach ($this->list as $index => $item) {
// create the request and set a known index
yield $index => new Request('GET', 'url', []);
}
}
/**
* @param ResponseInterface $response a successful response
* @param $index the index set at the yield
* @return void
*/
private function fulfilled(ResponseInterface $response, $index): void
{
// this is a successful response from the Request yield a generateRequests
// you have the response and do whatever you need to do with the item
$item = $this->list[$index];
}
/**
* @param Exception $reason this is an error response
* @param $index the index set at the yield
* @return void
*/
private function rejected(\Exception $reason, $index): void
{
// $reason you might want to check reason->hasResponse() depends on the version of Guzzle you are working on
// you have the response and do whatever you need to do with the item
$item = $this->list[$index];
}
}