Non-blocking Mojo::UserAgent inside Minion

[Flavio Poletti]

• Minion version: 5.09
• Mojolicious version: 7.05
• Perl version: 5.18.1
• Operating system: Linux 3.2.0-4-686-pae #1 SMP Debian 3.2.81-1 i686 GNU/Linux

Steps to reproduce the behavior

The following test script defines a Mojolicious::Lite app that allows to do a few tests with Mojo::UserAgent from within Minion:

#!/usr/bin/env perl
use strict;
use warnings;
use Mojolicious::Lite;
use Data::Dumper;
$Data::Dumper::Indent = 1;

my $uri = 'http://mojolicious.org';

plugin Minion => {SQLite => 'sqlite:test4bug.db'};
app->minion->add_task(poke_block   => \&poke_block);
app->minion->add_task(poke_noblock => \&poke_noblock);

my $allowed_uas = [ua => [qw< app new >]];

get '/minion/:op/:ua' => $allowed_uas => sub {
   my $c         = shift;
   my $operation = $c->stash('op');
   my $uas       = $c->stash('ua');
   $c->minion->enqueue('poke_' . $operation, [$uri, $uas]);
   $c->render(text => "minion $operation/$uas will be performed soon\n");
};

app->start;

sub poke_block {
   my ($job, $uri, $uas) = @_;
   my $log     = $job->app->log;
   my $ua      = $uas eq 'app' ? $job->app->ua : Mojo::UserAgent->new();
   my $tx      = $ua->get($uri);
   my $outcome = log_outcome($log, $tx, "/minion/block/$uas");
   $tx->success ? $job->finish($outcome) : $job->fail($outcome);
} ## end sub poke_block

sub poke_noblock {
   my ($job, $uri, $uas) = @_;
   my $log = $job->app->log;
   my $ua = $uas eq 'app' ? $job->app->ua : Mojo::UserAgent->new();
   $ua->get(
      $uri => sub {
         my ($ua, $tx) = @_;
         my $outcome = log_outcome($log, $tx, "/minion/noblock/$uas");
         $tx->success ? $job->finish($outcome) : $job->fail($outcome);
      }
   );
} ## end sub poke_noblock

sub log_outcome {
   my ($log, $tx, $msg) = @_;
   $msg = defined($msg) ? $msg . ' ' : '';
   my $outcome = $tx->success ? 'SUCCESS!' : 'FAILURE!';
   $log->info($msg . $outcome);
   $log->debug(Dumper $tx->res) unless $tx->success;
   return $outcome;
} ## end sub log_outcome

I started the the worker in one shell and then run the following requests in another:

shell$ ./test4bug-minion.pl get /minion/block/app
[Sun Sep 11 08:40:46 2016] [debug] GET "/minion/block/app"
[Sun Sep 11 08:40:46 2016] [debug] Routing to a callback
[Sun Sep 11 08:40:46 2016] [debug] 200 OK (0.004269s, 234.247/s)
minion block/app will be performed soon
shell$ ./test4bug-minion.pl get /minion/block/new
[Sun Sep 11 08:40:55 2016] [debug] GET "/minion/block/new"
[Sun Sep 11 08:40:55 2016] [debug] Routing to a callback
[Sun Sep 11 08:40:55 2016] [debug] 200 OK (0.002441s, 409.668/s)
minion block/new will be performed soon
shell$ ./test4bug-minion.pl get /minion/noblock/app
[Sun Sep 11 08:41:05 2016] [debug] GET "/minion/noblock/app"
[Sun Sep 11 08:41:05 2016] [debug] Routing to a callback
[Sun Sep 11 08:41:05 2016] [debug] 200 OK (0.002478s, 403.551/s)
minion noblock/app will be performed soon
shell$ ./test4bug-minion.pl get /minion/noblock/new
[Sun Sep 11 08:41:12 2016] [debug] GET "/minion/noblock/new"
[Sun Sep 11 08:41:12 2016] [debug] Routing to a callback
[Sun Sep 11 08:41:12 2016] [debug] 200 OK (0.002429s, 411.692/s)
minion noblock/new will be performed soon

Targets are:

• /minion/block/app blocking request using $job->app->ua
• /minion/block/new blocking request using Mojo::UserAgent->new
• /minion/noblock/app non-blocking request using $job->app->ua
• /minion/noblock/new non-blocking request using Mojo::UserAgent->new

Expected behavior

This is what I expected:

• /minion/block/app prints out a SUCCESS! log message in the Minion worker
• /minion/block/new prints out a SUCCESS! log message in the Minion worker
• /minion/noblock/app prints out a SUCCESS! log message in the Minion worker
• /minion/noblock/new prints out a SUCCESS! log message in the Minion worker

i.e. all requests to eventually succeed.

Actual behavior

This is what I got instead:

• /minion/block/app behaves as expected
• /minion/block/new behaves as expected
• /minion/noblock/app disappears
• /minion/noblock/new fails

Minion worker's log follows:

[Sun Sep 11 08:30:06 2016] [debug] Performing job "34" with task "poke_block" in process 9955
[Sun Sep 11 08:30:06 2016] [info] /minion/block/app SUCCESS!
[Sun Sep 11 08:30:11 2016] [debug] Performing job "35" with task "poke_block" in process 9968
[Sun Sep 11 08:30:11 2016] [info] /minion/block/new SUCCESS!
[Sun Sep 11 08:30:16 2016] [debug] Performing job "36" with task "poke_noblock" in process 9981
[Sun Sep 11 08:30:26 2016] [debug] Performing job "37" with task "poke_noblock" in process 9994
[Sun Sep 11 08:30:26 2016] [info] /minion/noblock/new FAILURE!
[Sun Sep 11 08:30:26 2016] [debug] $VAR1 = bless( {
  'finished' => 2,
  'state' => 'finished',
  'error' => {
    'message' => 'Premature connection close'
  },
  'events' => {},
  'content' => bless( {
    'read' => sub { "DUMMY" },
    'events' => {
      'read' => [
        $VAR1->{'content'}{'read'}
      ]
    },
    'headers' => bless( {
      'headers' => {}
    }, 'Mojo::Headers' )
  }, 'Mojo::Content::Single' )
}, 'Mojo::Message::Response' );

[Sebastian Riedel]

Afraid nothing here looks like a bug to me, you've just forgotten to start the event loop. Please use our official support channels next time. http://mojolicious.org/perldoc#SUPPORT

[Flavio Poletti]

Sorry for posting in the wrong place, I thought app->start would start the event loop.