t/response.t randomly fails
kentfredric opened this issue · comments
I had this failure occur, and then I responded "retry" to my cpanm prompt and it went away:
# Failed test at t/response.t line 143.
# got: '180.2'
# expected: '180'
# Looks like you failed 1 test of 68.
t/response.t ................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/68 subtests
Test Summary Report
-------------------
t/response.t (Wstat: 256 Tests: 68 Failed: 1)
Failed test: 47
Non-zero exit status: 1
Files=17, Tests=772, 2 wallclock secs ( 0.19 usr 0.04 sys + 2.81 cusr 0.47 csys = 3.51 CPU)]
There's one reproduction on CPAN Testers on Dragonfly BSD with Perl 5.18.4
But I'm failing here on Linux + 5.27.9
Summary of my perl5 (revision 5 version 27 subversion 9) configuration:
Platform:
osname=linux
osvers=4.10.13-gentoo
archname=x86_64-linux
uname='linux katipo2 4.10.13-gentoo #48 smp preempt tue may 2 15:18:23 nzst 2017 x86_64 intel(r) core(tm) i5-2410m cpu @ 2.30ghz genuineintel gnulinux '
config_args='-de -Dprefix=/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot -Ddefault_inc_excludes_dot -Doptimize= -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -Dman1dir=none -Dman3dir=none -Dusedevel -Accflags= -fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -Aldflags= -fstack-protector-strong -fno-stack-protector -Aeval:scriptdir=/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/bin'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
optimize=' -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native'
cppflags='-fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe'
ccversion=''
gccversion='7.3.0'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -fno-stack-protector -L/usr/local/lib'
libpth=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed /usr/lib /usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
libc=libc-2.26.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.26'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DISABLE_PMC
PERL_DONT_CREATE_GVSV
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
PERL_USE_DEVEL
USE_64_BIT_ALL
USE_64_BIT_INT
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
Built under linux
Compiled at Mar 19 2018 14:35:38
%ENV:
PERLBREW_BASHRC_VERSION="0.80"
PERLBREW_HOME="/home/kent/.perlbrew"
PERLBREW_MANPATH="/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/man"
PERLBREW_PATH="/home/kent/perl5/perlbrew/bin:/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/bin"
PERLBREW_PERL="5.27.9-nossp-oaath-nopmc-nodot"
PERLBREW_ROOT="/home/kent/perl5/perlbrew"
PERLBREW_VERSION="0.80"
@INC:
/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/site_perl/5.27.9/x86_64-linux
/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/site_perl/5.27.9
/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/5.27.9/x86_64-linux
/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/5.27.9
Just saw a variation of this failure (perl 5.20.2, debian/jessie):
# Failed test at t/response.t line 142.
# got: '90.1'
# expected: '90'
# Failed test at t/response.t line 143.
# got: '180.2'
# expected: '180'
# Looks like you failed 2 tests of 68.
t/response.t ................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/68 subtests
Looks like a timing issue. May be reproduced by putting a sleep() within the test, e.g.
diff --git a/t/response.t b/t/response.t
index 2412e52..4f5a2fd 100644
--- a/t/response.t
+++ b/t/response.t
@@ -139,6 +139,7 @@ is($r2->freshness_lifetime(h_default => 900), 900);
is($r2->freshness_lifetime(h_min => 7200), 7200);
is($r2->freshness_lifetime(time => time), 3600);
$r2->last_modified(time - 900);
+sleep 1;
is($r2->freshness_lifetime, 90);
is($r2->freshness_lifetime(h_lastmod_fraction => 0.2), 180);
is($r2->freshness_lifetime(h_min => 300), 300);
A possible solution is to hack around system's time() to make sure that the clock does not move: eserte@e045b2c
However, I feel somewhat uncomfortable using CORE::GLOBAL stuff and overriding builtins. Maybe a simpler and more pragmatic solution would be to just accept both return values, 90 and 90.1 resp. 180 and 180.2.
I like this module - https://metacpan.org/pod/Test::MockTime
However not perfect: http://matrix.cpantesters.org/?dist=Test-MockTime%200.17;reports=1
(This reminds me that I wanted to create an issue about the possible test failures)
Here's another possible solution: eserte@d57c758
The test code does not look pretty anymore, but it is quite robust against all kind of delays (try sleep $something with any value after the last_modified call, it should always work).
Here's another possible solution: eserte/HTTP-Message@d57c758
LGTM.
How about using Test::Number::Delta, so
is($r2->freshness_lifetime, 90);
becomes something like:
use Test::Number::Delta qw/ delta_within /;
delta_within($r2->freshness_lifetime, 90, 0.2);
Sounds great!