t/07-taint.t may fail (on env var TEST_ACTIVE)
eserte opened this issue · comments
In rare cases t/07-taint.t fails:
# Failed test 'Sanity - ENV vars are tainted'
# at t/07_taint.t line 21.
# Failed test 'Sanity - Evil zero is tainted'
# at t/07_taint.t line 25.
# Failed test 'Single-arg, tainted data'
# at t/07_taint.t line 41.
# ''
# doesn't match '(?^:called with tainted argument)'
# Failed test 'multi-arg, tainted data'
# at t/07_taint.t line 44.
# ''
# doesn't match '(?^:called with tainted argument)'
# Looks like you failed 4 tests of 13.
t/07_taint.t ..............
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/13 subtests
The rare failures can be made happen more often by cleaning the environment (just keeping PATH):
$ env -i PATH=$PATH perl5.25.12 -Mblib -T t/07_taint.t
1..13
ok 1 - $perl_path is clean
ok 2 - use IPC::System::Simple;
not ok 3 - Sanity - ENV vars are tainted
# Failed test 'Sanity - ENV vars are tainted'
# at t/07_taint.t line 21.
not ok 4 - Sanity - Evil zero is tainted
# Failed test 'Sanity - Evil zero is tainted'
# at t/07_taint.t line 25.
ok 5 - Sanity - Evil zero is still zero
ok 6 - Single-arg, tainted ENV
ok 7 - Multi-arg, tainted ENV
not ok 8 - Single-arg, tainted data
# Failed test 'Single-arg, tainted data'
# at t/07_taint.t line 41.
# ''
# doesn't match '(?^:called with tainted argument)'
not ok 9 - multi-arg, tainted data
# Failed test 'multi-arg, tainted data'
# at t/07_taint.t line 44.
# ''
# doesn't match '(?^:called with tainted argument)'
ok 10 - Single-arg, clean data and ENV
ok 11 - Multi-arg, clean data and ENV
ok 12 - Returns of multi-arg capture should be tainted
ok 13 - Returns of single-arg capture should be tainted
# Looks like you failed 4 tests of 13.
If adding some diagnostics into the test file to print out the first key in %ENV, then it's always TEST_ACTIVE --- probably added by the testing framework, and it seems that this environment variable is treated specially.
Some further thoughts for reproducing the problem:
- probably needs at least Test::Simple 1.302065, see https://metacpan.org/source/EXODIST/Test-Simple-1.302083/Changes#L93
- probably needs a perl with hash randomization (> 5.17.6); otherwise it's unlikely that
TEST_ACTIVEis ever the first key in%ENV
[edit, added comment about sucesses and failures at the bottom]
I just stumbled across this too, in a Jenkins job:
cpanm (App::cpanminus) 1.7044 on perl 5.024001 built for x86_64-linux
[...]
--> Working on IPC::System::Simple
Fetching file:///tmp/ft-cimr-worker-dir/workspace/ci-push-master-or-develop/pinto/repository/authors/id/P/PJ/PJF/IPC-System-Simple-1.25.tar.gz
-> OK
Unpacking IPC-System-Simple-1.25.tar.gz
Entering IPC-System-Simple-1.25
Checking configure dependencies from META.json
Checking if you have ExtUtils::MakeMaker 6.58 ... Yes (7.10_02)
Configuring IPC-System-Simple-1.25
Running Makefile.PL
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for IPC::System::Simple
Writing MYMETA.yml and MYMETA.json
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have constant 0 ... Yes (1.33)
Checking if you have warnings 0 ... Yes (1.36)
Checking if you have Test::More 0 ... Yes (1.302133)
Checking if you have re 0 ... Yes (0.32)
Checking if you have File::Basename 0 ... Yes (2.85)
Checking if you have strict 0 ... Yes (1.11)
Checking if you have Carp 0 ... Yes (1.40)
Checking if you have Exporter 0 ... Yes (5.72)
Checking if you have Test 0 ... Yes (1.28_01)
Checking if you have Scalar::Util 0 ... Yes (1.50)
Checking if you have List::Util 0 ... Yes (1.50)
Checking if you have POSIX 0 ... Yes (1.65)
Building and testing IPC-System-Simple-1.25
make[1]: Entering directory `/home/hartzell/.cpanm/work/1525193014.4674/IPC-System-Simple-1.25'
cp lib/IPC/System/Simple.pm blib/lib/IPC/System/Simple.pm
make[1]: Leaving directory `/home/hartzell/.cpanm/work/1525193014.4674/IPC-System-Simple-1.25'
make[1]: Entering directory `/home/hartzell/.cpanm/work/1525193014.4674/IPC-System-Simple-1.25'
PERL_DL_NONLAZY=1 "/opt/perl5/perls/perl-5.24.1/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/01_load.t ............... ok
t/02_exit.t ............... ok
t/03_signal.t ............. ok
t/04_capture.t ............ ok
t/05_multi_capture.t ...... ok
t/06_fail.t ............... ok
# Failed test 'Sanity - ENV vars are tainted'
# at t/07_taint.t line 21.
# Failed test 'Sanity - Evil zero is tainted'
# at t/07_taint.t line 25.
# Failed test 'Single-arg, tainted data'
# at t/07_taint.t line 41.
# ''
# doesn't match '(?^:called with tainted argument)'
# Failed test 'multi-arg, tainted data'
# at t/07_taint.t line 44.
# ''
# doesn't match '(?^:called with tainted argument)'
# Looks like you failed 4 tests of 13.
t/07_taint.t ..............
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/13 subtests
t/08_core.t ............... skipped: BSD::Resource required for coredump tests
t/09_system.t ............. ok
t/10_formatting.t ......... ok
t/11_newlines.t ........... ok
t/12_systemx.t ............ ok
t/13_exports.t ............ ok
t/14_uninitialised.t ...... skipped: Test::NoWarnings required for testing undef warnings
t/author-critic.t ......... skipped: these tests are for testing by the author
t/internal.t .............. ok
t/release-pod-coverage.t .. skipped: these tests are for release candidate testing
t/release-pod-syntax.t .... skipped: these tests are for release candidate testing
t/win32.t ................. skipped: Win32 only tests
Test Summary Report
-------------------
t/07_taint.t (Wstat: 1024 Tests: 13 Failed: 4)
Failed tests: 3-4, 8-9
Non-zero exit status: 4
Files=19, Tests=125, 1 wallclock secs ( 0.05 usr 0.01 sys + 0.81 cusr 0.24 csys = 1.11 CPU)
Result: FAIL
Failed 1/19 test programs. 4/125 subtests failed.
make[1]: *** [test_dynamic] Error 255
make[1]: Leaving directory `/home/hartzell/.cpanm/work/1525193014.4674/IPC-System-Simple-1.25'
-> FAIL Installing IPC::System::Simple failed. See /home/hartzell/.cpanm/work/1525193014.4674/build.log for details. Retry with --force to force install it.
[...]
It had been working, and worked the next time the job fired.
Just failed for me on something that (like the previous commenter) usually works fine. Same four tests failed.
Test-Simple-1.302136
Perl 5.22.1
IPC-System-Simple-1.25
I then ran the install again and it was fine.
I got bit by this today, too.
I'm working on a program which will test the top 3000 of the "CPAN river" against perl-5.29.* monthly development releases. I was testing out the FreeBSD-11 VM in which we expect to run the program in production by testing those 3000 against perl-5.28.0-RC2, using cpanm as the installer program. Excerpt from the build.log:
Building and testing IPC-System-Simple-1.25
cp lib/IPC/System/Simple.pm blib/lib/IPC/System/Simple.pm
PERL_DL_NONLAZY=1 "/usr/home/jkeenan/var/tad/testing/perl-5.28.0/bin/perl" "-MExtUtils::Command::MM" "-MTest::
Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/01_load.t ............... ok
t/02_exit.t ............... ok
t/03_signal.t ............. ok
t/04_capture.t ............ ok
t/05_multi_capture.t ...... ok
t/06_fail.t ............... ok
# Failed test 'Sanity - ENV vars are tainted'
# at t/07_taint.t line 21.
# Failed test 'Sanity - Evil zero is tainted'
# at t/07_taint.t line 25.
# Failed test 'Single-arg, tainted data'
# at t/07_taint.t line 41.
# ''
# doesn't match '(?^:called with tainted argument)'
# Failed test 'multi-arg, tainted data'
# at t/07_taint.t line 44.
# ''
# doesn't match '(?^:called with tainted argument)'
# Looks like you failed 4 tests of 13.
t/07_taint.t ..............
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/13 subtests
t/08_core.t ............... skipped: BSD::Resource required for coredump tests
t/09_system.t ............. ok
t/10_formatting.t ......... ok
t/11_newlines.t ........... ok
t/12_systemx.t ............ ok
t/13_exports.t ............ ok
t/14_uninitialised.t ...... ok
t/author-critic.t ......... skipped: these tests are for testing by the author
t/internal.t .............. ok
t/release-pod-coverage.t .. skipped: these tests are for release candidate testing
t/release-pod-syntax.t .... skipped: these tests are for release candidate testing
t/win32.t ................. skipped: Win32 only tests
Test Summary Report
-------------------
t/07_taint.t (Wstat: 1024 Tests: 13 Failed: 4)
Failed tests: 3-4, 8-9
Non-zero exit status: 4
Files=19, Tests=127, 2 wallclock secs ( 0.05 usr 0.04 sys + 1.12 cusr 0.63 csys = 1.84 CPU)
Result: FAIL
Failed 1/19 test programs. 4/127 subtests failed.
*** Error code 255
Stop.
make: stopped in /usr/home/jkeenan/var/tad/testing/perl-5.28.0/.cpanm/work/1528899970.49796/IPC-System-Simple-1.25
-> FAIL Installing IPC::System::Simple failed. See /home/jkeenan/var/tad/testing/perl-5.28.0/.cpanm/work/1528899970.49796/build.log for details. Retry with --force to force install it.
This was a serious failure, as IPC-System-Simple has DateTime-Locale as a reverse dependency, which in turn means that anything that depends on DateTime or Moose will FAIL or perhaps not even be reached in an automated testing process like mine. I would have expected approximately 2875 modules to get a PASS grade from cpanm; instead, I got only 2575. This meant that the remaining 6 hours of running time of this program was effectively squandered.
(I should add that when I subsequently ran ./bin/cpanm IPC::System::Simple apart from my automated program, the module installed, just as it did for @realflash .)
I hope that the author/maintainer can address this problem quickly. If the ultimate cause of the problem cannot be determined, I recommend a fresh CPAN release which places those 4 tests in a TODO block so that failures don't prevent the module from being installed and serving as a prerequisite to literally thousands of CPAN distributions.
Thank you very much.
Jim Keenan
@pjf I got bit by this problem again yesterday. It rendered my CPAN River 3000 testing severely impeded. Please see https://www.nntp.perl.org/group/perl.perl5.porters/2019/02/msg253766.html. If you are no longer actively maintaining this CPAN distribution, could you designate a co-maintainer?
Thank you very much.
@jkeenan : Oh no, thank you all so very much for the nudge for this, and I'm so sorry for my slow response!
I'm in the process of an international move and I've not been able to give IPC::System::Simple the attention it deserves, and it should absolutely have a co-maintainer. If you'd be willing to volunteer then I'd be delighted with that, especially as you've both been caught by this before.
Many thanks again!
~ Paul
@jkeenan : Oh no, thank you all so very much for the nudge for this, and I'm so sorry for my slow response!
I'm in the process of an international move and I've not been able to give IPC::System::Simple the attention it deserves, and it should absolutely have a co-maintainer. If you'd be willing to volunteer then I'd be delighted with that, especially as you've both been caught by this before.
Many thanks again!
~ Paul
I would be willing to accept COMAINT on this module -- though that would be largely for the purpose of resolving this issue and getting a new CPAN release out. You can find instructions on designating a co-maintainer by logging in to PAUSE and going to the Add Comaintainers entry.
Have you considered this patch from ppisar? ppisar@ff8028c I think that would be the first step.
Thank you very much.
JIm Keenan
@jkeenan : Oh no, thank you all so very much for the nudge for this, and I'm so sorry for my slow response!
I'm in the process of an international move and I've not been able to give IPC::System::Simple the attention it deserves, and it should absolutely have a co-maintainer. If you'd be willing to volunteer then I'd be delighted with that, especially as you've both been caught by this before.
Many thanks again!
~ PaulI would be willing to accept COMAINT on this module -- though that would be largely for the purpose of resolving this issue and getting a new CPAN release out. You can find instructions on designating a co-maintainer by logging in to PAUSE and going to the Add Comaintainers entry.
Have you considered this patch from ppisar? ppisar@ff8028c I think that would be the first step.
Thank you very much.
JIm Keenan
Paul: I got bit by this test failure again this week when conducting testing of the CPAN-River-3000 against perl-5.31.5 dev release. As I've said previously:
This was a serious failure, as IPC-System-Simple has DateTime-Locale as a reverse dependency, which in turn means that anything that depends on DateTime or Moose will FAIL or perhaps not even be reached in an automated testing process like mine.
Would you please take any of the actions suggested above in this thread (test modification; new CPAN release; COMAINT) so that we can avoid this problem going forward?
Thank you very much.
Jim Keenan
Hi folk, Paul has had a difficult year, but I've nudged him directly and I think it likely that he'll get on to assigning COMAINT today or tomorrow. J
…
On Wed, Oct 23, 2019 at 6:49 PM James E Keenan @.> wrote: @jkeenan https://github.com/jkeenan : Oh no, thank you all so very much for the nudge for this, and I'm so sorry for my slow response! I'm in the process of an international move and I've not been able to give IPC::System::Simple the attention it deserves, and it should absolutely have a co-maintainer. If you'd be willing to volunteer then I'd be delighted with that, especially as you've both been caught by this before. Many thanks again! ~ Paul I would be willing to accept COMAINT on this module -- though that would be largely for the purpose of resolving this issue and getting a new CPAN release out. You can find instructions on designating a co-maintainer by logging in to PAUSE https://pause.perl.org/pause/query and going to the Add Comaintainers entry. Have you considered this patch from ppisar? @. <ppisar@ff8028c> I think that would be the first step. Thank you very much. JIm Keenan Paul: I got bit by this test failure again this week when conducting testing of the CPAN-River-3000 against perl-5.31.5 dev release. As I've said previously: This was a serious failure, as IPC-System-Simple has DateTime-Locale as a reverse dependency, which in turn means that anything that depends on DateTime or Moose will FAIL or perhaps not even be reached in an automated testing process like mine. Would you please take any of the actions suggested above in this thread (test modification; new CPAN release; COMAINT) so that we can avoid this problem going forward? Thank you very much. Jim Keenan — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=AAADI6KVLZ7G5GRXL5EACSTQQACJRA5CNFSM4DJRNZJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECAQTPA#issuecomment-545327548>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAADI6P52NZ4PNWQ3XC7BVTQQACJRANCNFSM4DJRNZJQ .
This test failure was encountered again today in the course of testing the "CPAN River 3000" against perl-5.31.8. Could a COMAINT be assigned as previously discussed?
Thank you very much.
Jim Keenan
@jkeenan : I am so sorry this took so long! As of a few minutes ago you should now have comaint permissions!
I'll also add your permissions to the repo on github, but I'm happy to transfer the repo to an organisation if that makes things easier.
Many thanks again for all your patience!
Should be fixed in CPAN release 1.26, via application of @ppisar patch.