reduce process isolation overhead

[Markus Staab]

while looking more into phpunit performance, I was profilling subprocesses:

I think it is pretty interessting that running a test in isolation can be dominated by e.g. script compile time.

I wonder whether we can - per default - set a few options for the subprocesses used by process-isolation, e.g.

• -d opcache.jit=disable) to disable jit overhead
• -d opcache.validate_timestamps=0
• -d zend.enable_gc=0 disable GC?

---

another thing which came to mind: could we place the test beeing run separate from the "framework" code required to run in isolation, so the "bootstrapping" of the isolated processes don't need to re-compile everything but at best only the actual test-case?

---

one last thing: maybe we can utilize opcache preloading?

---

file based opcode caching?

https://stackoverflow.com/a/35880017

[Markus Staab]

some more food for thinking: maybe we can utilize pcntl_fork() instead of creating subprocesses.
thesis is: a forked process already contains everything the parent contained, so when forked 'early enough' all of phpunit would already be loaded and would not need to be loaded again

[Markus Staab]

bit of progress:

I wonder whether we can - per default - set a few options for the subprocesses used by process-isolation, e.g.

* `-d opcache.jit=disable)` to disable jit overhead

* `-d opcache.validate_timestamps=0`

* `-d zend.enable_gc=0` disable GC?

I tried these and did not find any measurable differences..

maybe we can utilize pcntl_fork() instead of creating subprocesses.

started looking into a POC for subprocess forking. stay tuned.

[Mason Malone]

I've also been investigating ways to speed up our test suite, which makes heavy use of process isolation, but unfortunately I haven't found anything that helps without major caveats.

You mentioned file-based opcaching. I tried that by setting opcache.file_cache=/tmpfs, but unfortunately this causes segfaults (both on aarch64 and amd64):

$ php --version
PHP 8.2.10 (cli) (built: Sep  4 2023 08:13:17) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.2.10, Copyright (c) Zend Technologies
    with Zend OPcache v8.2.10, Copyright (c), by Zend Technologies
    with Xdebug v3.2.1, Copyright (c) 2002-2023, by Derick Rethans

$  time phpunit -c ./phpunit.xml-dist tests/JsonHelperTest.php
PHPUnit 9.5.23 #StandWithUkraine

.....EE....EE.................................................    62 / 62 (100%)

Time: 00:00.645, Memory: 22.00 MB

There were 4 errors:

1) JsonHelperTest::testGetAssetData with data set "isAudio is false" (false, false, array('https://example.com', 'https://example123.com', 'false'))
PHPUnit\Framework\Exception: Segmentation fault

2) JsonHelperTest::testGetAssetData with data set "isAudio is true " (true, true, array('/v1/pics/thumbnails/audio_thu...ig.jpg', '/v1/pics/thumbnails/audio_thu...ig.jpg', 'true'))
PHPUnit\Framework\Exception: Segmentation fault

3) JsonHelperTest::testGetPaymentInstrumentData with data set "credit card" (array(array('AMEX', false, 'CREDIT_CARD', '123456', null, '7890', '12-01-2026', '7890'), array('US'), array('First Last'), array('123ABC'), array(true, true), array('amex')), array('
123ABC', 'CREDIT_CARD', 'AMEX', 'amex', 'First Last', '123456', '123456', false, '7890', '01/2026', '7890'))
PHPUnit\Framework\Exception: Segmentation fault

4) JsonHelperTest::testGetPaymentInstrumentData with data set "PayPal" (array(array('PAYPAL', true, 'PAYPAL', '123456', '[e-mail]', null, '', null), array('US'), array('First Last'), array('123ABC'), array(false, false), array('amex')), array('123ABC'
, 'PAYPAL', 'PAYPAL', 'amex', 'First Last', '123456', '123456', true, '[e-mail]'))
PHPUnit\Framework\Exception: Segmentation fault

ERRORS!
Tests: 62, Assertions: 58, Errors: 4.

real    0m0.919s
user    0m0.399s
sys     0m0.253s

[Markus Staab]

I am kind of specialized in perf analysis. If you can give me access to the test suite I can analyze for perf optimization possibilities. Feel free to contact me outside this issue

[Mason Malone]

@staabm Thanks for the offer, but this is proprietary source code.

One question: what did you use to generate the screenshot with the performance graph? I've been using XDebug to generate cachegrind profiles and analyzing them using qcachegrind, which works, but it's clunky.

[Markus Staab]

Blackfire.io

My tooling also works on closed source codebases ;-)

[Markus Staab]

another thing I tried unsuccessfully:

inspired by php/php-src#13778 I tried using proc_open without a shell.
it did not yield any perf improvements on macos though

[Markus Staab]

it did not yield any perf improvements on macos though

according to symfony/symfony#43162 (comment) I need to test again with php 8.3+ and maybe on a different os