Use PHP with Behat rather than expect

[Lewis Cowles]

I noticed the functional tests use expect command line utility. I Was surprised by this as Behat can be used for the same purpose.

My feature title

Use Behat instead of expect

Background/problem

This would avoid the need to install expect onto a system.

Proposal/solution

Use Behat and keep tests to PHP

Alternatives

Leave things as they are, or use another cucumber DSL

Additional context

While I was checking out a PHP 8.1 issue you'd reported and asked for help with, this surprised me

bash: expect: command not found

That error was actually swallowed and instead I saw

$ ./tests/bash-test ./tests/functional-tests.sh
bash-test v0.3.0 by Campanda GmbH and contributors.

functional-tests.sh


[FAILURE] Output for failed test config_01:


  ✗ _test_expect config_01


[FAILURE] Output for failed test config_02:


  ✗ _test_expect config_02


[FAILURE] Output for failed test config_03:


  ✗ _test_expect config_03


[FAILURE] Output for failed test config_04:


  ✗ _test_expect config_04


[FAILURE] Output for failed test config_05:


  ✗ _test_expect config_05


[FAILURE] Output for failed test config_with_defaults:


  ✗ _test_expect config_with_defaults


[FAILURE] Output for failed test prepare_basic:


  ✗ _test_expect prepare_basic


[FAILURE] Output for failed test prepare_with_alt_config_and_many_errors:


  ✗ _test_expect prepare_with_alt_config_and_many_errors


[FAILURE] Output for failed test prepare_with_invalid_type:


  ✗ _test_expect prepare_with_invalid_type


[FAILURE] Output for failed test prepare_with_long_desc_and_more_values:


  ✗ _test_expect prepare_with_long_desc_and_more_values

 10 of 10 tests failed.

real    0m4.192s
user    0m0.363s
sys     0m1.096s

I Couldn't find a FeatureContext or similar php file. So I went digging into what was happening and saw this.

[Ben Ramsey]

I'm open to this, but I don't know much about Behat, and when I've used it in the past, we had to write a lot of extra tooling around it to support our user story language, so it seemed like a lot of extra programming work for little gain.

The key here is that we need to execute the commands in a shell and verify the output. We can also do this from standard PHPUnit, which I would prefer over introducing another testing framework.