Missing unit test work after changing code snippet building process

[Chun-Sheng, Li]

Description

As title, since we've changed the code snippets building process, some works are missed during Travis CI build.

As we look at the .travis.yml, it only has the code snippets building and it doesn't have any unit test works to validate these code snippet behaviors reliable and expected.

And it also misses the section of how-to test cases writing on CONTRIBUTING.md

IMHO, we should add this back and add contributing tips about unit test writing.

@sohelamin, and @Chalarangelo, please concern this at your available time.

And recommend current solution to accomplish this issue.

Thanks :).

[Angelos Chalaris]

The test system was dropped by the @30-seconds/meta team during the transition, but its artifacts remain in the repository. The intent was for the @30-seconds/php team to pick this up and make it usable, open a PR and, after reviewing and checking, merge and update Travis integration.

I would love to see this updated with a system that extract the usable code from snippets and creates tests automagically, like the JS repo. I can't help much with this, however, as my PHP knowledge is rather limited and my time is even more limited lately.

[Chun-Sheng, Li]

Hi @Chalarangelo, thanks for your reply.

IMHO, my current solution is as follows:

• Editing the .travis.yml setting file and we can changed into following snippets:

language: node_js
cache:
  directories:
    - node_modules
node_js:
- lts/*
script:
- composer install
- phpunit
- npm run extractor
- npm run builder
after_success:
- chmod +x .travis/push.sh
- .travis/push.sh

The Travis CI build will do unit tests work firstly.

Once this unit test work is successful, the building code snippets will be triggered via npm scripts.

• Add the current tests to the current Test class on tests folder.

What do you think about this solution?

And now I'm going to research how to let Node.js and PHP can run on Travis CI build at the same moment 🤔.

[Angelos Chalaris]

Running both php and nodejs is tricky enoguh, but I think it's doable - I read somewhere there is a way to add the environments as an array or something.

The issue is that tests are still manual and snippets have to be added to a central file to be tested, correct? We should find a way to abstract this, if possible.

[Chun-Sheng, Li]

Hi @Chalarangelo, thanks for your reply. The test snippets still are manual for contributors.

Do you have any way to abstract this? Customizing the npm script or php scripts to generate test snippets abstract or other possible way to resolve this?

The code snippet guideline has abstract now.

[Angelos Chalaris]

If we can't automatically identify missing tests - meaning snippets without test coverage - then we will have to manually curate all of the codebase to know where tests are missing. Testing is great, don't get me wrong, but the maintenance cost might not be worth it if we have to follow up on every snippet to check if it is added to the list of snippets, if it has tests and if tests are working as intended. This will require the repo's team to manually check every PR before merging and, as it currently seems, we are all not very active in terms of maintaining this repo.

I'm not saying we shouldn't have tests, but automating them on each PR on Travis, without having a way to block PRs from merging when snippets are not properly tested, might not be the optimal solution. In my mind it's a situation where we either have both automations set up or none at all.

[Chun-Sheng, Li]

Thanks for your reply, @Chalarangelo.

I think we can resolve this automatic test problem.

Let every snippet split to single class PHP files
And use simple script to check whether the snippets are existing.
If not, the Travis CI build will be expected to be failed.

[Angelos Chalaris]

I am closing this yet again due to inactivity. If someone actually starts working on this feature, open a PR for it and we can track it there. Thank you.