Change the name of functional test to smoke tests

[Denis Costa]

The add semantic value to these tests name I'm changing the name.

[Denis Costa]

The change was done at 7eb4452

[Denis Costa]

I just think I messed with pixelated-platform-environments. To review this issue please take a look on that repo.

[Denis Costa]

To review it you can ssh to a environment that doesn't have ths changes and take a look on crontab (crontab -e) before and after deploy.

[Varac]

@deniscostadsc: I found two things:

The old crontab entry still exists (even when it does nothing):

# Puppet Name: run_functional_tests
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
27 0 * * * : # This cronjob is temporary, it need to be remove after it have been run in all environments

I propose to remove it by using:

cron {'run_functional_tests':
    ensure => absent,
    command     => ": # This cronjob is temporary, it need to be remove after it have been run in all environments",
   ...
  }

[Varac]

also, the old check_mk job is still registered at /var/lib/check_mk_agent/job/root/pixelated-functional-tests

With this file, check_mk/nagios will keep the job, even when it's not updated any more.
Removing this file will remove the nagios check on the next inventory.

Also, the tests themselves still live in the files/functional-tests directory, shouldn't they also get renamed to files/smoketests. And the directory they are deployed to on the server is also named /srv/leap/tests_custom/functional-tests.

[Denis Costa]

But that dummy cron was the easiest way to invalidate the previous entry. We just delete in a few weeks.

I didn't find out a better way to do it at that time.

[thaissiqueira]

We agreed that renaming these tests is not enough to solve the flakyness nor semantic.

• We are using functional tests (as smoke test) to monitor the application.
• Smoke testing should be preliminary testing to reveal simple failures severe enough to reject a commit [1]. In your case, we could do some requests in key parts of application right after the deploy.

Actually we need a better way to monitor the application and tests to ensure quickly the applications is working fine.

[1] - https://en.wikipedia.org/wiki/Smoke_testing_(software)

So we are closing this issue and we are create new card to deal with the actual problems.

[Varac]

@deniscostadsc:

But that dummy cron was the easiest way to invalidate the previous entry. We just delete in a few weeks.
I didn't find out a better way to do it at that time.

Don't worry, that's why i suggested to use the code above in my comment, which would immediately remove the old cron entry.
Imo it's good to finish this renaming because i fear we will forget about it soon.

I'm happy to pair on this together with you I just don't like a situation where the tests are called differently in separate places of the repo, this is just confusing.

[Renata Nobre]

hey, I just would like to put here my opinion.
After talked with @thaissiqueira, I disagreed that rename all these test to smoke test is not a good thing to do now. There are functional tests that we are running as monitoring. So don't make sense to call them as smoke tests.
I think one thing that we can do to have all the tests with the same name is do the rollback and keep them as functional tests.
:)

[Denis Costa]

@varac

I've made the changes we discussed on this branch https://github.com/pixelated/puppet-pixelated/commits/smoke-tests. After we can take a look together.

[Varac]

@deniscostadsc thx! looks good, I left two comments in the github commit view for you, these are minor nitpicks. So we can merge after you updated your commits and deploy and test on unstable

[Denis Costa]

@varac I did those things you mentioned and I merged to master. I put this issue to review.

[Varac]

thx @deniscostadsc, i saw you deployed to dev1 and the only thing that is left was deploying yo unstable.pix because nagios is only running on unstable and checks dev.pix as well. Before, nagios reported an unknown check (Job pixelated-smoke-tests):

root@unstable1:~# ncli_problems 
dev1.dev.pixelated-project.i
    Job pixelated-smoke-  UNKNOWN UNKNOWN - Got no information for this job

So deploying to unstable will re-inventorize the checks on dev.pix and will purge the check that got removed.