[Feature Request] option to disable auto-promote with disco_deploy.py test
dawu-amplify opened this issue · comments
Currently disco_deploy.py test
does provision and test new untested AMI in CI and automatically promote it to Staging if all tests pass.
But in Burst we ran into an issue where some db schema changes have been made manually in CI for development but not yet available in Staging. Because tests work fine in CI, new AMIs are tested and auto-promoted. This causes inconsistency between database and code, therefore raises error in Staging.
There are two ways to work around this problem.
- Add an option to only provision and test AMIs with
disco_deploy.py test
- Add integration tests on Staging as soon as possible, and mark any failed AMIs as failed and therefore no new instances are provisioned in Staging.
As an immediate solution, it looks that former option may be easier to implement.
Let me know your thoughts. Any suggests are welcomed.
Why not just remove the affected hostclasses from the staging and production pipelines until you are able to put integration tests in place?