Allow testing before publishing
splendido opened this issue · comments
...from a practical point of view it should be enough to add another command to this list
but the most part of it is providing some UI modal (i.e.) where to toggle the test before publish option.
I'd say: lets see how the project goes with some external library which does not need tests (in case we'll manage to get some into autopublish...) and then possibly expand features as required...
Support for meteor package testing might be a good idea as this will test the meteor part of the library. This will make sure a new release is working as expected with meteor before publishing.
I see no reason to add test support for the normal tests of the library. Most people probably already have a testing setup for this and I feel it's out of the scope for autopublish, travis-ci and the like is better suited for this. They might even have it testing meteor integration too, but running tests on autopublish is still useful. Just to make sure a broken new version isn't published by mistake.
I think this should be done by default as packages without meteor tests will pass and only a package with failing tests will error out. A notice could be shown for packages without meteor tests, recommending to add meteor tests. I don't think packages without meteor tests should be stopped.
Having autopublish use github api to create a new issue for failed test-runs might also be a good idea to make library authors aware of problems with meteor tests and that the new release wasn't published. Most authors will probably setup autopublish once and don't think about checking wether the meteor package was successfully published.
Maybe it's even possible to have autopublish be informed when this issue is closed and re-run the test/publication for this version? If not, maybe a manual re-run tests/publish should be available?
This should work the same for regular meteor packages like zimme:luhn too. Preventing a version which isn't passing tests from being published by mistake.
Yeah, it makes more than sense!
Notification by GitHub Issues is something I'm trying to add for succeeded/failed publish, but at this point using them also for testing results is a natural extension!
The big problem, at the moment, is that on Meteor Build Machines there's no Mongo so it's not possible to run tests there :(
I'll continue to use my personal droplet on DO to add this functionality!
I'll let you know when there's something working...
Thanks
....thinking about the 're-run' of tests + publish after fixes: that might be a problem since the tagged version would be the same also at the second run.
So either a new version is released and another autopublish flow starts or the previous release is deleted and a new one with the same tag.
This is hard to get automated, especially without having write access to repos, which is something we're trying to avoid...
Maybe Travis-ci could be used as a test runner?
And maybe a manual button on autopublish to rerun test/publications for the current tag is possible?
👍 for the button!
...but I'd say we should make it available only to 'admins' and package's owners?
Setting up a full working environment on Meteor Build Machines shouldn't be a problem since we've got interest from Matt which is willing to help with autopublish. Should be just a matter of time... :)
Sounds great... ping me if you need some help or need to bounce some ideas on someone