The goal of this issue is to decide on a process to ensure that the Piwik Product team has a workflow that will ensure that the CI builds across our 50+ repositories are green on every monday.
We could test something for a few weeks or months (eg. test the solution
i. ) and if that does not well, change it to another process...
Your feedback most welcome!
one feedback is that fixing the tests every week is quite time consuming - maybe we could schedule to fix the builds of all plugins only once a month, for example ~ two weeks before the release
minimum required piwikversion (as set in its plugin.json file)
Summary: our plugins should never break or get broken by a newer Piwik version.
minimum required Piwikwould fail build when the CI job fails
minimum required piwik
latest stable Piwikthat would fail the build if the job fails
stable version of Piwikhas been released (or when the plugin starts to break against the
minimum required piwik).
platform stable release, the plugin build fails due to a bug in platform stable release, but this should be a rare case considering that our plugins will only call public APIs and non-deprecated code...)
allowed failuresand would manually check if the failures are due to a cosmetic change or if any failure is due to a regression in core platform.)
what do you think?
may also check external plugins wp-piwik and woocommerce tracking.
e.g. they show data on dashboard which may is broken after update piwik.
@mattab I think this issue is a lot simpler now. PRO builds test against specific Piwik versions only, correct? In this case we only have to maintain the open source plugins that run tests against master.
I think this issue is a lot simpler now. PRO builds test against specific Piwik versions only, correct?
I believe that one suggestion by @mgazdzik and team is to run the PRO builds against the latest Piwik stable version only. @diosmosis I don't know if it's implemented yet though, or whether it's consistently implemented (eg. all premium plugins run builds against latest stable only, while open source plugins run builds against latest stable + master).
Maybe we need to confirm this is what we want to do, and then ensure it's consistently applied across all plugins, then hopefully we will be in a better position with regards to keeping builds green?
we are making some progress