@mattab opened this Issue on October 28th 2014 Owner

The goal of this issue is that every week we want our 40+ projects CI builds to be green with as little pain and effort as possible. Having green builds across the board will increase quality of our work in general, increase developer happiness and help prevent us from deploying broken code.

Scope: this issue covers builds for our official Piwik Plugins (VisitorGenerator, CustomAlerts, etc.) and components and all Piwik PRO plugins.

Proposed steps For The Win

  1. [x] Make all builds green on https://ci-status.com (at east once!)
  2. [x] Automatically trigger the build for each repository once a week #6542
  3. [x] in CI job against latest stable, allow failures when plugin requires a recent beta #6930

Related to

  1. Detect automatically when plugins use code that is <a class='mention' href='https://github.com/deprecated'>@deprecated</a> or not tagged with <a class='mention' href='https://github.com/api'>@api</a> #6539

What we already have today

  • Plugins run tests against piwik/piwik master, to check pro-actively that plugins work with bleeding edge Piwik core platform
  • Plugins run tests against the latest stable release, to check that plugins work with current stable Piwik release.
  • Plugins can include their own Unit / Integration / Systems tests and UI Screenshot tests
  • Each developer is responsible for fixing the build after he broke the build
  • Piwik core developers are responsible to fix failures in a plugin when the failure is caused by a BC break or change in core platform (when it occurs then plugin developer should create an issue on piwik/piwik issue tracker to notify core developers)

What else can we do to help us achieve green builds every week on 40+ repositories?

@mnapoli commented on October 28th 2014 Member

And we discussed "Detect automatically when plugins use code tagged with <a class='mention' href='https://github.com/deprecated'>@deprecated</a>"?

@mattab commented on October 28th 2014 Owner

@mnapoli Good point, it will be included in scope of #6539

@mattab commented on December 19th 2014 Owner

Current CI status: 26 builds failing out of 50 builds.

@mattab commented on January 5th 2015 Owner

Current CI status: 17 builds failing out of 45 builds.

@mattab commented on January 7th 2015 Owner

it's starting to look pretty green on CI status! 5 builds failing out of 45 builds

@mattab commented on January 7th 2015 Owner

CI Status: 9 builds failing out of 50 builds.

@mattab commented on January 11th 2015 Owner

after the automatic Sunday Builds refresh we have: 12 builds failing out of 50 builds.

@mattab commented on February 5th 2015 Owner

CI Status: 9 builds failing out of 49 builds.

@mattab commented on February 9th 2015 Owner

we're almost there! Ci Status: 6 builds failing out of 49 builds.

@mattab commented on February 27th 2015 Owner

This is getting really exciting! Summary: 2 builds failing out of 47 builds.

@mattab commented on March 13th 2015 Owner

Yesterday, 10 builds failing. Today 4 builds failing.

This new feature in ci-status will get rid of 2 failed builds: https://github.com/piwik/ci-status/issues/20

@mattab commented on March 13th 2015 Owner

When following issues are closed we can close this issue:

  • allow to exclude builds from being reported in the dashboard piwik/ci-status#20
  • Add statuses for the other branches piwik/ci-status#15
  • extend command 'development:sync-ui-test-screenshots' to sync UI screenshots of Plugins #7393

The next step will be:

  • Decide who is responsible to fix the builds so that they are green each monday.

maybe someone from the team will volunteer to "own this" (eg. for a few months), or maybe a person rotating each week or month?

@mattab commented on March 17th 2015 Owner

Created follow up issue: Decide who is responsible to fix the builds so that they are green each monday #7453

Well done to us, for the huge progress made in this area in the last few months... :+1:

This Issue was closed on March 17th 2015
Powered by GitHub Issue Mirror