@mnapoli opened this Issue on October 30th 2014 Member

We have enabled Scrutinizer for Piwik. This CI tools mainly does code quality inspections, which includes the classic ones (pdepend, copy paste detector, …) but also a proprietary one they have developed which is pretty powerful and useful.

Scrutinizer also offers to run tests and code coverage reports. For now I have set up code coverage for unit tests only: https://scrutinizer-ci.com/g/piwik/piwik

The goal is to remove code coverage completely from Travis so that builds are even faster.

I think we should not have code coverage for integration tests. If we agree on that, then we just have to remove code coverage in the Travis build.

If we decide to keep code coverage for integration tests, then we need to configure them on Scrutinizer, then we can remove them from Travis.

So should we have code coverage for integration tests?

@mattab commented on October 31st 2014 Owner

+1 to remove coveralls (cc @sgiehl) as using scrutinizer will make our builds faster and provide interesting code quality reports

So should we have code coverage for integration tests?

I don't have strong opinion so if others agree we could do coverage for unit tests only.

@sgiehl commented on October 31st 2014 Member

Do we already have a paid plan for scrutinizer? Otherwise it only supports sending external code coverage like coveralls does.
I'm not sure if it's good to remove coveralls.io. Currently we have the option to build code coverage for all plugins using coveralls.io. That can also be used (very easily) by third party plugins. Moving to scrutinizer will kill that possibility I guess, as we would remove the coveralls code.

@mnapoli commented on November 2nd 2014 Member

@sgiehl for Scrutinizer we can run the tests (+code coverage) for free (as long as the repo is open source).

We just discussed some difficulties about plugins friday, what we suggest:

  • move piwik/piwik code coverage builds to Scrutinizer (so that Travis builds faster)
  • leave code coverage builds on Travis for plugins, but use the support for sending eternal code coverage to Scrutinizer
  • drop Coveralls, since code coverage can be handled on Scrutinizer (and Scrutinizer does more than Coveralls)

Moving to scrutinizer will kill that possibility I guess, as we would remove the coveralls code.

3rd party plugins can also use Scrutinizer since it's free?

@mattab commented on November 3rd 2014 Owner

@sgiehl Scrutiniser also provides other reports beyond coverage that are quite interesting, likely they would help the Piwik developer community (including us) increase the code quality of plugins :+1:

followed up in #6931 Generate Unit tests code coverage report for Plugins

This Issue was closed on November 4th 2014
Powered by GitHub Issue Mirror