See discussion in https://github.com/piwik/travis-scripts/issues/19
When generating a
travis-yml file via
./console generate-travis-yml there is by default no job for a UI test generated (unless the plugin already contains some UI tests).
When generating a UI test someone can execute them locally but they won't be executed on travis and it is easy to forget to add this check manually and to never notice the tests fail (happened to me with CustomDimensions plugin).
When generating a new UI test we should check whether a
travis.yml exists, and if so add a line
TEST_SUITE=UITests MYSQL_ADAPTER=PDO_MYSQL TEST_AGAINST_PIWIK_BRANCH=$PIWIK_TEST_TARGET to
env/matrix in case the line is not present yet.
This is maybe even more important for Piwik PRO where a
travis-yml file is generated automatically on plugin creation and when they add a new UI test afterwards it won't be executed on Travis I believe (unless one adds mentioned line to travis-yml manually)
+1 to always run the UI test CI job so that there is no issue in the future when one will add the UI test.
FYI, this is not an issue for pro plugins w/ autogenerated .travis.yml files. UI test builds are forced for those plugins. Can be an issue, if the build is removed manually.
Changing title from When generating a UI test, add a UI test job to travis.yml"" to "When generating a travis.yml file, add all jobs by default and do not fail if there are no tests".
I think it's better to always run all jobs this would make it easier so we do not have to add any jobs later etc and might be even more encouraging to write ui tests etc. Maybe we could even create a default UI test with every plugin.
tests/travis/generator/Generator/PluginTravisYmlGenerator.php it would be easily to always generate all jobs by eg not checking for
isTargetPluginContainsUITests but I tested it and it would fail the build if there are eg no UI tests for a plugin.