We should actually move the specs of the UI tests into this Piwik repository. Otherwise we'd always have to create a branch of the UI-test repository when we are working in a branch in Piwik itself and merge later. Also in general we do want to have the test definitions in the Piwik repository similar to JS / PHP tests.
On the contrary the UI test library is in the Piwik repository and could be moved from
tests/lib/screenshot-testing into its own repository (or existing UI test repo). Ideally the expected screenshots itself would be in Piwik as well but I presume they are not there to keep the repo size smaller? I would be ok with that but might light to failures when expected screenshots and the specs are separated.
Currently it is the other way around, the lib is in Piwik (which barely changes) and the test definitions are in the UI test repo (which often changes). Goal would be to have specs in Piwik, the lib in a separate repository. I was thinking about this already for a while but I didn't really mind until I got annoyed by not being really able to write the new UI tests in a branch of Piwik
Ideally the expected screenshots itself would be in Piwik as well but I presume they are not there to keep the repo size smaller?
yes we keep the screenshots in separate repo as the repo size is huge with those thousands screenshots.
overall :+1: !
Guess it would be cool to move the Screenshot testing lib to a separate
repo. Maybe we could also make it easily reuseable by others.
Before making it resueable by others I'd switch to PhantomCSS . It is documented, has more features that we can use etc. But yes at least having it in a separate repo makes sense as suggested but not necessarily needed
Actually, I do not think it makes sense to move our UI Screenshot lib
tests/lib/screenshot-testing to a separate repository as we do not want to make it reusable by others anyway. It would only make everything more complicated. Let's keep it in the Piwik directory. Maybe we will use another lib at some point anyway (such as PhantomCSS or BackstopJS).
Moving the specs into the Piwik repository makes still sense.