Steps to complete: - ~~Move all of test environment setup to Fixture class.~~ (done in uiunittests branch) - ~~Extend mocha & chai to handle screenshot compare use-case.~~ - ~~Create JS script that will run a PHP program to setup the database and run each UI test suite.~~ - ~~Modify existing UI tests.~~ - ~~Modify travis config to run via phantomjs.~~
Reason for using Jasmine: phantomjs uses Jasmine internally for its tests so we know it'll work. mocha + chai can be used as an alternative.
CC-ing tsteur as per matt's request.
Update: Using mocha + chai is actually easier to get running and easier to extend.
+1 for MochaJS see #4691
In 71dee21d4988ae03cad321b0d9f1fd735151884b: Refs #4739, adding mocha + chai to tests/lib dir.
In 39c8961b418e151d13b8652049c257186df3962d: Refs #4739, remove unneeded persist_fixture_data.
In f2faa23854874b25cb0ac54656b7b3d90a1dd0b7: Refs #4739, make sure recursive directory links are created even when --persist-fixture-data is used.
BTW: probably not really useful but just mention it... I built something similar a year ago to run tests using PhantomJS + Jasmine 2.0 but would now use MochaJS. Reckon there is not really anything new for you but just wanted to mention it. There is a testrunner https://github.com/tsteur/titanium-alloy-jasmine-testing/blob/master/tests.js and a loader https://github.com/tsteur/titanium-alloy-jasmine-testing/blob/master/specs/jasmine/utils/loader.js who finds all the test files in a specs directory: https://github.com/piwik/piwik-mobile-2/tree/master/specs
@tsteur If the tests for the angularjs code will run in a webpage, then you won't need any code I've written. The screenshot tests run directly in phantomjs, not in a webpage.
In bf92e42b541553f4bc585c6a210e2c6632aa4d61: Refs #4739, add console command to run UI tests.
In 35586dfaf52c795cfc4aa07891364f7bcabbe7fd: Refs #4739, pass groups to new RunUITests console command.
In 8ba7f01679981cd9375acb091e638e437bd15a67: Refs #4739, move screenshot testing library code to core repository (from UI repository).
In 4ad2b987467fd57c0ef5a0bbaa8c7f66845d69e1: Refs #4739, make sure UI screenshots are stored in plugin directories if tests are for plugins, cleaned up path output in console and add .gitignore to DBStats plugin.
In 8d72a9fc7f1b8792f67f80dbb86bd55f681369b8: Refs #4739, fixing bug in processed screenshot dir path creation.
In ba6a60ac294e8663687ef67cb04a060346f7d809: Refs #4739, add information about new UI tests to screenshot testing README.
In f073d1056bd17ee0f6940b9e571a7e5c890d4b64: Refs #4739, modify SyncUITestScreenshots command to work with new screenshot testing code and display location hint in diffviewer output if expected file is stored within plugin and not the UI tests repo.
In 28a2f9b15cd146ad6bf5921e351767a6d9bd9392: Refs #4739, add travis only command line options to UI test runner: --store-in-ui-tests-repo, --use-github-expected.
@tsteur FYI, I did some thinking and I think the screenshot testing code could be used for angularjs tests too. For example:
I'm not sure how much extra work it will be to get it to work or if there are any big benefits, but I think it's possible.
In bb5e2b37ae5f04f78ec3eae636d70e38bb34f89f: Refs #4739, remove old capture.js file, add section about screenshot testing config to README and remove unused constant from Config.php.
In f44b9529a1cc8961782be8b240d9ab77d309789a: Refs #4739, move segment setup code to UITestFixture and make sure it is run on every test run.
In 9c9a91fc883b61bd3cd3330e23f9a64f29d2e1fc: Refs #4739, remove zeitgeist query parameter handling in TestingEnvironment.
In c9f13e1cc9c8348df8a3f4679116c8e85df2455e: Refs #4739, add dragDrop utility function to screenshot test page renderer and capture alert()'s during screenshot tests.
In d789162805fda622eb46c717b4d8c846d5b4b3ad: Refs #4739, display page logs when a screenshot test times out.
In 0a983024f0845a4f29e6c8bbc492858b7968b39e: Refs #4739, in the screenshot test diffviewer, display the location hint using the processed file path if the expected file does not exist yet.
In a43cae6db06688d289eea9a8474dae563f66bc13: Refs #4739, pass along errors to callback in testEnvironment Piwik API/Controller calls.
In 202671abdf55b4bfaa6d04af8cfebb5990da9b8c: Refs #4739, only pad widgets in screenshot tests when the element is a direct child of a .widget element.
awesome work Benaka there! is this ticket closed?
In e4b1c9c738d8ee9db0bea59a79c4c9373168ccd0: Refs #4739 Run the UI tests build as early as possible so we don't have to wait 30 minutes to see screenshots