@diosmosis opened this issue on October 29th 2014

node-webkit is similar to phantomjs in that it uses webkit to render pages and provides ability for JS to script webpages. Using it to run UI tests (if possible) may result in faster test execution. It will also provide the full node.js API, which may be useful in the future.

@tsteur commented on October 29th 2014

Before investing any time in this we should check what is slow when running the UI tests. Is it server time? Is it network, JavaScript execution, CSS, ...? Maybe the application/library is only waiting for some resources to load or ... A first step could be to use remote debugging which is supported by Chrome, Firefox, Safari, to see where the time is actually spent. Maybe also worth profiling some tests/lib/screenshot-testing code.

Would be nice if PhantomJS could be still used so that we can use https://github.com/Huddle/PhantomCSS or similar at some point which is documented and has many more features etc

@tsteur commented on October 30th 2014

I just reduced defaultWaitTime here https://github.com/piwik/piwik/blob/master/tests/lib/screenshot-testing/support/page-renderer.js#L22 from 1 second to 100ms and it reduced the execution time from 15minutes to 7minutes on AWS. Unfortunately, also some tests fail now but it shows there seems to be potential to improve performance? Screenshots were taken about 3-4 times faster. I couldn't really understand what the code does so it might be nonsense.

I also noticed there are often wait times specified for instance along a click. See for example here 10 seconds wait time: https://github.com/piwik/piwik-ui-tests/blob/master/specs/Dashboard_spec.js#L182

If I understand it correct it waits in this case for 10 seconds before it continues to proceed? Maybe we can rather use something like "waitForCssSelectorPresent('.cssSelectorThatIsThereOnceLoaded')" to avoid such wait times? We used this a lot when writing selenium tests but it is more effort to write and maintain tests this way...

Edit: If I understand correct for each click there is a wait time of 1 second at least (there are > 170 clicks), sometimes wait time is up to 10seconds. Also sometimes there is a .wait(3000) which waits for 3 seconds. I think we wouldn't improve performance much with another engine

@mnapoli commented on October 30th 2014

A similar solution is something like "spin" in Behat: remove all waits and retry each step until it works (with a 5-10 seconds timeout).

@diosmosis commented on October 30th 2014

The timeouts are wait times before invoking the next action. This is mostly for waiting for animations I think. Including a way to check for those would mean you wouldn't need to wait for 1s. The 3s timeouts should be specifically for animations (so the screenshots don't look half faded in or whatever). The 10s timeouts are when we (me or matt) gave up on figuring out why the wait was needed.

Having wait times is to avoid "spin"-ing which would incur constant cpu use for something unimportant.

@tsteur commented on October 30th 2014

Would sometimes methods like those help? http://docs.casperjs.org/en/latest/modules/casper.html#waitforselector or http://docs.casperjs.org/en/latest/modules/casper.html#waituntilvisible

That's what we used to do in selenium tests in other projects and worked fine but there we didn't take screenshots. Or maybe we can check like every 50ms whether something is finished / done to avoid cpu usage. Anyway, this seems to be the slow part of the tests and probably not phantomjs. I was only wondering why they are so slow yesterday and I do understand it now

@diosmosis commented on October 30th 2014

I don't think waitforselector will work since the element will be there, just animated. I don't know if being partially visible is considered visible for waituntilvisible.

Or maybe we can check like every 50ms whether something is finished / done to avoid cpu usage.

This is what is done now for AJAX requests, image loading & whatever else I could find a way to do it for. I couldn't figure out how to do it for jquery animations.

Note: I use LightTable sometimes which runs on node-webkit. The entire app is written in clojurescript, so it loads the entire app, the clojurescript compiler, every plugin's JS. Additionally w/ my plugin it loads angularjs + jquery + some other node extensions. And it does this in < 5s. Whereas phantomjs takes ~20-30s to load a Piwik page. Maybe Piwik's JS is ridiculously slow (or it's all the AJAX when loading a page), but there's definitely something amiss.

@diosmosis commented on November 5th 2014

Could also try w/ https://github.com/atom/atom-shell (maintained by github).

@mattab commented on October 4th 2016

We've migrated to Phantomjs2 for now

@mattab commented on February 28th 2017

PhantomJS works well now. Can maybe reopen later if it becomes an issue again

This issue was closed on February 28th 2017
Powered by GitHub Issue Mirror