@tsteur opened this Issue on October 28th 2014 Owner

For example we should hide several generate commands, tests commands, sync screenshots, ... etc if development mode is disabled. It is as easy as adding the following method.

    public function isEnabled()
    {
        return Development::isEnabled();
    }

This makes it much easier for regular users. Maybe we can also enable development mode in GIT by default but disable it in build. So developers will always directly see all commands.

@mattab commented on October 28th 2014 Owner

Maybe we can also enable development mode in GIT by default but disable it in build. So developers will always directly see all commands.

:+1: and FYI there are users deploying Piwik in production from Git (faq)

@tsteur commented on October 28th 2014 Owner

See #6536 re development config enabled by default

@tsteur commented on October 28th 2014 Owner

One problem that I see here...

For example tests:run is a command that is only needed in development. Kinda. It is also needed to run the tests on Travis. But there you do not want to have the development mode flag enabled as the tests should be as close to "production" as possible. Or to say it differently: When development mode is enabled some code paths may be different and we might test code that is not used in production and on the other side we might not test code that is in production but development related code.

What development mode basically does is:

  • Performing some additional checks such as checking if a registered method in WidgetsList actually exists => It's good to perform such checks in tests as it is only additional code that is executed and this way we might detect some errors earlier! It might make a few tests a bit slower but shouldn't be a problem
  • Disabling caching => Here I am a bit worried. On one side we certainly do not want to have anything cached between tests as each test should be atomic. On the other side not using caching would make the tests much slower and we would maybe not catch some errors related to this. Ideally, we disable caching by default and whenever a cache is used in code we test this functionality separately in an integration test. So I think the only remaining problem when enabling development mode in tests is it makes our tests slower.

Usually one would have more Unit and Integration tests (both with caching disabled by default) and less System Tests (with caching enabled) and speed etc would be less of a problem.

So actual question is shall we enable or disable development mode in tests? Maybe it is worth doing one test run to see how much slower the tests will be. If we choose to keep development mode in tests disabled we should not hide tests:run command when development mode is disabled. Here we would rather remove the TestRunner plugin from latest.zip to achieve the same.

As mentioned above I would probably enable development mode in Unit and Integration tests but not for System tests. Alternatively maybe we need a separate config flag for caching or so?

I wrote much more than I wanted, sorry! :)

@mattab commented on October 28th 2014 Owner

So actual question is shall we enable or disable development mode in tests? Maybe it is worth doing one test run to see how much slower the tests will be.

yes it would be useful to know how much slower it makes the tests.

As mentioned above I would probably enable development mode in Unit and Integration tests but not for System tests

:+1:

@tsteur commented on October 28th 2014 Owner

@sgiehl I have prepared a patch to disable all translation commands if development mode is disabled. Haven't committed it yet as I am not sure if they are used in some other scripts? Where maybe development mode is not enabled yet? Can I commit it?

@sgiehl commented on October 29th 2014 Member

I do not see any problems with having the translation commands available only in developer mode.

@tsteur commented on October 29th 2014 Owner

FYI: I did a test run on AWS and did not see a difference in execution time for system tests when development mode is enabled which is a bit surprising. Maybe it does not use the actual config when running the tests? On the other side we do not really have to enable development mode in tests. Not sure what to do. Maybe we simply leave it like it is ^^

@tsteur commented on October 29th 2014 Owner

Closing it for now as done for most commands. Only test related commands are still visible which could be removed by removing the TestRunner plugin in latest.zip

This Issue was closed on October 29th 2014
Powered by GitHub Issue Mirror