@patriiiiiiiiiick opened this issue on July 1st 2015

Run: ./console core:archive --force-all-websites --force-date-range=2015-06-01,2015-06-25 which for some reason quits in the middle of the list of 20 sites.

Later on, run: ./console core:archive --force-all-websites --force-date-range=2015-05-01,2015-05-31 the output of which gives Will ignore websites and help finish a previous started queue instead. IDs: 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 while the rest of the output will be like Archived website id = 15, period = year, 4 segments, 1352807 visits in years included in: 2015-05-01,2015-05-31, Time elapsed: 495.273s

I hereby conclude that the previous job was finished but with the new parameters (date range). It means that my instruction to archive May was only half honoured while the one to archive June was never actually finished.

@tsteur commented on July 1st 2015

I'm not quite sure if I understand. I think one bug might be that if someone uses --force-all-websites we should not process a previously started queue. This might have been regressed.

Also the queue should maybe not used in general if a --force-data-range is given since it would be not really correct. Eg the queue might have been started with a different data range.

@mattab commented on July 15th 2015

I think one bug might be that if someone uses --force-all-websites we should not process a previously started queue. This might have been regressed.

This was actually changed on purpose, see: #7682 and #7614

Also the queue should maybe not used in general if a --force-data-range is given since it would be not really correct.

Good point! I think it is the right fix, to not use the shared queue when --force-date-range and --force-all-websites are used at the same time.

Maybe there are other parameters that when set, should not use a shared queue?

@tsteur commented on July 16th 2015

This was actually changed on purpose, see: #7682 and #7614

I don't think it's actually good to do it this way. I'd expect --force-all-websites not to resume the queue. At least it should reset it or so as that behaviour can be buggy in some ways.

@tsteur commented on August 13th 2015

By the way there is also a related problem that whenever we run the archiver we set the archiving time at the end: https://github.com/piwik/piwik/blob/2.14.3/core/CronArchive.php#L399

This means when executing the archiver the next time we will just look for that lastSuccesRunTime but we may have actually only processed one site at lastSuccessRunTimestamp, all other sites have a completely different lastSuccessRunTimestamp. This can also lead to some side effects when using multiple archivers at the same time. When using force-siteids etc we should maybe not set the lastSuccessRunTime, only when actually all websites were processed.

@tsteur commented on August 13th 2015

Maybe it might be worth having a look at this in 2.15 as it will be the stable version for a year @mattab

@mattab commented on August 13th 2015

@tsteur thanks for suggestion, moving to 2.15.0 :+1:

Let me know how you see a fix for this, or we can discuss in the PR.

@tsteur commented on August 13th 2015

I'm not working on this one

Powered by GitHub Issue Mirror