@mattab opened this issue on March 21st 2010

Currently, archive.sh simply loops over the periods and triggers archiving for all websites: https://github.com/piwik/piwik/blob/master/misc/cron/archive.sh

because there are memory leaks issues during archiving, see #766, when a Piwik has hundreds or thousands of websites in the system, the memory usage peaks and can lead to ridiculous values.

Until #766 is addressed and as a useful safe guard, we should looop over each website and trigger archiving separately for each. This is a quick fix that will help many users get past the memory usage issues.

@anonymous-piwik-user commented on March 21st 2010

Subscribing to this ticket as I would really appreciate that method. Let me know if I can / should test something :)

@mattab commented on March 31st 2010

(In [2025]) Fixes #1227 the archive script now loops over all websites and triggers a different request for each archive, helping out with the archive memory exhausted issue (refs #766)

@anonymous-piwik-user commented on April 1st 2010

Thanks for your work on this. Overall I would say its better. Its still aching slow running over the sites and my most active site (15k visits / 20k actions per day, out of around 150k per day) is taking 2 GB to process the yearly stats.

@anonymous-piwik-user commented on April 8th 2010

I tried the new archive.sh script, but it didn't work for me, because the list of site id's seems to be utf encoded it doesn't pass the isnumeric test. The convertToUnicode=0 param to the api call doesn't seem to do anything here.

@mattab commented on April 8th 2010

alx, did you apply the full patch? only applying a subset will obviously fail. does it work for you?

@anonymous-piwik-user commented on April 8th 2010

Oops, my bad. I didn't apply the full patch, sorry 'bout that.

This issue was closed on April 8th 2010
Powered by GitHub Issue Mirror