Our goal is to create a new open source plugin that will let Super Users easily invalidate any report in Piwik. This will provide a user friendly way to make this frequent action.
Invalidate historical reports
You can invalidate all historical reports for a specific website, or for all websites, by clicking the button "Invalidate historical reports". When you invalidate historical reports, they will be re-processed from the raw logs the next time archiving will run. This is useful when you want to force Piwik to re-process historical data for all reports, for example when:
- you created a new Custom Report and want the Custom Reports to be processed for all historical data
- you created a new Funnel and want your Funnel reports to be processed for all historical data
- you have modified the raw visitor information (for example by importing new visitor logs in the past) and want these changes to raw logs reflected in all your reports.
You are about to invalidate all historical reports, which will cause Piwik to re-process all historical reports the next time archiving run. Re-processing all historical reports may take a long time. Are you sure you want to invalidate all historical reports for [website $WebsiteName|All websites]?
and for segment $segmentName
and for all Segments.
CoreAdminHome.invalidateArchivedReportsis called with those parameters that user confirmed
$websiteCreationTime,todayto invalidate all reports (method
$earliestWebsiteCreationTime,todayto invalidate all reports (use method
periodto false so that all periods are invalidated
Reports have been successfully invalidated. These reports will be reprocessed the next time reports archiving will run.
Invalidate reports (maybe there's a better name?)
From a user POV, I would say we don't want to "invalidate" anything. We really want to "recreate reports" or re-generate", "reconstitute" reports. The action to invalidate is a side-effect hidden under the hood.
I know we can't estimate time or CPU needed but maybe add a generic warning for exemple if the period is more than X days if entire DB is > 1 GB ?
I started to write a response this morning. The only reason I have used the invalidate feature, is because I needed to recreate an historic report, or archive a new segment for a certain period. It's more about the need to force archiving a segment or period. New segments are not archived from the very first date of tracking, for performance reasons. By being able to force archiving a fine grained set of data, we keep the load to a minimum and still be able to provide the requester their reports.
Hi @sgiehl Updated the description/spec, could you take a look?
- For the date parameter, we can use $websiteCreationTime,today to invalidate all reports (method getCreationDateFor)
- If "All websites" was selected, we can use $earliestWebsiteCreationTime,today to invalidate all reports (use method getMinMaxDateAcrossWebsites)
We can do that for the first version, but we defiantly need to do that better way later. If purging old log data is active, invalidating old reports could remove data that can't be reprocessed. So maybe it would be better to check for which day log data is available and leave all older data untouched...
So maybe it would be better to check for which day log data is available and leave all older data untouched...
It should already be implemented, see method
findOlderDateWithLogs, but it would be great to double check this works as expected in this case?
Plugin will soon be published: https://github.com/piwik/plugin-InvalidateReports
Plugin available here: https://plugins.piwik.org/InvalidateReports