The goal of this issue is to investigate and solve a Live API request performance issue, when the request contains a high
filter_offset paramater value.
The system runs out of memory, when requesting the following Query:
This API request will return only one row. Instead, Piwik API currently errors out with a memory error (1G exhausted) as it seems to iterate over rows in memory. Instead we should aim to make this API call blazing fast and memory efficient, as users expect it to be.
I had a quick look and noticed it is not easy to fix. It takes probably 1.5 days to find a proper working solution.
Problem 1: If we apply
getLastVisitDetails then the same would be applied again during
GenericFilter and in the end it would not return anything. That's why we currently fetch
filter_limit + filter_offset so the
GenericFilter can apply the limit.
Problem 2: Adding a
LIMIT 5, $offset is not trivial. While the API supports already to specify a limit it does not support to specify a
$offset. Changing this results in touching many classes as it goes down till Segments etc.