@mattab opened this Issue on July 9th 2015 Owner

Reproduce: go to http://demo.piwik.org/misc/cron/archive.php?token_auth=XXXX (with correct token auth),

  • Expected: same output as when running ./console core:archive in ssh
  • Got instead, a little strange output:
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] ---------------------------
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] INIT
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] Running Piwik 2.14.0 as Super User
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] ---------------------------
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] NOTES
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] - Reports for today will be processed at most every 3000 seconds. You can change this value in Piwik UI > Settings > General Settings.
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
INFO CoreAdminHome[2015-07-09 16:26:17] [601a8] - Archiving was last executed without error 29 min 40s ago
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] - Will process 13 websites with new visits since 29 min 40s , IDs: 1, 7, 13, 20, 25, 26, 33, 34, 35, 36, 43, 47, 48
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] - Will process 1 other websites because some old data reports have been invalidated (eg. using the Log Import script) , IDs: 33
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] ---------------------------
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] START
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Starting Piwik reports archiving...
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Skipped website id 1, already done 31 min 15s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Skipped website id 7, already done 30 min 53s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Skipped website id 13, already done 30 min 42s ago, Time elapsed: 0.001s
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Skipped website id 20, already done 30 min 35s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Skipped website id 25, already done 30 min 32s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Skipped website id 26, already done 30 min 28s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] Will pre-process for website id = 33, period = day, date = last52
INFO CoreAdminHome[2015-07-09 16:26:20] [601a8] - pre-processing all visits
ERROR CoreAdminHome[2015-07-09 16:26:21] [601a8] Empty or invalid response '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <html>  <head>   <title>Index of /misc/cron</title>  </head>  <body> <h1>Index of /misc/cron</h1> <pre><img src="/default_apache_icons/blank.gif" alt="Icon "> <a href="?C=N;O=D">Name</a>                    <a href="?C=M;O=A">Last modified</a>      <a href="?C=S;O=A">Size</a>  <a href="?C=D;O=A">Description</a><hr><img src="/default_apache_icons/back.gif" alt="[DIR]"> <a href="/misc/">Parent Directory</a>                             -    <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="archive.php">archive.php</a>             09-Jul-2015 13:52  1.8K   <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="archive.php_">archive.php_</a>            21-Dec-2012 10:57   26K   <img src="/default_apache_icons/text.gif" alt="[TXT]"> <a href="archive.sh">archive.sh</a>              09-Jul-2015 13:52  1.2K   <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="archive.windows.ps1">archive.windows.ps1</a>     30-Jan-2014 02:38  3.4K   <img src="/default_apache_icons/text.gif" alt="[TXT]"> <a href="archive2.sh">archive2.sh</a>             21-Dec-2012 10:57  2.7K   <img src="/default_apache_icons/text.gif" alt="[TXT]"> <a href="backup.sh">backup.sh</a>               21-Dec-2012 10:57  3.2K   <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="updatetoken.php">updatetoken.php</a>         09-Jul-2015 13:52  1.8K   <hr></pre> </body></html>' for website id 33, Time elapsed: 0.117s, skipping
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped website id 34, already done 30 min 13s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped website id 35, already done 30 min 9s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped website id 36, already done 30 min 5s ago, Time elapsed: 0.000s
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped website id 43, already done 29 min 57s ago, Time elapsed: 0.001s
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Will pre-process for website id = 47, period = day, date = last52
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] - pre-processing all visits
ERROR CoreAdminHome[2015-07-09 16:26:21] [601a8] Empty or invalid response '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <html>  <head>   <title>Index of /misc/cron</title>  </head>  <body> <h1>Index of /misc/cron</h1> <pre><img src="/default_apache_icons/blank.gif" alt="Icon "> <a href="?C=N;O=D">Name</a>                    <a href="?C=M;O=A">Last modified</a>      <a href="?C=S;O=A">Size</a>  <a href="?C=D;O=A">Description</a><hr><img src="/default_apache_icons/back.gif" alt="[DIR]"> <a href="/misc/">Parent Directory</a>                             -    <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="archive.php">archive.php</a>             09-Jul-2015 13:52  1.8K   <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="archive.php_">archive.php_</a>            21-Dec-2012 10:57   26K   <img src="/default_apache_icons/text.gif" alt="[TXT]"> <a href="archive.sh">archive.sh</a>              09-Jul-2015 13:52  1.2K   <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="archive.windows.ps1">archive.windows.ps1</a>     30-Jan-2014 02:38  3.4K   <img src="/default_apache_icons/text.gif" alt="[TXT]"> <a href="archive2.sh">archive2.sh</a>             21-Dec-2012 10:57  2.7K   <img src="/default_apache_icons/text.gif" alt="[TXT]"> <a href="backup.sh">backup.sh</a>               21-Dec-2012 10:57  3.2K   <img src="/default_apache_icons/unknown.gif" alt="[   ]"> <a href="updatetoken.php">updatetoken.php</a>         09-Jul-2015 13:52  1.8K   <hr></pre> </body></html>' for website id 47, Time elapsed: 0.103s, skipping
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped website id 48, already done 29 min 49s ago, Time elapsed: 0.001s
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Done archiving!
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] ---------------------------
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] SUMMARY
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Total visits for today across archived websites: 0
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Archived today's reports for 0 websites
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Archived week/month/year for 0 websites
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped 34 websites: no new visit since the last script execution
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped 11 websites day archiving: existing daily reports are less than 3000 seconds old
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Skipped 0 websites week/month/year archiving: existing periods reports are less than 3600 seconds old
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] Total API requests: 0
INFO CoreAdminHome[2015-07-09 16:26:21] [601a8] done: 0/13 0%, 0 vtoday, 0 wtoday, 0 wperiods, 0 req, 445 ms, 2 errors.
[...]

Notes:

  • Some HTML is output
  • we have some automated test for this, but somehow didn't detect this bug
@Riimu commented on July 11th 2015

The web archiving seems to be broken since the SettingsPiwik::getPiwikUrl() appears to be returning an url with the /misc/cron/ path in the url for the cron script.

As a temporary fix for myself, I added a hard coded return statement to the beginning of that function (in core/SettingsPiwik.php line 168) to return the correct piwik url (e.g. http://www.example.com/ instead of http://www.example.com/misc/cron/).

@nietonfir commented on July 11th 2015

Can confirm that @Riimu's suggestion works.

This Issue was closed on July 12th 2015
Powered by GitHub Issue Mirror