@espringe opened this Issue on July 29th 2014

Log format documentation:
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html

Here's an example of it:

#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken
2014-07-21      01:58:45        DFW3    563     180.76.5.149    GET     d116n0k3gjrs63.cloudfront.net   /robots.txt     301     -       Mozilla/5.0%2520(Windows%2520NT%25205.1;%2520rv:6.0.2)%2520Gecko/20100101%2520Firefox/6.0.2     -       -       Redirect        NLmAqKRfyqQreOK6jMmjVhh8vaUV-CbEM7m_Kta_eoZIxl0VWTEmcQ==        www.moneypot.com        http    188     0.000
#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken
2014-07-21      01:44:46        AMS50   3920    5.134.58.69     GET     d116n0k3gjrs63.cloudfront.net   /img/icons/chart.png    200     https://www.moneypot.com/       Mozilla/5.0%2520(Windows%2520NT%25205.1;%2520rv:30.0)%2520Gecko/20100101%2520Firefox/30.0       -       -       Hit     j2Rusy95IefqGwjXxxfdEp1r53CLEXf7KlcHBHqyHKHN6GnQSrN8-A==        www.moneypot.com        https   329     0.002
#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken
2014-07-21      01:59:06        FRA50   1096    41.227.234.26   GET     d116n0k3gjrs63.cloudfront.net   /css/header.css 200     https://www.moneypot.com/       Mozilla/5.0%2520(Windows%2520NT%25206.2;%2520WOW64;%2520rv:30.0)%2520Gecko/20100101%2520Firefox/30.0    -       -       Miss    0_y6dyIn9nsl1lI1kewczEf8BCktKcQS1hOiuAjiGunQpHnOzpgBfQ==        www.moneypot.com        https   316     0.417
2014-07-21      01:59:06        FRA50   7129    41.227.234.26   GET     d116n0k3gjrs63.cloudfront.net   /img/icons/bitcoin-ic.png       200     https://www.moneypot.com/       Mozilla/5.0%2520(Windows%2520NT%25206.2;%2520WOW64;%2520rv:30.0)%2520Gecko/20100101%2520Firefox/30.0    -       -       Miss    YSRgbarSQU48CjLSnxey1dGb7El85f1z_Ez1MM8fcdEzzBaX_GZS9Q==        www.moneypot.com        https   341     0.415
#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken
2014-07-21      01:44:46        AMS50   2878    5.134.58.69     GET     d116n0k3gjrs63.cloudfront.net   /img/icons/secure.png   200     https://www.moneypot.com/       Mozilla/5.0%2520(Windows%2520NT%25205.1;%2520rv:30.0)%2520Gecko/20100101%2520Firefox/30.0       -       -       Hit     kDqn-NGq9YfCnUhK_6tYPVyMScysHrnFfySMMkWvkV43PHpZMX2Xaw==        www.moneypot.com        https   330     0.002

Since it uses w3c extended format, it is related to #5418

Cloudfront stores the logs in a directory, full of gzipped log files, e.g.

E2S2NV7MT2UOQA.2014-07-29-14.yEaJrFBy.gz
E2S2NV7MT2UOQA.2014-07-29-14.ZkCwYK8H.gz
E2S2NV7MT2UOQA.2014-07-29-14.ZpGqzm7o.gz

So it would be extra nice if one could just specify the containing directory.

@diosmosis commented on December 15th 2014 Member

Fixed in https://github.com/piwik/piwik/pull/6851. Specifying a directory w/ log files is out of scope for the log importer.

This Issue was closed on December 15th 2014
Powered by GitHub Issue Mirror