@halfdan opened this Issue on July 3rd 2010 Member

Ticket for http://forum.piwik.org/index.php?showtopic=11321

I haven't had the time to investigate this any deeper, but the number of people reporting that Piwik is not tracking any visits is increasing and we should take a look at it.

Quote from mutor:
I am suspecting that the problem is that the tracking URL structure (query string after piwik.php) has changed slightly with the new version (0.6.3) and many hosts don't allow to have an URL as query string variable because of security reasons (injecting malicious code from external websites).

@robocoder commented on July 3rd 2010 Contributor

One of the posts referenced HostGator ... sure enough, I verified they have mod_security and are filtering any URL that contains "http:/" or "https:/" (and escaped variants; note: one forward slash) Reiterating/paraphrasing #564, mod_security users don't do the due diligence to inspect the rules before installing them -- some rules are absolute sh*t. The following return a 403 Forbidden om HostGator:

Vote wontfix.

@mattab commented on July 3rd 2010 Owner

I think we did something earlier, strip http:// or https:// (maybe mailto: as well?) in front of the URLs, and put them back in the tracker.

This could easily be done in trunk, probably would have to wait for a next release coming in maybe 2 weeks?

@robocoder commented on July 3rd 2010 Contributor

We used to strip the scheme/protocol on the server-side.

If we simply strip the protocol on the client-side, we'd lose outlinks that begin with ftp:// (also blocked by HostGator), svn://, mailto:, etc.

To reassemble this on the server implies the workaround would be to separate the schema from the rest of the URL, e.g.,

  • http;//example.com/piwik.php?url=site1.com%2fpath&proto=http&urlref=site2.com%path&protoref=http

Note: this should be affecting the installer (first website setup) and the sitemanager -- as it would explain http://forum.piwik.org/index.php?showtopic=11291

@robocoder commented on July 4th 2010 Contributor

HostGator and users in the forum confirm whitelisting the mod_security rule as the workaround.

@robocoder commented on July 5th 2010 Contributor

At the moment, the HG mod_security rule (purportedly as a security measure against "XSS accounts") affects only $_GET parameters.

@mattab commented on July 21st 2010 Owner

maybe it is possible to white list mod_security in .htaccess?

Otherwise, is it wontfix? if we update piwik.js to not pass http:// , we also have to update the PiwikTracker client.

@robocoder commented on July 23rd 2010 Contributor

I proposed that we would still send the protocol (ftp or http) but as separate parameters. The parameters (containing the protocol) can be optional. If specified, the server concatenates it with the url. This allow for backward compatibility and cases where the browser has cached piwik.js.

Newer versions of mod_security (since 2008?) can no longer be disabled via .htaccess (eg "SecFilterEngine off" causes error code 500 Internal Server Error).

@robocoder commented on August 5th 2010 Contributor

I'm leaning towards making this an installation systemcheck since the mod_security rules broadly impact Piwik, eg tracker, REST API functions that accept URLs, customData that might contain URLs, etc.

@robocoder commented on September 10th 2010 Contributor

The installation system check will test an actual URL to see if it fails. If so, it's an error that can't be skipped.

(An alternative would be to query a remote server to get this server's IP address, then reverse whois to see if the IP range is assigned to HostGator or The Planet. One disadvantage of this approach is that it is host provider specific...not that they should be singled out, but they are the leading source for support requests.)

These workarounds are ugly, but do-able.

  • piwik.js: strip http:// and https:// from url, urlref, link, download, and redirect parameters; server-side: prepend urls with http:// if protocol is missing (to maintain backwards compatibility); make a similar change to PiwikTracker for consistency; similar changes to SitesManager (API.php and SitesManager.js) required
    • doesn't help with customdata that might contain user-specified URLs
  • piwik.js: add option to POST the request to the server; add a similar option to PiwikTracker; in both cases, the default is GET; note: subject to same-origin policy
    • some headers may be suppressed, so Cookie and UserAgent must be sent in the payload
    • some browsers provide limited information, e.g., no version number in Safari's user agent; Accept-Language not accessible via JavaScript -- might be able to fake it with navigator.language, navigator.browserLanguage, navigator.systemLanguage, and/or navigator.userLanguage

References:

@robocoder commented on October 4th 2010 Contributor

(In [3231]) refs #1460 - add .setRequestMethod("POST") to piwik.js

Use cases:

  • workaround GET request length limit
  • workaround mod_security or .htaccess rules that block request containing "ht$

Notes:

  • subject to same origin policy
@robocoder commented on November 8th 2010 Contributor

(In [3301]) fixes #1460 - SitesManager - add/update site uses POST

@mattab commented on November 8th 2010 Owner

Thanks vipsoft, great work on this one. Reopening to remember to document the procedure in a FAQ answer, and also on http://piwik.org/docs/javascript-tracking/

@robocoder commented on November 8th 2010 Contributor

I already documented .setRequestMethod().

I just updated the FAQ http://piwik.org/faq/troubleshooting/#faq_58.

@robocoder commented on November 8th 2010 Contributor

(In [3304]) fixes #1460 - add Installation check for the tracker (GET piwik.php?url=http://example.com); fix the tracker proxy when using POST

@robocoder commented on November 8th 2010 Contributor

(In [3305]) refs #1460 - comment out checkForErrors on Installation welcome screen as it gets a false positive from the inline JavaScript source

@robocoder commented on November 8th 2010 Contributor

(In [3306]) refs #1460 - add retry loop and pause for tracking test to complete

@robocoder commented on November 8th 2010 Contributor

(In [3307]) refs #1460 - typo

@mattab commented on December 24th 2010 Owner

I'll update the warning message to a new FAQ answer about mod_security support

  • disable mod_security
  • or set POST method
@mattab commented on January 13th 2011 Owner

Anthon's message is fine

This Issue was closed on January 13th 2011
Powered by GitHub Issue Mirror