@chaoaretasty opened this Issue on November 11th 2014

Currently Piwik adds a delay on clicks in order to ensure the links are tracked properly. As Chrome has now shipped navigator.sendBeacon can we get it to use this instead of the delay if the browser supports it?

@tsteur commented on November 11th 2014 Owner

This is nice. Also Firefox implemented it etc

:+1: for giving it a try

@mattab commented on November 12th 2014 Owner

Never heard of it until now! This looks very useful :+1:

@fredli74 commented on December 4th 2014

Tried implementing this into the JS tracker and my browser does fire the events correctly. However it seems the server side is not logging the requests because nothing shows up in the visitor logs.

sendBeacon is POST only and if sending the request as a string it will be transmitted as "text/plain;charset=UTF-8". I saw that piwik POST requests are transmitted as "application/x-www-form-urlencoded; charset=UTF-8". It is possible to use FormData with sendBeacon but it will be transmitted as "multipart/form-data", I did not try this approach. Is there a check or limitation on which content-type the server expects for POST tracking?

@mattab commented on December 4th 2014 Owner
@fredli74 commented on December 5th 2014

Hi, turned on debug tracking and this is what a sendBeacon post request looks like from Chrome (copied as cURL to show results)

curl 'http://piwik.elysian.se/piwik.php' -H 'Cookie: OVHCDN=R2163973028' -H 'Origin: http://soniccharge.com' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_101) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2239.2 Safari/537.36' -H 'Content-Type: text/plain;charset=UTF-8' -H 'Accept: /_' -H 'Cache-Control: max-age=0' -H 'Referer: http://soniccharge.com/download' -H 'Connection: keep-alive' --data-binary 'action_name=Sonic%20Charge%20-%20download&idsite=2&rec=1&r=289293&h=12&m=53&s=13&url=http%3A%2F%2Fsoniccharge.com%2Fdownload&_id=75819894e2914cd2&_idts=1417780393&_idvc=1&_idn=1&_refts=0&_viewts=1417780393&send_image=0&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=1&java=1&gears=0&ag=0&cookie=1&res=2880x1800&gt_ms=585' --compressed


<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] Debug enabled - Input parameters:</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] array (</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] )</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] Nothing to notice =&gt; default behaviour</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] End of the page.</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] array (</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a]   &<a href='/039'>#039</a>;OVHCDN&<a href='/039'>#039</a>; =&gt; &<a href='/039'>#039</a>;R2163973028&<a href='/039'>#039</a>;,</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] )</pre>
<pre>DEBUG Piwik\Common[2014-12-05 11:54:26] [8a63a] Time elapsed: 0.001s</pre>

Looks like the tracker side does not read the posted data only the cookie in the header.

@fredli74 commented on December 5th 2014

Using sendBeacon with FormData was recognized by the tracker. Now unfortunately it generates a lot of unnecessary data as FormData is transmitted as "multipart/form-data". It also required a small URI to FormData converter but it is still code that we can get rid of if the server side tracker is taught to parse "Content-Type: text/plain;charset=UTF-8".

At least the implementation is now working. =)

Form data example

--data-binary $'------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="action_name"\r\n\r\nSonic Charge - download\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="idsite"\r\n\r\n2\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="rec"\r\n\r\n1\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="r"\r\n\r\n745302\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="h"\r\n\r\n13\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="m"\r\n\r\n43\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="s"\r\n\r\n56\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="url"\r\n\r\nhttp://soniccharge.com/download\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="_id"\r\n\r\n75819894e2914cd2\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="_idts"\r\n\r\n1417780393\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="_idvc"\r\n\r\n2\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="_idn"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="_refts"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="_viewts"\r\n\r\n1417781167\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="send_image"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="pdf"\r\n\r\n1\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="qt"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="realp"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="wma"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="dir"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="fla"\r\n\r\n1\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="java"\r\n\r\n1\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="gears"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="ag"\r\n\r\n0\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="cookie"\r\n\r\n1\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="res"\r\n\r\n2880x1800\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T\r\nContent-Disposition: form-data; name="gt_ms"\r\n\r\n691\r\n------WebKitFormBoundaryoWAmoJTVjOZTsg0T--\r\n'

as compared to text/plain version

--data-binary 'action_name=Sonic%20Charge%20-%20download&idsite=2&rec=1&r=289293&h=12&m=53&s=13&url=http%3A%2F%2Fsoniccharge.com%2Fdownload&_id=75819894e2914cd2&_idts=1417780393&_idvc=1&_idn=1&_refts=0&_viewts=1417780393&send_image=0&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=1&java=1&gears=0&ag=0&cookie=1&res=2880x1800&gt_ms=585'
@fredli74 commented on December 6th 2014

Started looking into adding the text/plain support on the server side implementing php://input just to realize that is exactly what bulk requests already does. Why re-invent the wheel... I'll make a new sendBeacon implementation that uses the bulk-request format instead.

@tsteur commented on December 7th 2014 Owner

Just FYI: The bulk tracker does currently not check whether there are scheduled tasks to run etc. We might want to enable this at some point in case we merge this (at the latest once sendBeacon is supported by more browsers).

@fredli74 commented on December 7th 2014

Ah I see. As long as I have visitors without Firefox or Chrome, normal tracking requests till be made as well. I guess this is good enough for now? I have implemented this for all sites I track, mainly because the sendBeacon method (if available) queues the request and returns instantly which avoids round-trip delays on downloads and outlinks.

I do have one concern regarding the server side implementation. When using sendBeacon you do not know exactly when the request is made, it is up to the browser to queue and dispatch these at its earliest convenience. Technically I suspect that multiple requests can reach the server at the same time. I noticed there was a delay in the code to avoid this because multiple visits could be created by mistake. If it is still needed, I am guessing there should be a better solution implemented for this as an arbitrary delay is far from reliable. This should be handled on the server side imho (if it is not done already).

@tsteur commented on December 7th 2014 Owner

Another thing that comes to my mind: When using Content Tracking we might use the piwik.php tracking request to redirect the user to the URL that was clicked originally. This means we do not track with XHR but instead open piwik.php?idsite....&redirecturl=$clickedUrl which in turn redirects the user to $clickedUrl after the request was tracked. This was a requirement to make "right click => open in new window" etc track the click as well. There is a ticket to implement the possibility to disable this behavior: #6265 . We should make this the default behavior (redirect via piwik.php disabled) to make sendBeacon work by default for everyone. Otherwise those redirects would not work. I think to disable those redirects via piwik.php by default is a good idea anyway.

Powered by GitHub Issue Mirror