Could anyone please have a closer look before merging? Thx.
See previous two comments for feedback. Otherwise, looks great!
Only thing maybe to think about is, could we somehow make this detection faster? I'm worried that at scale, it may be slow to detect (but haven't tested it).
Btw I created a ticket for these many great improvements! http://dev.piwik.org/trac/ticket/4617
Well done Stefan!! :+1:
please revert the 'added detection of LG Browser' you made in your devicedetectionimprovements, because there is no LG Browser. This is against the logic of the PlugIn. This is a LG TV with a branded webkit browser. If we would do that for every vendor who did that, we have thousands of browsers, but there are NOT more than a couple of real browser. The only vendor who has an own browser ist Samsung with Maple, but this engine is discontinued ... ty
ref. this commit: https://github.com/piwik/piwik/commit/121e1510fba1eab30f9ea87b0f71d889214b8f7f
added detection of 'LG Browser' and 'Espial TV Browser' added a bunch of test cases
It is a LG 42LE5500-SA Television in browser mode and not in HbbTV mode, that is the only different !
user_agent: Mozilla/5.0 (DirectFB; U; Linux mips; en) AppleWebKit/531.2+ (KHTML, like Gecko) Safari/531.2+ LG Browser/4.0.10(+SCREEN+TUNER; LGE; 42LE5500-SA; 04.02.02; 0x00000001;); LG NetCast.TV-2010
Has every vendor it's own modified browser engine? LG seems to have an own one. See http://developer.lgappstv.com/TV_HELP/index.jsp?topic=%2Flge.tvsdk.developing.book%2Fhtml%2FDeveloping+Web+App%2FDeveloping+Web+App%2FWeb+Engine.htm If I revert the change the mentioned user agent would be detected as Safari which isn't correct aswell.
I am very familiar with this and I know a lot developer of the most popular TV brands by person. Believe me this is only a modified webkit browser. Every Browser on a Television needs a modification to use the remote control for input and a self written extension for the HbbTV standard. LG in special, because they have an additional remote device for gestures. On a TV there are two modes (HbbTV mode and Browser mode) but it is the same browser only with a different UA String and activated plugin - nothing else... The whole crap in the UA is only for compatibility to view normal websites.
Wir können uns gerne darüber auch noch detaillierter unterhalten :)
Ok. I'll remove it again. Would it make sense to detect all TVs with a 'Generic TV Browser' or something like that?
I would prefer to detect the engine of the Browser, because I think it is important to know whats inside. So you can make improvements to your website or web app based on this information. It helps nobody if there is a 'generic TV browser'... it is a webkit/opera/ant/netfront on a TV with some special input plugins... But you are right - we should think about these things. I updated this plugin because i need to see the browsers / brands / models of all devices that are using my applications in HbbTV mode.
In the old user settings there was browser engine which was much more clear than browser family. I think this is not available in the new detection and there is only browser family. We have much more browser families already than there exist browser engines. I am not really sure where the distinctions between browser families are. I think "generic TV Browser" or "other mobile" can be browser families, but then we should have a browser engine information. In the german translation browser family and browser engine have the same translation which is also quite confusing.
I did also make a mistake when I add Sailfish. I created a Sailfish Browser family, but as far as I could get to know this seems to be a Firefox, so we should also add it to the Firefox family, or other mobile family?
@Solfire Ok. If you have any useful thought about it or any other suggestions for improving the TV detection just let me know. If you want we can also talk about that in IRC or Skype. @hdi-kw Already thought about that, too. Browser families, the way they are right now, are not really interesting at all I think. Browser engines might be much more interesting. @mattab What do you think. Should we remove the browser families again and move the browser engines to devicesdetection instead?
Yes indeed. I would say it is Firefox on a Sailfish OS. For now that is a valid information and enough to know.
We should think about that and make a concept for handle the entire information from this device detection. It is good for now, but i am not really happy. If it is possible we could split the browser section into NAME, ENGINE, VERSION and so on. For the TVs it was easy, because i know exactly what its inside of the different devices.
@sgiehl let us talk next week about this plugin and what we can improve to get better arranged information in the front end and what its need from the detection side. I prefer skype...
Should we remove the browser families again and move the browser engines to devicesdetection instead?
I looked on the demo at Visitors>Devices report. There I see "Browser families" is actually the same as "Browsers without version numbers". So maybe we can simply rename browser families to "Browsers"?
I looked also at "Operating System families" and it seems useful as it is, as it groups all windows together, all linux together, etc.
To me it makes sense (appart from Browser families to rename to Browsers?), but I haven't looked in details. Are there other pending questions / problems?
Browser families is not the same as browsers without version number right now. The browsers are additionally grouped by the useragentparserenhanced. Currently there are only 11 groups, all browsers that are not in a group will be counted as unknown. Shall I change the logic so that every browser not in the list automatically represents his own family. And the browser families will only group mobile/desktop variants and maybe branded versions of specific browsers.
Same for os families. There are around 20 groups and I'm not sure if the grouping is good that way. It's hard to sort in some os. Should all linux based systems be in the group linux? Why is BSD in the Unix group and Mac OS isn't?
There shouldn't be a Unix group at all. Otherwise we would need to classify older Windows versions as NT to be consistent.
Mac OSX is Mac OSX, Linux is Linux and *BSD is BSD. There are a few other OS' we can list separately like Solaris, BeOS (and others) and they deserve to be listed :)
I think mattab is right. Changing the headline / definition helps a lot for understanding. But the information should be expendable (onclick) like in the browser widget.
I'll merge that pull request now. There are already a lot of improvements in it. We now have 1015 tests for the useragent parser (covering nearly all os and browsers that are detectable by now). The detections takes around 2.3 seconds, which means around 2,3 ms per detection. We still need to improve some detections and we need to think about the browser & os families again as they are currently not very useful. Followups to this and other improvements will be discussed here: http://dev.piwik.org/trac/ticket/4215
Excellent work Stefan! I plan to put some work into this to place some extension points to the Device Detection and some settings. I think this will lead into the opposite direction regarding the optimisation efforts discussed here, but we need some tweaks to use the full potential of piwik and would take the downside of this. Just a few Examples: - We would like to add a lot of specific device Information to distinguish devices. Small example: the current Version would detect "Dalvik/1.4.0 (Linux; U; Android 2.3.3; GT-I9100 Build/GINGERBREAD)" as "GT-I9100". But this device is well known as Galaxy S II and the Modell in our opinion should be "GT-I9100 (Galaxy S II)". I guess all this information isn't appreciatet by all piwik users (additional regexes), so my thought was to indroduce a folder where people who love more detailed information can place custom regexes for mobiles etc. which take precedence against the "normal" regexes. - We have several sites where iPhone Apps connect to webservices using UserAgents like this: "Ourapp/2086 CFNetwork/672.0.8 Darwin/14.0.0". Where Ourapp is the appname and 2086 the build number. I would like to introduce a setting for the device detection to translate this into dynamically created Browsers so the Device Section has some insights on used app versions.
Any thoughts on this? Should i file this somewhere else since a closed pull request dosn't some the right place to discuss these additions.
The next step will be to combine UserSettings and DevicesDetection plugins and to enable it by default. Maybe you could create a ticket for the changes you would like to have at http://dev.piwik.org. If you plan to implement anything on your own, feel free to create a pull request.