1 2"core" ping 3============ 4 5This mobile-specific ping is intended to provide the most critical 6data in a concise format, allowing for frequent uploads. 7 8Since this ping is used to measure retention, it should be sent 9each time the browser is opened. 10 11Submission will be per the Edge server specification:: 12 13 /submit/telemetry/docId/docType/appName/appVersion/appUpdateChannel/appBuildID 14 15* ``docId`` is a UUID for deduping 16* ``docType`` is “core” 17* ``appName`` is “Fennec” 18* ``appVersion`` is the version of the application (e.g. "46.0a1") 19* ``appUpdateChannel`` is “release”, “beta”, etc. 20* ``appBuildID`` is the build number 21 22Note: Counts below (e.g. search & usage times) are “since the last 23ping”, not total for the whole application lifetime. 24 25Structure: 26 27.. code-block:: js 28 29 { 30 "v": 9, // ping format version 31 "clientId": <string>, // client id, e.g. 32 // "c641eacf-c30c-4171-b403-f077724e848a" 33 "seq": <positive integer>, // running ping counter, e.g. 3 34 "locale": <string>, // application locale, e.g. "en-US" 35 "os": <string>, // OS name. 36 "osversion": <string>, // OS version. 37 "device": <string>, // Build.MANUFACTURER + " - " + Build.MODEL 38 // where manufacturer is truncated to 12 characters 39 // & model is truncated to 19 characters 40 "arch": <string>, // e.g. "arm", "x86" 41 "profileDate": <pos integer>, // Profile creation date in days since 42 // UNIX epoch. 43 "defaultSearch": <string>, // Identifier of the default search engine, 44 // e.g. "yahoo". 45 "distributionId": <string>, // Distribution identifier (optional) 46 "campaignId": <string>, // Adjust's campaign identifier (optional) 47 "created": <string>, // date the ping was created 48 // in local time, "yyyy-mm-dd" 49 "tz": <integer>, // timezone offset (in minutes) of the 50 // device when the ping was created 51 "sessions": <integer>, // number of sessions since last upload 52 "durations": <integer>, // combined duration, in seconds, of all 53 // sessions since last upload 54 "searches": <object>, // Optional, object of search use counts in the 55 // format: { "engine.source": <pos integer> } 56 // e.g.: { "yahoo.suggestion": 3, "other.listitem": 1 } 57 "experiments": [<string>, /* … */], // Optional, array of identifiers 58 // for the active experiments 59 "accessibilityServices": [<string>, /* … */], // Optional, array of identifiers for 60 // enabled accessibility services that 61 // interact with our android products. 62 "flashUsage": <integer>, // number of times flash plugin is played since last upload 63 "defaultBrowser": <boolean> // true if the user has set Firefox as default browser 64 65 } 66 67Field details 68------------- 69 70device 71~~~~~~ 72The ``device`` field is filled in with information specified by the hardware 73manufacturer. As such, it could be excessively long and use excessive amounts 74of limited user data. To avoid this, we limit the length of the field. We're 75more likely have collisions for models within a manufacturer (e.g. "Galaxy S5" 76vs. "Galaxy Note") than we are for shortened manufacturer names so we provide 77more characters for the model than the manufacturer. 78 79distributionId 80~~~~~~~~~~~~~~ 81The ``distributionId`` contains the distribution ID as specified by 82preferences.json for a given distribution. More information on distributions 83can be found `here <https://wiki.mozilla.org/Mobile/Distribution_Files>`_. 84 85It is optional. 86 87campaignId 88~~~~~~~~~~~~~~ 89The ``campaignId`` contains the campaign identifier like '3ly8t0'. 90It's generated by `Adjust <https://docs.adjust.com/en/tracker-generation/#segmenting-users-dynamically-with-campaign-structure-parameters>`_, 91It can only used to identify a campaign, but can't target to a specific user. 92 93It is optional because not everyone has a campaign to begin with. 94 95defaultSearch 96~~~~~~~~~~~~~ 97On Android, this field may be ``null``. To get the engine, we rely on 98``SearchEngineManager#getDefaultEngine``, which searches in several places in 99order to find the search engine identifier: 100 101* Shared Preferences 102* The distribution (if it exists) 103* The localized default engine 104 105If the identifier could not be retrieved, this field is ``null``. If the 106identifier is retrieved, we attempt to create an instance of the search 107engine from the search plugins (in order): 108 109* In the distribution 110* From the localized plugins shipped with the browser 111* The third-party plugins that are installed in the profile directory 112 113If the plugins fail to create a search engine instance, this field is also 114``null``. 115 116This field can also be ``null`` when a custom search engine is set as the 117default. 118 119sessions & durations 120~~~~~~~~~~~~~~~~~~~~ 121On Android, a session is the time when Firefox is focused in the foreground. 122`sessions` tracks the number of sessions since the last upload and 123`durations` is the accumulated duration in seconds of all of these 124sessions. Note that showing a dialog (including a Firefox dialog) will 125take Firefox out of focus & end the current session. 126 127An implementation that records a session when Firefox is completely hidden is 128preferrable (e.g. to avoid the dialog issue above), however, it's more complex 129to implement and so we chose not to, at least for the initial implementation. 130 131profileDate 132~~~~~~~~~~~ 133On Android, this value is created at profile creation time and retrieved or, 134for legacy profiles, taken from the package install time (note: this is not the 135same exact metric as profile creation time but we compromised in favor of ease 136of implementation). 137 138Additionally on Android, this field may be ``null`` in the unlikely event that 139all of the following events occur: 140 141#. The times.json file does not exist 142#. The package install date could not be persisted to disk 143 144The reason we don't just return the package install time even if the date could 145not be persisted to disk is to ensure the value doesn't change once we start 146sending it: we only want to send consistent values. 147 148searches 149~~~~~~~~ 150This describes the search engine usage(count). The format is { "<engine identifier>.<source>"" : count } 151This is optional because the users may have never used the search feature. 152There's no difference if extended telemetry is enabled (prerelease builds) or not. 153 154Possible value : 155 156.. code-block:: js 157 158 { 159 "yahoo.listitem":2, 160 "duckduckgo.listitem":1, 161 "google-nocodes.suggestion":1 162 } 163 164**<engine identifier>**: the identifier of the the search engine. The identifier is collected the way same as desktop. 165we only record the search engine name when: 166 167* builtin or suggested search engines with an ID (includes partner search engines in various distribution scenarios). 168 If it's not a built-in engine, we show "null" or "other". 169* If the user has "Health Report" and core ping enabled. 170 171**<sources>**: it's from one of the 'method's in UI telemetry. Possible values: 172 173* actionbar: the user types in the url bar and hits enter to use the default 174 search engine 175* listitem: the user selects a search engine from the list of secondary search 176 engines at the bottom of the screen 177* suggestion: the user clicks on a search suggestion or, in the case that 178 suggestions are disabled, the row corresponding with the main engine 179 180accessibilityServices 181~~~~~~~~~~~~~~~~~~~~~ 182This describes which accessibility services are currently enabled on user's device and could be interacting with out 183products. This is optional because users often do not have any accessibility services enabled. If present, the value is 184a list of accessibility service ids. 185 186Other parameters 187---------------- 188 189HTTP "Date" header 190~~~~~~~~~~~~~~~~~~ 191This header is used to track the submission date of the core ping in the format 192specified by 193`rfc 2616 sec 14.18 <https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.18>`_, 194et al (e.g. "Tue, 01 Feb 2011 14:00:00 GMT"). 195 196 197Version history 198--------------- 199* v10: added ``defaultBrowser`` to know if the user has set Firefox as default browser 200* v9: changed ``arch`` to contain device arch rather than the one we built against & ``accessibilityServices`` 201* v8: added ``flashUsage`` 202* v7: added ``sessionCount`` & ``sessionDuration`` & ``campaignId`` 203* v6: added ``searches`` 204* v5: added ``created`` & ``tz`` 205* v4: ``profileDate`` will return package install time when times.json is not available 206* v3: added ``defaultSearch`` 207* v2: added ``distributionId`` 208* v1: initial version - shipped in `Fennec 45 <https://bugzilla.mozilla.org/show_bug.cgi?id=1205835>`_. 209 210Notes 211~~~~~ 212 213* ``distributionId`` (v2) actually landed after ``profileDate`` (v4) but was 214 uplifted to 46, whereas ``profileDate`` landed on 47. The version numbers in 215 code were updated to be increasing (bug 1264492) and the version history docs 216 rearranged accordingly. 217 218Android implementation notes 219---------------------------- 220On Android, the uploader has a high probability of delivering the complete data 221for a given client but not a 100% probability. This was a conscious decision to 222keep the code simple. The cases where we can lose data: 223 224* Resetting the field measurements (including incrementing the sequence number) 225 and storing a ping for upload are not atomic. Android can kill our process 226 for memory pressure in between these distinct operations so we can just lose 227 a ping's worth of data. That sequence number will be missing on the server. 228* If we exceed some number of pings on disk that have not yet been uploaded, 229 we remove old pings to save storage space. For those pings, we will lose 230 their data and their sequence numbers will be missing on the server. 231 232Note: we never expect to drop data without also dropping a sequence number so 233we are able to determine when data loss occurs. 234