1 2"core" ping 3============ 4 5This mobile-specific ping is intended to provide the most critical 6data in a concise format, allowing for frequent uploads. 7 8Since this ping is used to measure retention, it should be sent 9each time the browser is opened. 10 11Submission will be per the Edge server specification:: 12 13 /submit/telemetry/docId/docType/appName/appVersion/appUpdateChannel/appBuildID 14 15* ``docId`` is a UUID for deduping 16* ``docType`` is “core” 17* ``appName`` is “Fennec” 18* ``appVersion`` is the version of the application (e.g. "46.0a1") 19* ``appUpdateChannel`` is “release”, “beta”, etc. 20* ``appBuildID`` is the build number 21 22Note: Counts below (e.g. search & usage times) are “since the last 23ping”, not total for the whole application lifetime. 24 25Structure: 26 27.. code-block:: js 28 29 { 30 "v": 10, // ping format version 31 "clientId": <string>, // client id, e.g. 32 // "c641eacf-c30c-4171-b403-f077724e848a" 33 "seq": <positive integer>, // running ping counter, e.g. 3 34 "locale": <string>, // application locale, e.g. "en-US" 35 "os": <string>, // OS name. 36 "osversion": <string>, // OS version. 37 "device": <string>, // Build.MANUFACTURER + " - " + Build.MODEL 38 // where manufacturer is truncated to 12 characters 39 // & model is truncated to 19 characters 40 "arch": <string>, // e.g. "arm", "x86" 41 "profileDate": <pos integer>, // Profile creation date in days since 42 // UNIX epoch. 43 "defaultSearch": <string>, // Identifier of the default search engine, 44 // e.g. "yahoo". 45 "displayVersion": <string>, // Version displayed to user, e.g. 57.0b3 (optional) 46 "distributionId": <string>, // Distribution identifier (optional) 47 "campaignId": <string>, // Adjust's campaign identifier (optional) 48 "created": <string>, // date the ping was created 49 // in local time, "yyyy-mm-dd" 50 "tz": <integer>, // timezone offset (in minutes) of the 51 // device when the ping was created 52 "sessions": <integer>, // number of sessions since last upload 53 "durations": <integer>, // combined duration, in seconds, of all 54 // sessions since last upload 55 "searches": <object>, // Optional, object of search use counts in the 56 // format: { "engine.source": <pos integer> } 57 // e.g.: { "yahoo.suggestion": 3, "other.listitem": 1 } 58 "experiments": [<string>, /* … */], // Optional, array of identifiers 59 // for the active experiments 60 "accessibilityServices": [<string>, /* … */], // Optional, array of identifiers for 61 // enabled accessibility services that 62 // interact with our android products. 63 "flashUsage": <integer>, // number of times flash plugin is played since last upload 64 "defaultBrowser": <boolean> // true if the user has set Firefox as default browser 65 "bug_1501329_affected": <boolean> // true if Firefox previously used canary clientId 66 // when submitting telemetry 67 "fennec": <object> // Fennec only. 68 // Block of a variety of fields of different types. 69 // Used to understand the usage of Fennec features in the release population 70 // to understand when Fenix is ready to support Fennec users. 71 { 72 "new_tab": { 73 "top_sites_clicked": <int>, // Number of times a Top Site was opened from the Awesome Screen. 74 // Resets after each sent core ping. 75 "pocket_stories_clicked": <int>, // Number of time a Pocket Recommended website was opened 76 // from the Awesome Screen. 77 // Resets after each sent core ping. 78 } 79 "settings_advanced": { 80 "restore_tabs": <boolean>, // State of the "Settings/Advanced/Restore Tabs" setting 81 "show_images": <string>, // State of the "Settings/Advanced/Show images" setting 82 // Value will be be "user-specified" for any non-default values 83 "show_web_fonts": <boolean>, // State of the "Settings/Advanced/Show web fonts" setting 84 }, 85 "settings_general": { 86 "full_screen_browsing": <boolean>, // Current state of the 87 // "Settings/General/Full-screen browsing" setting. 88 "tab_queue": <boolean>, // State of the "Settings/General/Tab queue" setting. 89 "tab_queue_usage_count": <int>, // Number of tabs opened through Tab Queue. 90 // Resets after each sent core ping. 91 "compact_tabs": <boolean>, // State of the "Settings/General/Compact tabs" setting. 92 "homepage": { 93 "custom_homepage": <boolean>, // "true" if not "about:home". 94 "custom_homepage_use_for_newtab": <boolean>, // If the "Settings/General/Home/Also use for new tabs" 95 // setting is enabled. 96 "topsites_enabled": <boolean>, // If the "Settings/General/Home/Top Sites" 97 // setting is set to "Show". 98 "pocket_enabled": <boolean>, // If the "Settings/General/Home/Top Sites/Recommended by Pocket" 99 // setting is enabled. 100 "recent_bookmarks_enabled": <boolean>, // If the "Settings/General/Home/Top Sites/ 101 // Additional Content/Recent Bookmarks" 102 // setting is enabled. 103 "visited_enabled": <boolean>, // If the "Settings/General/Home/Top Sites/Additional Content/Visited" 104 // setting is enabled. 105 bookmarks_enabled": <boolean>, // If the "Settings/General/Home/Bookmarks" setting is set to "Show". 106 "history_enabled": <boolean>, // If the "Settings/General/Home/History" setting is set to "Show". 107 } 108 }, 109 "settings_privacy": { 110 "do_not_track": <boolean>, // If the "Settings/Privacy/Do not track" is enabled. 111 "master_password": <boolean>, // If the "Settings/Privacy/Use master password" is enabled. 112 "master_password_usage_count": <int>, // Number of times the user has entered their master password. 113 // Resets after each sent core ping. 114 }, 115 "settings_notifications": { 116 "product_feature_tips": <boolean>, // If the "Settings/Notifications/Product and feature tips" 117 // setting is enabled. 118 }, 119 "addons": { 120 "active": [addon_id_1, addon_id_2, …, ], // From all installed addons, which ones are active. 121 "disabled": [addon_id_1, addon_id_2, …], // From all installed addons, which ones are disabled. 122 }, 123 "page_options": { 124 "save_as_pdf": <int>, // Number of times the user has used "Page/Save to PDF". 125 // Resets after each sent core ping. 126 "print": <int>, // Number of times the user has used the "Page/Print". 127 // Resets after each sent core ping. 128 "total_added_search_engines": <int>, // The absolute number of user added search engines, 129 // not just those added during this session. 130 "total_sites_pinned_to_topsites": <int>, // The absolute number of times the user has used 131 // the "Pin Site" functionality. 132 // Not just those added during this session. 133 "view_source": <int>, // Number of times the user has used the "Page/View Page Source". 134 // Resets after each sent core ping. 135 "bookmark_with_star": <int>, // The number of times the user has used the "Menu / <Star>". 136 // Resets after each sent core ping. 137 "current_pwas_count": <int>, // On Android >=25 - a positive number of PWAs currently on 138 // homescreen, installed from this app. 139 // On Android <25 - a default of "-1". 140 }, 141 "sync": { 142 "only_over_wifi": <boolean>, // "true" if the "Settings/Sync/Sync only over Wi-Fi" 143 // setting is enabled. 144 // null if the user is not signed into Sync. 145 } 146 } 147 } 148 149Field details 150------------- 151 152device 153~~~~~~ 154The ``device`` field is filled in with information specified by the hardware 155manufacturer. As such, it could be excessively long and use excessive amounts 156of limited user data. To avoid this, we limit the length of the field. We're 157more likely have collisions for models within a manufacturer (e.g. "Galaxy S5" 158vs. "Galaxy Note") than we are for shortened manufacturer names so we provide 159more characters for the model than the manufacturer. 160 161distributionId 162~~~~~~~~~~~~~~ 163The ``distributionId`` contains the distribution ID as specified by 164preferences.json for a given distribution. More information on distributions 165can be found `here <https://wiki.mozilla.org/Mobile/Distribution_Files>`_. 166 167It is optional. 168 169campaignId 170~~~~~~~~~~~~~~ 171The ``campaignId`` contains the campaign identifier like '3ly8t0'. 172It's generated by `Adjust <https://docs.adjust.com/en/tracker-generation/#segmenting-users-dynamically-with-campaign-structure-parameters>`_, 173It can only used to identify a campaign, but can't target to a specific user. 174 175It is optional because not everyone has a campaign to begin with. 176 177defaultSearch 178~~~~~~~~~~~~~ 179On Android, this field may be ``null``. To get the engine, we rely on 180``SearchEngineManager#getDefaultEngine``, which searches in several places in 181order to find the search engine identifier: 182 183* Shared Preferences 184* The distribution (if it exists) 185* The localized default engine 186 187If the identifier could not be retrieved, this field is ``null``. If the 188identifier is retrieved, we attempt to create an instance of the search 189engine from the search plugins (in order): 190 191* In the distribution 192* From the localized plugins shipped with the browser 193* The third-party plugins that are installed in the profile directory 194 195If the plugins fail to create a search engine instance, this field is also 196``null``. 197 198This field can also be ``null`` when a custom search engine is set as the 199default. 200 201sessions & durations 202~~~~~~~~~~~~~~~~~~~~ 203On Android, a session is the time when Firefox is focused in the foreground. 204`sessions` tracks the number of sessions since the last upload and 205`durations` is the accumulated duration in seconds of all of these 206sessions. Note that showing a dialog (including a Firefox dialog) will 207take Firefox out of focus & end the current session. 208 209An implementation that records a session when Firefox is completely hidden is 210preferable (e.g. to avoid the dialog issue above), however, it's more complex 211to implement and so we chose not to, at least for the initial implementation. 212 213profileDate 214~~~~~~~~~~~ 215On Android, this value is created at profile creation time and retrieved or, 216for legacy profiles, taken from the package install time (note: this is not the 217same exact metric as profile creation time but we compromised in favor of ease 218of implementation). 219 220Additionally on Android, this field may be ``null`` in the unlikely event that 221all of the following events occur: 222 223#. The times.json file does not exist 224#. The package install date could not be persisted to disk 225 226The reason we don't just return the package install time even if the date could 227not be persisted to disk is to ensure the value doesn't change once we start 228sending it: we only want to send consistent values. 229 230searches 231~~~~~~~~ 232This describes the search engine usage(count). The format is { "<engine identifier>.<source>"" : count } 233This is optional because the users may have never used the search feature. 234There's no difference if extended telemetry is enabled (prerelease builds) or not. 235 236Possible value : 237 238.. code-block:: js 239 240 { 241 "yahoo.listitem":2, 242 "duckduckgo.listitem":1, 243 "google.suggestion":1 244 } 245 246**<engine identifier>**: the identifier of the the search engine. The identifier is collected the way same as desktop. 247we only record the search engine name when: 248 249* builtin or suggested search engines with an ID (includes partner search engines in various distribution scenarios). 250 If it's not a built-in engine, we show "null" or "other". 251* If the user has "Health Report" and core ping enabled. 252 253**<sources>**: it's from one of the 'method's in UI telemetry. Possible values: 254 255* actionbar: the user types in the url bar and hits enter to use the default 256 search engine 257* listitem: the user selects a search engine from the list of secondary search 258 engines at the bottom of the screen 259* suggestion: the user clicks on a search suggestion or, in the case that 260 suggestions are disabled, the row corresponding with the main engine 261 262accessibilityServices 263~~~~~~~~~~~~~~~~~~~~~ 264This describes which accessibility services are currently enabled on user's device and could be interacting with our 265products. This is optional because users often do not have any accessibility services enabled. If present, the value is 266a list of accessibility service ids. 267 268fennec.new_tab.top_sites_clicked 269~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 270The `top_sites_clicked` field contains the number of times a top site was 271opened from the new tab page since the last time the core ping was sent. 272This counter is reset when the core ping has been sent. 273 274 275Fennec.new_tab.pocket_stories_clicked 276~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 277The `pocket_stories_clicked` contains the number of times a pocket story was 278opened from the new tab page since the last time the core ping was sent. 279This counter is reset when the core ping has been sent. 280 281Fennec.settings_advanced.restore_tabs 282~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 283The `restore_tabs` field contains state of the "Settings/Advanced/Restore Tabs" 284setting. It is true for "Always Restore" and false for "Don’t restore after 285quitting Firefox". 286The value is determined at the time of sending the core ping. 287 288Fennec.settings_advanced.show_images 289~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 290The `show_images` field contains the state of the 291"Settings/Advanced/Show images" settings. 292It is a string value set to "default" if the setting is "Always", or 293"user~specified" for any of the other options. 294The value is determined at the time of sending the core ping. 295 296Fennec.settings_advanced.show_web_fonts 297~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 298The `show_web_fonts` field is a boolean that contains the current state of the 299"Settings/Advanced/Show web fonts" setting. 300The value is determined at the time of sending the core ping. 301 302Fennec.settings_general.full_screen_browsing 303~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 304The `full_screen_browsing` field is a boolean that contains the current state 305of the "Settings/General/Full~screen browsing" setting. 306The value is determined at the time of sending the core ping. 307 308Fennec.settings_general.tab_queue 309~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 310The `tab_queue` field is a boolean that contains the current state of the 311"Settings/General/Tab queue" setting. 312The value is determined at the time of sending the core ping. 313 314Fennec.settings_general.tab_queue_usage_count 315~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 316The `tab_queue_usage_count` is a counter that increments with the number of 317tabs opened through the tab queue. 318It contains the total number of queued tabs opened since the last time the 319Core Ping was sent. 320This counter is reset when the core ping has been sent. 321 322Fennec.settings_general.compact_tabs 323~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 324The `compact_tabs` field is a boolean that contains the current state of the 325"Settings/General/Compact tabs" setting. 326The value is determined at the time of sending the core ping. 327 328Fennec.settings_general.homepage.custom_homepage 329~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 330The `custom_homepage` field is set to true if the homepage is not set to the 331the default `about:home`. 332The value is determined at the time of sending the core ping. 333 334Fennec.settings_general.homepage.custom_homepage_use_for_newtab 335~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 336The `custom_homepage_use_for_newtab` field is set to true if the 337"Settings/General/Home/Also use for new tabs" setting is enabled. 338The value is determined at the time of sending the core ping. 339 340Fennec.settings_general.homepage.topsites_enabled 341~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 342The `topsites_enabled` setting is true if the "Settings/General/Home/Top Sites" 343setting is set to "Show". 344The value is determined at the time of sending the core ping. 345 346Fennec.settings_general.homepage.pocket_enabled 347~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 348The `pocket_enabled` setting is true if the 349"Settings/General/Home/Top Sites/Recommended by Pocket" setting is enabled. 350The value is determined at the time of sending the core ping. 351 352Fennec.settings_general.homepage.recent_bookmarks_enabled 353~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 354The `recent_bookmarks_enabled` setting is true if the 355"Settings/General/Home/Top Sites/Additional Content/Recent Bookmarks" setting 356is enabled. 357The value is determined at the time of sending the core ping. 358 359Fennec.settings_general.homepage.visited_enabled 360~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 361The `visited_enabled` setting is true if the 362"Settings/General/Home/Top Sites/Additional Content/Visited" setting is 363enabled. 364The value is determined at the time of sending the core ping. 365 366Fennec.settings_general.homepage.bookmarks_enabled 367~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 368The `bookmarks_enabled` setting is true if the 369"Settings/General/Home/Bookmarks" setting is set to "Show". 370The value is determined at the time of sending the core ping. 371 372Fennec.settings_general.homepage.history_enabled 373~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 374The `history_enabled` setting is true if the "Settings/General/Home/History" 375setting is set to "Show". 376The value is determined at the time of sending the core ping. 377 378Fennec.settings_privacy.do_not_track 379~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 380The `do_not_track` setting is true if the "Settings/Privacy/Do not track" is 381enabled. 382The value is determined at the time of sending the core ping. 383 384Fennec.settings_privacy.master_password 385~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 386The `master_password` setting is true if the 387"Settings/Privacy/Use master password" is enabled. 388The value is determined at the time of sending the core ping. 389 390Fennec.settings_privacy.master_password_usage_count 391~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 392The `master_password_usage_count` field contains the number of times the user 393has entered their master password since the last time the core ping was sent. 394This counter is reset when the core ping has been sent. 395 396Fennec.settings_notifications.product_feature_tips 397~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 398The `product_feature_tips` setting is true if the 399"Settings/Notifications/Product and feature tips" setting is enabled. 400The value is determined at the time of sending the core ping. 401 402fennec.page_options.save_as_pdf 403~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 404The `save_as_pdf` field contains the number of times the user has used the 405"Page/Save to PDF" feature since the last time the core ping was sent. 406This counter is reset when the core ping has been sent. 407 408fennec.page_options.print 409~~~~~~~~~~~~~~~~~~~~~~~~~ 410The `print` field contains the number of times the user has used the 411"Page/Print" feature since the last time the core ping was sent. 412This counter is reset when the core ping has been sent. 413 414fennec.page_options.total_added_search_engines 415~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 416The `total_added_search_engines` is an absolute value that contains the number 417of search engines the user has added manually. 418The value is determined at the time of sending the core ping and never reset 419to zero. 420 421fennec.page_options.total_sites_pinned_to_topsites 422~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 423The `total_sites_pinned_to_topsites` is an absolute value that contains the 424number of sites the user has pinned to top sites. 425The value is determined at the time of sending the core ping and never reset 426to zero. 427 428fennec.page_options.view_source 429~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 430The `view_source` field contains the number of times the user has used the 431"Page/View Page Source" feature since the last time the core ping was sent. 432This counter is reset when the core ping has been sent. 433 434Fennec.page_options.bookmark_with_star 435~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 436The `bookmark_with_star` field contains the number of times the user has used 437the "Menu / <Star>"" feature since the last time the core ping was sent. 438This counter is reset when the core ping has been sent. 439 440Fennec.page_options.current_pwas_count 441~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 442The `current_pwas_count` field contains the number of currently installed PWAs 443from this application. 444As Android APIs for querying this are only available on Android >=25 for lower 445versions of Android the value of this key will be "-1". 446The value is determined at the time of sending the core ping. 447 448Fennec.sync.only_over_wifi 449~~~~~~~~~~~~~~~~~~~~~~~~~~ 450The `only_over_wifi` setting is true if the 451"Settings/Sync/Sync only over Wi~Fi" setting is enabled. 452The value is determined at the time of sending the core ping. 453If the user is not signed into sync, then this value is set to `null`. 454The value is determined at the time of sending the core ping. 455 456Other parameters 457---------------- 458 459HTTP "Date" header 460~~~~~~~~~~~~~~~~~~ 461This header is used to track the submission date of the core ping in the format 462specified by 463`rfc 2616 sec 14.18 <https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.18>`_, 464et al (e.g. "Tue, 01 Feb 2011 14:00:00 GMT"). 465 466 467Version history 468--------------- 469* v10: added ``bug_1501329_affected`` 470* v9: 471 472 - Apr 2017: changed ``arch`` to contain device arch rather than the one we 473 built against & ``accessibilityServices`` 474 - Dec 2017: added ``defaultBrowser`` to know if the user has set Firefox as 475 default browser (Dec 2017) 476 - May 2018: added (optional) ``displayVersion`` to distinguish Firefox beta versions easily 477 478* v8: added ``flashUsage`` 479* v7: added ``sessionCount`` & ``sessionDuration`` & ``campaignId`` 480* v6: added ``searches`` 481* v5: added ``created`` & ``tz`` 482* v4: ``profileDate`` will return package install time when times.json is not available 483* v3: added ``defaultSearch`` 484* v2: added ``distributionId`` 485* v1: initial version - shipped in `Fennec 45 <https://bugzilla.mozilla.org/show_bug.cgi?id=1205835>`_. 486 487Notes 488~~~~~ 489 490* ``distributionId`` (v2) actually landed after ``profileDate`` (v4) but was 491 uplifted to 46, whereas ``profileDate`` landed on 47. The version numbers in 492 code were updated to be increasing (bug 1264492) and the version history docs 493 rearranged accordingly. 494 495Android implementation notes 496---------------------------- 497On Android, the uploader has a high probability of delivering the complete data 498for a given client but not a 100% probability. This was a conscious decision to 499keep the code simple. The cases where we can lose data: 500 501* Resetting the field measurements (including incrementing the sequence number) 502 and storing a ping for upload are not atomic. Android can kill our process 503 for memory pressure in between these distinct operations so we can just lose 504 a ping's worth of data. That sequence number will be missing on the server. 505* If we exceed some number of pings on disk that have not yet been uploaded, 506 we remove old pings to save storage space. For those pings, we will lose 507 their data and their sequence numbers will be missing on the server. 508 509Note: we never expect to drop data without also dropping a sequence number so 510we are able to determine when data loss occurs. 511