1Markers 2======= 3 4Markers are packets of arbitrary data that are added to a profile by the Firefox code, usually to 5indicate something important happening at a point in time, or during an interval of time. 6 7Each marker has a name, a category, some common optional information (timing, backtrace, etc.), 8and an optional payload of a specific type (containing arbitrary data relevant to that type). 9 10.. note:: 11 This guide explains C++ markers in depth. To learn more about how to add a 12 marker in JavaScript or Rust, please take a look at their documentation 13 in :doc:`instrumenting-javascript` or :doc:`instrumenting-rust` respectively. 14 15Example 16------- 17 18Short example, details below. 19 20Note: Most marker-related identifiers are in the ``mozilla`` namespace, to be added where necessary. 21 22.. code-block:: c++ 23 24 // Record a simple marker with the category of DOM. 25 PROFILER_MARKER_UNTYPED("Marker Name", DOM); 26 27 // Create a marker with some additional text information. (Be wary of printf!) 28 PROFILER_MARKER_TEXT("Marker Name", JS, MarkerOptions{}, "Additional text information."); 29 30 // Record a custom marker of type `ExampleNumberMarker` (see definition below). 31 PROFILER_MARKER("Number", OTHER, MarkerOptions{}, ExampleNumberMarker, 42); 32 33.. code-block:: c++ 34 35 // Marker type definition. 36 struct ExampleNumberMarker { 37 // Unique marker type name. 38 static constexpr Span<const char> MarkerTypeName() { return MakeStringSpan("number"); } 39 // Data specific to this marker type, serialized to JSON for profiler.firefox.com. 40 static void StreamJSONMarkerData(SpliceableJSONWriter& aWriter, int aNumber) { 41 aWriter.IntProperty("number", aNumber); 42 } 43 // Where and how to display the marker and its data. 44 static MarkerSchema MarkerTypeDisplay() { 45 using MS = MarkerSchema; 46 MS schema(MS::Location::MarkerChart, MS::Location::MarkerTable); 47 schema.SetChartLabel("Number: {marker.data.number}"); 48 schema.AddKeyLabelFormat("number", "Number", MS::Format::Number); 49 return schema; 50 } 51 }; 52 53 54How to Record Markers 55--------------------- 56 57Header to Include 58^^^^^^^^^^^^^^^^^ 59 60If the compilation unit only defines and records untyped, text, and/or its own markers, include 61`the main profiler markers header <https://searchfox.org/mozilla-central/source/tools/profiler/public/ProfilerMarkers.h>`_: 62 63.. code-block:: c++ 64 65 #include "mozilla/ProfilerMarkers.h" 66 67If it also records one of the other common markers defined in 68`ProfilerMarkerTypes.h <https://searchfox.org/mozilla-central/source/tools/profiler/public/ProfilerMarkerTypes.h>`_, 69include that one instead: 70 71.. code-block:: c++ 72 73 #include "mozilla/ProfilerMarkerTypes.h" 74 75And if it uses any other profiler functions (e.g., labels), use 76`the main Gecko Profiler header <https://searchfox.org/mozilla-central/source/tools/profiler/public/GeckoProfiler.h>`_ 77instead: 78 79.. code-block:: c++ 80 81 #include "GeckoProfiler.h" 82 83The above works from source files that end up in libxul, which is true for the majority 84of Firefox source code. But some files live outside of libxul, such as mfbt, in which 85case the advice is the same but the equivalent headers are from the Base Profiler instead: 86 87.. code-block:: c++ 88 89 #include "mozilla/BaseProfilerMarkers.h" // Only own/untyped/text markers 90 #include "mozilla/BaseProfilerMarkerTypes.h" // Only common markers 91 #include "BaseProfiler.h" // Markers and other profiler functions 92 93Untyped Markers 94^^^^^^^^^^^^^^^ 95 96Untyped markers don't carry any information apart from common marker data: 97Name, category, options. 98 99.. code-block:: c++ 100 101 PROFILER_MARKER_UNTYPED( 102 // Name, and category pair. 103 "Marker Name", OTHER, 104 // Marker options, may be omitted if all defaults are acceptable. 105 MarkerOptions(MarkerStack::Capture(), ...)); 106 107``PROFILER_MARKER_UNTYPED`` is a macro that simplifies the use of the main 108``profiler_add_marker`` function, by adding the appropriate namespaces, and a surrounding 109``#ifdef MOZ_GECKO_PROFILER`` guard. 110 1111. Marker name 112 The first argument is the name of this marker. This will be displayed in most places 113 the marker is shown. It can be a literal C string, or any dynamic string object. 1142. `Category pair name <https://searchfox.org/mozilla-central/define?q=M_174bb0de187ee7d9>`_ 115 Choose a category + subcategory from the `the list of categories <https://searchfox.org/mozilla-central/define?q=M_174bb0de187ee7d9>`_. 116 This is the second parameter of each ``SUBCATEGORY`` line, for instance ``LAYOUT_Reflow``. 117 (Internally, this is really a `MarkerCategory <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerCategory>`_ 118 object, in case you need to construct it elsewhere.) 1193. `MarkerOptions <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerOptions>`_ 120 See the options below. It can be omitted if there are no other arguments, ``{}``, or 121 ``MarkerOptions()`` (no specified options); only one of the following option types 122 alone; or ``MarkerOptions(...)`` with one or more of the following options types: 123 124 * `MarkerThreadId <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerThreadId>`_ 125 Rarely used, as it defaults to the current thread. Otherwise it specifies the target 126 "thread id" (aka "track") where the marker should appear; This may be useful when 127 referring to something that happened on another thread (use ``profiler_current_thread_id()`` 128 from the original thread to get its id); or for some important markers, they may be 129 sent to the "main thread", which can be specified with ``MarkerThreadId::MainThread()``. 130 * `MarkerTiming <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerTiming>`_ 131 This specifies an instant or interval of time. It defaults to the current instant if 132 left unspecified. Otherwise use ``MarkerTiming::InstantAt(timestamp)`` or 133 ``MarkerTiming::Interval(ts1, ts2)``; timestamps are usually captured with 134 ``TimeStamp::Now()``. It is also possible to record only the start or the end of an 135 interval, pairs of start/end markers will be matched by their name. *Note: The 136 upcoming "marker sets" feature will make this pairing more reliable, and also 137 allow more than two markers to be connected*. 138 * `MarkerStack <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerStack>`_ 139 By default, markers do not record a "stack" (or "backtrace"). To record a stack at 140 this point, in the most efficient manner, specify ``MarkerStack::Capture()``. To 141 record a previously captured stack, first store a stack into a 142 ``UniquePtr<ProfileChunkedBuffer>`` with ``profiler_capture_backtrace()``, then pass 143 it to the marker with ``MarkerStack::TakeBacktrace(std::move(stack))``. 144 * `MarkerInnerWindowId <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerInnerWindowId>`_ 145 If you have access to an "inner window id", consider specifying it as an option, to 146 help profiler.firefox.com to classify them by tab. 147 148Text Markers 149^^^^^^^^^^^^ 150 151Text markers are very common, they carry an extra text as a fourth argument, in addition to 152the marker name. Use the following macro: 153 154.. code-block:: c++ 155 156 PROFILER_MARKER_TEXT( 157 // Name, category pair, options. 158 "Marker Name", OTHER, {}, 159 // Text string. 160 "Here are some more details." 161 ); 162 163As useful as it is, using an expensive ``printf`` operation to generate a complex text 164comes with a variety of issues string. It can leak potentially sensitive information 165such as URLs can be leaked during the profile sharing step. profiler.firefox.com cannot 166access the information programmatically. It won't get the formatting benefits of the 167built-in marker schema. Please consider using a custom marker type to separate and 168better present the data. 169 170Other Typed Markers 171^^^^^^^^^^^^^^^^^^^ 172 173From C++ code, a marker of some type ``YourMarker`` (details about type definition follow) can be 174recorded like this: 175 176.. code-block:: c++ 177 178 PROFILER_MARKER( 179 "YourMarker name", OTHER, 180 MarkerOptions(MarkerTiming::IntervalUntilNowFrom(someStartTimestamp), 181 MarkerInnerWindowId(innerWindowId))), 182 YourMarker, "some string", 12345, "http://example.com", someTimeStamp); 183 184After the first three common arguments (like in ``PROFILER_MARKER_UNTYPED``), there are: 185 1864. The marker type, which is the name of the C++ ``struct`` that defines that type. 1875. A variadic list of type-specific argument. They must match the number of, and must 188 be convertible to, ``StreamJSONMarkerData`` parameters as specified in the marker type definition. 189 190"Auto" Scoped Interval Markers 191^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 192 193To capture time intervals around some important operations, it is common to store a timestamp, do the work, 194and then record a marker, e.g.: 195 196.. code-block:: c++ 197 198 void DoTimedWork() { 199 TimeStamp start = TimeStamp::Now(); 200 DoWork(); 201 PROFILER_MARKER_TEXT("Timed work", OTHER, MarkerTiming::IntervalUntilNowFrom(start), "Details"); 202 } 203 204`RAII <https://en.cppreference.com/w/cpp/language/raii>`_ objects automate this, by recording the time 205when the object is constructed, and later recording the marker when the object is destroyed at the end 206of its C++ scope. 207This is especially useful if there are multiple scope exit points. 208 209``AUTO_PROFILER_MARKER_TEXT`` is `the only one implemented <https://searchfox.org/mozilla-central/define?q=M_ac7b392646edf5a5>`_ at this time. 210 211.. code-block:: c++ 212 213 void MaybeDoTimedWork(bool aDoIt) { 214 AUTO_PROFILER_MARKER_TEXT("Timed work", OTHER, "Details"); 215 if (!aDoIt) { /* Marker recorded here... */ return; } 216 DoWork(); 217 /* ... or here. */ 218 } 219 220Note that these RAII objects only record one marker. In some situation, a very long 221operation could be missed if it hasn't completed by the end of the profiling session. 222In this case, consider recording two distinct markers, using 223``MarkerTiming::IntervalStart()`` and ``MarkerTiming::IntervalEnd()``. 224 225Where to Define New Marker Types 226-------------------------------- 227 228The first step is to determine the location of the marker type definition: 229 230* If this type is only used in one function, or a component, it can be defined in a 231 local common place relative to its use. 232* For a more common type that could be used from multiple locations: 233 234 * If there is no dependency on XUL, it can be defined in the Base Profiler, which can 235 be used in most locations in the codebase: 236 `mozglue/baseprofiler/public/BaseProfilerMarkerTypes.h <https://searchfox.org/mozilla-central/source/mozglue/baseprofiler/public/BaseProfilerMarkerTypes.h>`__ 237 238 * However, if there is a XUL dependency, then it needs to be defined in the Gecko Profiler: 239 `tools/profiler/public/ProfilerMarkerTypes.h <https://searchfox.org/mozilla-central/source/tools/profiler/public/ProfilerMarkerTypes.h>`__ 240 241.. _how-to-define-new-marker-types: 242 243How to Define New Marker Types 244------------------------------ 245 246Each marker type must be defined once and only once. 247The definition is a C++ ``struct``, its identifier is used when recording 248markers of that type in C++. 249By convention, the suffix "Marker" is recommended to better distinguish them 250from non-profiler entities in the source. 251 252.. code-block:: c++ 253 254 struct YourMarker { 255 256Marker Type Name 257^^^^^^^^^^^^^^^^ 258 259A marker type must have a unique name, it is used to keep track of the type of 260markers in the profiler storage, and to identify them uniquely on profiler.firefox.com. 261(It does not need to be the same as the ``struct``'s name.) 262 263This name is defined in a special static member function ``MarkerTypeName``: 264 265.. code-block:: c++ 266 267 // … 268 static constexpr Span<const char> MarkerTypeName() { 269 return MakeStringSpan("YourMarker"); 270 } 271 272Marker Type Data 273^^^^^^^^^^^^^^^^ 274 275All markers of any type have some common data: A name, a category, options like 276timing, etc. as previously explained. 277 278In addition, a certain marker type may carry zero of more arbitrary pieces of 279information, and they are always the same for all markers of that type. 280 281These are defined in a special static member function ``StreamJSONMarkerData``. 282 283The first function parameters is always ``SpliceableJSONWriter& aWriter``, 284it will be used to stream the data as JSON, to later be read by 285profiler.firefox.com. 286 287.. code-block:: c++ 288 289 // … 290 static void StreamJSONMarkerData(SpliceableJSONWriter& aWriter, 291 292The following function parameters is how the data is received as C++ objects 293from the call sites. 294 295* Most C/C++ `POD (Plain Old Data) <https://en.cppreference.com/w/cpp/named_req/PODType>`_ 296 and `trivially-copyable <https://en.cppreference.com/w/cpp/named_req/TriviallyCopyable>`_ 297 types should work as-is, including ``TimeStamp``. 298* Character strings should be passed using ``const ProfilerString8View&`` (this handles 299 literal strings, and various ``std::string`` and ``nsCString`` types, and spans with or 300 without null terminator). Use ``const ProfilerString16View&`` for 16-bit strings such as 301 ``nsString``. 302* Other types can be used if they define specializations for ``ProfileBufferEntryWriter::Serializer`` 303 and ``ProfileBufferEntryReader::Deserializer``. You should rarely need to define new 304 ones, but if needed see how existing specializations are written, or contact the 305 `perf-tools team for help <https://chat.mozilla.org/#/room/#profiler:mozilla.org>`_. 306 307Passing by value or by reference-to-const is recommended, because arguments are serialized 308in binary form (i.e., there are no optimizable ``move`` operations). 309 310For example, here's how to handle a string, a 64-bit number, another string, and 311a timestamp: 312 313.. code-block:: c++ 314 315 // … 316 const ProfilerString8View& aString, 317 const int64_t aBytes, 318 const ProfilerString8View& aURL, 319 const TimeStamp& aTime) { 320 321Then the body of the function turns these parameters into a JSON stream. 322 323When this function is called, the writer has just started a JSON object, so 324everything that is written should be a named object property. Use 325``SpliceableJSONWriter`` functions, in most cases ``...Property`` functions 326from its parent class ``JSONWriter``: ``NullProperty``, ``BoolProperty``, 327``IntProperty``, ``DoubleProperty``, ``StringProperty``. (Other nested JSON 328types like arrays or objects are not supported by the profiler.) 329 330As a special case, ``TimeStamps`` must be streamed using ``aWriter.TimeProperty(timestamp)``. 331 332The property names will be used to identify where each piece of data is stored and 333how it should be displayed on profiler.firefox.com (see next section). 334 335Here's how the above functions parameters could be streamed: 336 337.. code-block:: c++ 338 339 // … 340 aWriter.StringProperty("myString", aString); 341 aWriter.IntProperty("myBytes", aBytes); 342 aWriter.StringProperty("myURL", aURL); 343 aWriter.TimeProperty("myTime", aTime); 344 } 345 346.. _marker-type-display-schema: 347 348Marker Type Display Schema 349^^^^^^^^^^^^^^^^^^^^^^^^^^ 350 351Now that we have defined how to stream type-specific data (from Firefox to 352profiler.firefox.com), we need to describe where and how this data will be 353displayed on profiler.firefox.com. 354 355The static member function ``MarkerTypeDisplay`` returns an opaque ``MarkerSchema`` 356object, which will be forwarded to profiler.firefox.com. 357 358.. code-block:: c++ 359 360 // … 361 static MarkerSchema MarkerTypeDisplay() { 362 363The ``MarkerSchema`` type will be used repeatedly, so for convenience we can define 364a local type alias: 365 366.. code-block:: c++ 367 368 // … 369 using MS = MarkerSchema; 370 371First, we construct the ``MarkerSchema`` object to be returned at the end. 372 373One or more constructor arguments determine where this marker will be displayed in 374the profiler.firefox.com UI. See the `MarkerSchema::Location enumeration for the 375full list <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerSchema%3A%3ALocation>`_. 376 377Here is the most common set of locations, showing markers of that type in both the 378Marker Chart and the Marker Table panels: 379 380.. code-block:: c++ 381 382 // … 383 MS schema(MS::Location::MarkerChart, MS::Location::MarkerTable); 384 385Some labels can optionally be specified, to display certain information in different 386locations: ``SetChartLabel``, ``SetTooltipLabel``, and ``SetTableLabel``; or 387``SetAllLabels`` to define all of them the same way. 388 389The arguments is a string that may refer to marker data within braces: 390 391* ``{marker.name}``: Marker name. 392* ``{marker.data.X}``: Type-specific data, as streamed with property name "X" from ``StreamJSONMarkerData`` (e.g., ``aWriter.IntProperty("X", aNumber);`` 393 394For example, here's how to set the Marker Chart label to show the marker name and the 395``myBytes`` number of bytes: 396 397.. code-block:: c++ 398 399 // … 400 schema.SetChartLabel("{marker.name} – {marker.data.myBytes}"); 401 402profiler.firefox.com will apply the label with the data in a consistent manner. For 403example, with this label definition, it could display marker information like the 404following in the Firefox Profiler's Marker Chart: 405 406 * "Marker Name – 10B" 407 * "Marker Name – 25.204KB" 408 * "Marker Name – 512.54MB" 409 410For implementation details on this processing, see `src/profiler-logic/marker-schema.js <https://github.com/firefox-devtools/profiler/blob/main/src/profile-logic/marker-schema.js>`_ 411in the profiler's front-end. 412 413Next, define the main display of marker data, which will appear in the Marker 414Chart tooltips and the Marker Table sidebar. 415 416Each row may either be: 417 418* A dynamic key-value pair, using one of the ``MarkerSchema::AddKey...`` functions. Each function is given: 419 420 * Key: Element property name as streamed in ``StreamJSONMarkerData``. 421 * Label: Optional prefix. Defaults to the key name. 422 * Format: How to format the data element value, see `MarkerSchema::Format for details <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerSchema%3A%3AFormat>`_. 423 * Searchable: Optional boolean, indicates if the value is used in searches, defaults to false. 424 425* Or a fixed label and value strings, using ``MarkerSchema::AddStaticLabelValue``. 426 427.. code-block:: c++ 428 429 // … 430 schema.AddKeyLabelFormatSearchable( 431 "myString", "My String", MS::Format::String, true); 432 schema.AddKeyLabelFormat( 433 "myBytes", "My Bytes", MS::Format::Bytes); 434 schema.AddKeyLabelFormat( 435 "myUrl", "My URL", MS::Format::Url); 436 schema.AddKeyLabelFormat( 437 "myTime", "Event time", MS::Format::Time); 438 439Finally the ``schema`` object is returned from the function: 440 441.. code-block:: c++ 442 443 // … 444 return schema; 445 } 446 447Any other ``struct`` member function is ignored. There could be utility functions used by the above 448compulsory functions, to make the code clearer. 449 450And that is the end of the marker definition ``struct``. 451 452.. code-block:: c++ 453 454 // … 455 }; 456 457Performance Considerations 458-------------------------- 459 460During profiling, it is best to reduce the amount of work spent doing profiler 461operations, as they can influence the performance of the code that you want to profile. 462 463Whenever possible, consider passing simple types to marker functions, such that 464``StreamJSONMarkerData`` will do the minimum amount of work necessary to serialize 465the marker type-specific arguments to its internal buffer representation. POD types 466(numbers) and strings are the easiest and cheapest to serialize. Look at the 467corresponding ``ProfileBufferEntryWriter::Serializer`` specializations if you 468want to better understand the work done. 469 470Avoid doing expensive operations when recording markers. E.g.: ``printf`` of 471different things into a string, or complex computations; instead pass the 472``printf``/computation arguments straight through to the marker function, so that 473``StreamJSONMarkerData`` can do the expensive work at the end of the profiling session. 474 475Marker Architecture Description 476------------------------------- 477 478The above sections should give all the information needed for adding your own marker 479types. However, if you are wanting to work on the marker architecture itself, this 480section will describe how the system works. 481 482TODO: 483 * Briefly describe the buffer and serialization. 484 * Describe the template strategy for generating marker types 485 * Describe the serialization and link to profiler front-end docs on marker processing (if they exist) 486