1 /* 2 * Licensed to the Apache Software Foundation (ASF) under one 3 * or more contributor license agreements. See the NOTICE file 4 * distributed with this work for additional information 5 * regarding copyright ownership. The ASF licenses this file 6 * to you under the Apache License, Version 2.0 (the 7 * "License"); you may not use this file except in compliance 8 * with the License. You may obtain a copy of the License at 9 * 10 * http://www.apache.org/licenses/LICENSE-2.0 11 * 12 * Unless required by applicable law or agreed to in writing, software 13 * distributed under the License is distributed on an "AS IS" BASIS, 14 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 15 * See the License for the specific language governing permissions and 16 * limitations under the License. 17 */ 18 19 /** 20 <h1>Metrics 2.0</h1> 21 <ul id="toc"> 22 <li><a href="#overview">Overview</a></li> 23 <li><a href="#gettingstarted">Getting Started</a></li> 24 <li><a href="#config">Configuration</a></li> 25 <li><a href="#filtering">Metrics Filtering</a></li> 26 <li><a href="#instrumentation">Metrics Instrumentation Strategy</a></li> 27 <li><a href="#migration">Migration from previous system</a></li> 28 </ul> 29 <h2><a name="overview">Overview</a></h2> 30 <p>This package provides a framework for metrics instrumentation 31 and publication. 32 </p> 33 34 <p>The framework provides a variety of ways to implement metrics 35 instrumentation easily via the simple 36 {@link org.apache.hadoop.metrics2.MetricsSource} interface 37 or the even simpler and more concise and declarative metrics annotations. 38 The consumers of metrics just need to implement the simple 39 {@link org.apache.hadoop.metrics2.MetricsSink} interface. Producers 40 register the metrics sources with a metrics system, while consumers 41 register the sinks. A default metrics system is provided to marshal 42 metrics from sources to sinks based on (per source/sink) configuration 43 options. All the metrics are also published and queryable via the 44 standard JMX MBean interface. This document targets the framework users. 45 Framework developers could also consult the 46 <a href="http://wiki.apache.org/hadoop/HADOOP-6728-MetricsV2">design 47 document</a> for architecture and implementation notes. 48 </p> 49 <h3>Sub-packages</h3> 50 <dl> 51 <dt><code>org.apache.hadoop.metrics2.annotation</code></dt> 52 <dd>Public annotation interfaces for simpler metrics instrumentation. 53 </dd> 54 <dt><code>org.apache.hadoop.metrics2.impl</code></dt> 55 <dd>Implementation classes of the framework for interface and/or 56 abstract classes defined in the top-level package. Sink plugin code 57 usually does not need to reference any class here. 58 </dd> 59 <dt> <code>org.apache.hadoop.metrics2.lib</code></dt> 60 <dd>Convenience classes for implementing metrics sources, including the 61 Mutable[{@link org.apache.hadoop.metrics2.lib.MutableGauge Gauge}*| 62 {@link org.apache.hadoop.metrics2.lib.MutableCounter Counter}*| 63 {@link org.apache.hadoop.metrics2.lib.MutableStat Stat}] and 64 {@link org.apache.hadoop.metrics2.lib.MetricsRegistry}. 65 </dd> 66 <dt> <code>org.apache.hadoop.metrics2.filter</code></dt> 67 <dd>Builtin metrics filter implementations include the 68 {@link org.apache.hadoop.metrics2.filter.GlobFilter} and 69 {@link org.apache.hadoop.metrics2.filter.RegexFilter}. 70 </dd> 71 <dt><code>org.apache.hadoop.metrics2.source</code></dt> 72 <dd>Builtin metrics source implementations including the 73 {@link org.apache.hadoop.metrics2.source.JvmMetrics}. 74 </dd> 75 <dt> <code>org.apache.hadoop.metrics2.sink</code></dt> 76 <dd>Builtin metrics sink implementations including the 77 {@link org.apache.hadoop.metrics2.sink.FileSink}. 78 </dd> 79 <dt> <code>org.apache.hadoop.metrics2.util</code></dt> 80 <dd>General utilities for implementing metrics sinks etc., including the 81 {@link org.apache.hadoop.metrics2.util.MetricsCache}. 82 </dd> 83 </dl> 84 85 <h2><a name="gettingstarted">Getting started</a></h2> 86 <h3>Implementing metrics sources</h3> 87 <table width="99%" border="1" cellspacing="0" cellpadding="4"> 88 <tbody> 89 <tr> 90 <th>Using annotations</th><th>Using MetricsSource interface</th> 91 </tr> 92 <tr><td> 93 <pre> 94 @Metrics(context="MyContext") 95 class MyStat { 96 97 @Metric("My metric description") 98 public int getMyMetric() { 99 return 42; 100 } 101 }</pre></td><td> 102 <pre> 103 class MyStat implements MetricsSource { 104 105 @Override 106 public void getMetrics(MetricsCollector collector, boolean all) { 107 collector.addRecord("MyStat") 108 .setContext("MyContext") 109 .addGauge(info("MyMetric", "My metric description"), 42); 110 } 111 } 112 </pre> 113 </td> 114 </tr> 115 </tbody> 116 </table> 117 <p>In this example we introduced the following:</p> 118 <dl> 119 <dt><em>@Metrics</em></dt> 120 <dd>The {@link org.apache.hadoop.metrics2.annotation.Metrics} annotation is 121 used to indicate that the class is a metrics source. 122 </dd> 123 124 <dt><em>MyContext</em></dt> 125 <dd>The optional context name typically identifies either the 126 application, or a group of modules within an application or 127 library. 128 </dd> 129 130 <dt><em>MyStat</em></dt> 131 <dd>The class name is used (by default, or specified by name=value parameter 132 in the Metrics annotation) as the metrics record name for 133 which a set of metrics are to be reported. For example, you could have a 134 record named "CacheStat" for reporting a number of statistics relating to 135 the usage of some cache in your application.</dd> 136 137 <dt><em>@Metric</em></dt> 138 <dd>The {@link org.apache.hadoop.metrics2.annotation.Metric} annotation 139 identifies a particular metric, which in this case, is the 140 result of the method call getMyMetric of the "gauge" (default) type, 141 which means it can vary in both directions, compared with a "counter" 142 type, which can only increase or stay the same. The name of the metric 143 is "MyMetric" (inferred from getMyMetric method name by default.) The 42 144 here is the value of the metric which can be substituted with any valid 145 java expressions. 146 </dd> 147 </dl> 148 <p>Note, the {@link org.apache.hadoop.metrics2.MetricsSource} interface is 149 more verbose but more flexible, 150 allowing generated metrics names and multiple records. In fact, the 151 annotation interface is implemented with the MetricsSource interface 152 internally.</p> 153 <h3>Implementing metrics sinks</h3> 154 <pre> 155 public class MySink implements MetricsSink { 156 public void putMetrics(MetricsRecord record) { 157 System.out.print(record); 158 } 159 public void init(SubsetConfiguration conf) {} 160 public void flush() {} 161 }</pre> 162 <p>In this example there are three additional concepts:</p> 163 <dl> 164 <dt><em>record</em></dt> 165 <dd>This object corresponds to the record created in metrics sources 166 e.g., the "MyStat" in previous example. 167 </dd> 168 <dt><em>conf</em></dt> 169 <dd>The configuration object for the sink instance with prefix removed. 170 So you can get any sink specific configuration using the usual 171 get* method. 172 </dd> 173 <dt><em>flush</em></dt> 174 <dd>This method is called for each update cycle, which may involve 175 more than one record. The sink should try to flush any buffered metrics 176 to its backend upon the call. But it's not required that the 177 implementation is synchronous. 178 </dd> 179 </dl> 180 <p>In order to make use our <code>MyMetrics</code> and <code>MySink</code>, 181 they need to be hooked up to a metrics system. In this case (and most 182 cases), the <code>DefaultMetricsSystem</code> would suffice. 183 </p> 184 <pre> 185 DefaultMetricsSystem.initialize("test"); // called once per application 186 DefaultMetricsSystem.register(new MyStat());</pre> 187 <h2><a name="config">Metrics system configuration</a></h2> 188 <p>Sinks are usually specified in a configuration file, say, 189 "hadoop-metrics2-test.properties", as: 190 </p> 191 <pre> 192 test.sink.mysink0.class=com.example.hadoop.metrics.MySink</pre> 193 <p>The configuration syntax is:</p> 194 <pre> 195 [prefix].[source|sink|jmx|].[instance].[option]</pre> 196 <p>In the previous example, <code>test</code> is the prefix and 197 <code>mysink0</code> is an instance name. 198 <code>DefaultMetricsSystem</code> would try to load 199 <code>hadoop-metrics2-[prefix].properties</code> first, and if not found, 200 try the default <code>hadoop-metrics2.properties</code> in the class path. 201 Note, the <code>[instance]</code> is an arbitrary name to uniquely 202 identify a particular sink instance. The asterisk (<code>*</code>) can be 203 used to specify default options. 204 </p> 205 <p>Consult the metrics instrumentation in jvm, rpc, hdfs and mapred, etc. 206 for more examples. 207 </p> 208 209 <h2><a name="filtering">Metrics Filtering</a></h2> 210 <p>One of the features of the default metrics system is metrics filtering 211 configuration by source, context, record/tags and metrics. The least 212 expensive way to filter out metrics would be at the source level, e.g., 213 filtering out source named "MyMetrics". The most expensive way would be 214 per metric filtering. 215 </p> 216 <p>Here are some examples:</p> 217 <pre> 218 test.sink.file0.class=org.apache.hadoop.metrics2.sink.FileSink 219 test.sink.file0.context=foo</pre> 220 <p>In this example, we configured one sink instance that would 221 accept metrics from context <code>foo</code> only. 222 </p> 223 <pre> 224 *.source.filter.class=org.apache.hadoop.metrics2.filter.GlobFilter 225 test.*.source.filter.include=foo 226 test.*.source.filter.exclude=bar</pre> 227 <p>In this example, we specify a source filter that includes source 228 <code>foo</code> and excludes <code>bar</code>. When only include 229 patterns are specified, the filter operates in the white listing mode, 230 where only matched sources are included. Likewise, when only exclude 231 patterns are specified, only matched sources are excluded. Sources that 232 are not matched in either patterns are included as well when both patterns 233 are present. Note, the include patterns have precedence over the exclude 234 patterns. 235 </p> 236 <p>Similarly, you can specify the <code>record.filter</code> and 237 <code>metric.filter</code> options, which operate at record and metric 238 level, respectively. Filters can be combined to optimize 239 the filtering efficiency.</p> 240 241 <h2><a name="instrumentation">Metrics instrumentation strategy</a></h2> 242 243 In previous examples, we showed a minimal example to use the 244 metrics framework. In a larger system (like Hadoop) that allows 245 custom metrics instrumentation, we recommend the following strategy: 246 <pre> 247 @Metrics(about="My metrics description", context="MyContext") 248 class MyMetrics extends MyInstrumentation { 249 250 @Metric("My gauge description") MutableGaugeInt gauge0; 251 @Metric("My counter description") MutableCounterLong counter0; 252 @Metric("My rate description") MutableRate rate0; 253 254 @Override public void setGauge0(int value) { gauge0.set(value); } 255 @Override public void incrCounter0() { counter0.incr(); } 256 @Override public void addRate0(long elapsed) { rate0.add(elapsed); } 257 } 258 </pre> 259 260 Note, in this example we introduced the following: 261 <dl> 262 <dt><em>MyInstrumentation</em></dt> 263 <dd>This is usually an abstract class (or interface) to define an 264 instrumentation interface (incrCounter0 etc.) that allows different 265 implementations. This could be a mechanism to allow different metrics 266 systems to be used at runtime via configuration. 267 </dd> 268 <dt><em>Mutable[Gauge*|Counter*|Rate]</em></dt> 269 <dd>These are library classes to manage mutable metrics for 270 implementations of metrics sources. They produce immutable gauge and 271 counters (Metric[Gauge*|Counter*]) for downstream consumption (sinks) 272 upon <code>snapshot</code>. The <code>MutableRate</code> 273 in particular, provides a way to measure latency and throughput of an 274 operation. In this particular case, it produces a long counter 275 "Rate0NumOps" and double gauge "Rate0AvgTime" when snapshotted. 276 </dd> 277 </dl> 278 279 <h2><a name="migration">Migration from previous system</a></h2> 280 <p>Users of the previous metrics system would notice the lack of 281 <code>context</code> prefix in the configuration examples. The new 282 metrics system decouples the concept for context (for grouping) with the 283 implementation where a particular context object does the updating and 284 publishing of metrics, which causes problems when you want to have a 285 single context to be consumed by multiple backends. You would also have to 286 configure an implementation instance per context, even if you have a 287 backend that can handle multiple contexts (file, gangalia etc.): 288 </p> 289 <table width="99%" border="1" cellspacing="0" cellpadding="4"> 290 <tbody> 291 <tr> 292 <th width="40%">Before</th><th>After</th> 293 </tr> 294 <tr> 295 <td><pre> 296 context1.class=org.hadoop.metrics.file.FileContext 297 context2.class=org.hadoop.metrics.file.FileContext 298 ... 299 contextn.class=org.hadoop.metrics.file.FileContext</pre> 300 </td> 301 <td><pre> 302 myprefix.sink.file.class=org.hadoop.metrics2.sink.FileSink</pre> 303 </td> 304 </tr> 305 </tbody> 306 </table> 307 <p>In the new metrics system, you can simulate the previous behavior by 308 using the context option in the sink options like the following: 309 </p> 310 <table width="99%" border="1" cellspacing="0" cellpadding="4"> 311 <tbody> 312 <tr> 313 <th width="40%">Before</th><th>After</th> 314 </tr> 315 <tr> 316 <td><pre> 317 context0.class=org.hadoop.metrics.file.FileContext 318 context0.fileName=context0.out 319 context1.class=org.hadoop.metrics.file.FileContext 320 context1.fileName=context1.out 321 ... 322 contextn.class=org.hadoop.metrics.file.FileContext 323 contextn.fileName=contextn.out</pre> 324 </td> 325 <td><pre> 326 myprefix.sink.*.class=org.apache.hadoop.metrics2.sink.FileSink 327 myprefix.sink.file0.context=context0 328 myprefix.sink.file0.filename=context1.out 329 myprefix.sink.file1.context=context1 330 myprefix.sink.file1.filename=context1.out 331 ... 332 myprefix.sink.filen.context=contextn 333 myprefix.sink.filen.filename=contextn.out</pre> 334 </td> 335 </tr> 336 </tbody> 337 </table> 338 <p>to send metrics of a particular context to a particular backend. Note, 339 <code>myprefix</code> is an arbitrary prefix for configuration groupings, 340 typically they are the name of a particular process 341 (<code>namenode</code>, <code>jobtracker</code>, etc.) 342 </p> 343 */ 344 @InterfaceAudience.Public 345 @InterfaceStability.Evolving 346 package org.apache.hadoop.metrics2; 347 348 import org.apache.hadoop.classification.InterfaceAudience; 349 import org.apache.hadoop.classification.InterfaceStability;