1.\" $OpenBSD: pctr.4,v 1.15 2001/08/03 14:30:26 mpech Exp $ 2.\" 3.\" Pentium performance counter driver for OpenBSD. 4.\" Copyright 1996 David Mazieres <dm@lcs.mit.edu>. 5.\" 6.\" Modification and redistribution in source and binary forms is 7.\" permitted provided that due credit is given to the author and the 8.\" OpenBSD project by leaving this copyright notice intact. 9.\" 10.Dd August 15, 1996 11.Dt PCTR 4 i386 12.Os 13.Sh NAME 14.Nm pctr 15.Nd driver for CPU performance counters 16.Sh SYNOPSIS 17.Cd pseudo-device pctr 18.Sh DESCRIPTION 19The 20.Nm 21device provides access to the performance counters on Intel brand processors, 22and to the TSC on others. 23.Pp 24Intel processors have two 40-bit performance 25counters which can be programmed to count events such as cache misses, 26branch target buffer hits, TLB misses, dual-issues, interrupts, 27pipeline flushes, and more. 28.Pp 29There is one 30.Em ioctl 31call to read the status of all counters, and one 32.Em ioctl 33call to program the function of each counter. 34All require the following includes: 35.Bd -literal -offset indent 36#include <sys/types.h> 37#include <machine/cpu.h> 38#include <machine/pctr.h> 39.Ed 40.Pp 41The current state of all counters can be read with the 42.Dv PCIOCRD 43.Em ioctl , 44which takes an argument of type 45.Dv "struct pctrst" : 46.Bd -literal -offset indent 47#define PCTR_NUM 2 48struct pctrst { 49 u_int pctr_fn[PCTR_NUM]; 50 pctrval pctr_tsc; 51 pctrval pctr_hwc[PCTR_NUM]; 52 pctrval pctr_idl; 53}; 54.Ed 55.Pp 56In this structure, 57.Dv ctr_fn 58contains the functions of the two counters, as previously set by the 59.Dv PCIOCS0 60and 61.Dv PCIOCS1 62ioctls (see below). 63.Dv pctr_hwc 64contains the actual value of the two hardware counters. 65.Dv pctr_tsc 66is a free-running, 64-bit cycle counter. 67Finally, 68.Dv pctr_idl 69is a 64-bit count of idle-loop iterations. 70.Pp 71The functions of the two counters can be programmed with ioctls 72.Dv PCIOCS0 73and 74.Dv PCIOCS1 , 75which require a writeable file descriptor and take an argument of type 76.Dv "unsigned int" . \& 77The meaning of this integer is dependent on the particular CPU. 78.\" The 79.\" following procedure can be used to determine which counters are 80.\" available on a given cpu: 81.\" .Bd -literal -offset indent 82.\" ctrval id = __cpuid(); 83.\" if (__hasp5ctr(id)) { 84.\" /* The machine has Pentium counters */ 85.\" } else if (__hasp6ctr(id)) { 86.\" /* The machine has Pentium Pro counters */ 87.\" } else if (__hastsc(id)) { 88.\" /* The machine just has a time stamp counter */ 89.\" } else { 90.\" /* No counters at all */ 91.\"} 92.\" .Ed 93.Ss Time stamp counter 94The time stamp counter is available on all machines with Pentium and 95Pentium Pro counters, as well as on some 486s and non-intel CPUs. 96It is set to zero at boot time, and then increments with each cycle. 97Because the counter is 64-bits wide, it does not overflow. 98.Pp 99The time stamp counter can be read directly from user-mode using 100the 101.Fn rdtsc 102macro, which returns a 64-bit value of type 103.Dv pctrval . 104The following example illustrates a simple use of 105.Dv rdtsc 106to measure the execution time of a hypothetical subroutine called 107.Fn functionx : 108.Bd -literal -offset indent 109void 110time_functionx(void) 111{ 112 pctrval tsc; 113 114 tsc = rdtsc(); 115 functionx(); 116 tsc = rdtsc() - tsc; 117 printf ("Functionx took %qd cycles.\en", tsc); 118} 119.Ed 120.Pp 121The value of the time stamp counter is also returned by the 122.Dv PCIOCRD 123.Em ioctl , 124so that one can get an exact timestamp on readings of the hardware 125event counters. 126.Ss Pentium counters 127The Pentium counters are programmed with a 9 bit function. 128The top three bits contain the following flags: 129.Pp 130.Bl -tag -width P5CTR_C 131.It Dv P5CTR_K 132Enables counting of events that occur in kernel mode. 133.It Dv P5CTR_U 134Enables counting of events that occur in user mode. 135You must set at least one of 136.Dv P5CTR_U 137and 138.Dv P5CTR_K 139to count anything. 140.It Dv P5CTR_C 141When this flag is set, the counter attempts to count the number of 142cycles spent servicing a particular event, rather than simply the 143number of occurrences of that event. 144.El 145.Pp 146The bottom 6 bits set the particular event counted. 147Here is the event type of each permissible value for the bottom 6 bits of the 148counter function: 149.Pp 150.Bl -tag -width 0x00 -compact -offset indent 151.It 0x00 152Data read 153.It 0x01 154Data write 155.It 0x02 156Data TLB miss 157.It 0x03 158Data read miss 159.It 0x04 160Data write miss 161.It 0x05 162Write (hit) to M or E state lines 163.It 0x06 164Data cache lines written back 165.It 0x07 166Data cache snoops 167.It 0x08 168Data cache snoop hits 169.It 0x09 170Memory accesses in both pipes 171.It 0x0a 172Bank conflicts 173.It 0x0b 174Misaligned data memory references 175.It 0x0c 176Code read 177.It 0x0d 178Code TLB miss 179.It 0x0e 180Code cache miss 181.It 0x0f 182Any segment register load 183.It 0x12 184Branches 185.It 0x13 186BTB hits 187.It 0x14 188Taken branch or BTB hit 189.It 0x15 190Pipeline flushes 191.It 0x16 192Instructions executed 193.It 0x17 194Instructions executed in the V-pipe 195.It 0x18 196Bus utilization (clocks) 197.It 0x19 198Pipeline stalled by write backup 199.It 0x1a 200Pipeline stalled by data memory read 201.It 0x1b 202Pipeline stalled by write to E or M line 203.It 0x1c 204Locked bus cycle 205.It 0x1d 206I/O read or write cycle 207.It 0x1e 208Noncacheable memory references 209.It 0x1f 210AGI (Address Generation Interlock) 211.It 0x22 212Floating-point operations 213.It 0x23 214Breakpoint 0 match 215.It 0x24 216Breakpoint 1 match 217.It 0x25 218Breakpoint 2 match 219.It 0x26 220Breakpoint 3 match 221.It 0x27 222Hardware interrupts 223.It 0x28 224Data read or data write 225.It 0x29 226Data read miss or data write miss 227.El 228.Ss Pentium Pro counters 229The Pentium Pro counter functions contain several parts. 230The most significant byte (an 8-bit integer shifted left by 231.Dv P6CTR_CM_SHIFT ) 232contains a 233.Em "counter mask" . \& 234If non-zero, this sets a threshold for the number of times an event 235must occur in one cycle for the counter to be incremented. 236The 237.Em "counter mask" 238can therefore be used to count cycles in which an event 239occurs at least some number of times. 240The next byte contains several flags: 241.Pp 242.Bl -tag -width P6CTR_EN 243.It Dv P6CTR_U 244Enables counting of events that occur in user mode. 245.It Dv P6CTR_K 246Enables counting of events that occur in kernel mode. 247You must set at least one of 248.Dv P6CTR_K 249and 250.Dv P6CTR_U 251to count anything. 252.It Dv P6CTR_E 253Counts edges rather than cycles. 254For some functions this allows you 255to get an estimate of the number of events rather than the number of 256cycles occupied by those events. 257.It Dv P6CTR_EN 258Enable counters. 259This bit must be set in the function for counter 0 260in order for either of the counters to be enabled. 261This bit should probably be set in counter 1 as well. 262.It Dv P6CTR_I 263Inverts the sense of the 264.Em "counter mask" . \& 265When this bit is set, the counter only increments on cycles in which 266there are no 267.Em more 268events than specified in the 269.Em "counter mask" . 270.El 271.Pp 272The next byte, also known as the 273.Em "unit mask" , 274contains flags specific to the event being counted. 275For events dealing with the L2 cache, the following flags are valid: 276.Pp 277.Bl -tag -width P6CTR_UM_M 278.It Dv P6CTR_UM_M 279Count events involving modified cache lines. 280.It Dv P6CTR_UM_E 281Count events involving exclusive cache lines. 282.It Dv P6CTR_UM_S 283Count events involving shared cache lines. 284.It Dv P6CTR_UM_I 285Count events involving invalid cache lines. 286.El 287To measure all L2 cache activity, all these bits should be set. 288They can be set with the macro 289.Dv P6CTR_UM_MESI 290which contains the bitwise or of all of the above. 291.Pp 292For event types dealing with bus transactions, there is another flag 293that can be set in the 294.Em "unit mask" : 295.Bl -tag -width P6CTR_UM_A 296.It Dv P6CTR_UM_A 297Count all appropriate bus events, not just those initiated by the 298processor. 299.El 300.Pp 301Finally, the least significant byte of the counter function is the 302event type to count. 303The following values are available: 304.Pp 305.Bl -tag -width 0x00 -compact 306.It 0x03 LD_BLOCKS 307Number of store buffer blocks. 308.It 0x04 SB_DRAINS 309Number of store buffer drain cycles. 310.It 0x05 MISALIGN_MEM_REF 311Number of misaligned data memory references. 312.It 0x06 SEGMENT_REG_LOADS 313Number of segment register loads. 314.It 0x10 FP_COMP_OPS_EXE (ctr0 only) 315Number of computational floating-point operations executed. 316.It 0x11 FP_ASSIST (ctr1 only) 317Number of floating-point exception cases handled by microcode. 318.It 0x12 MUL (ctr1 only) 319Number of multiplies. 320.It 0x13 DIV (ctr1 only) 321Number of divides. 322.It 0x14 CYCLES_DIV_BUSY (ctr0 only) 323Number of cycles during which the divider is busy. 324.It 0x21 L2_ADS 325Number of L2 address strobes. 326.It 0x22 L2_DBUS_BUSY 327Number of cycles during which the data bus was busy. 328.It 0x23 L2_DBUS_BUSY_RD 329Number of cycles during which the data bus was busy transferring data 330from L2 to the processor. 331.It 0x24 L2_LINES_IN 332Number of lines allocated in the L2. 333.It 0x25 L2_M_LINES_INM 334Number of modified lines allocated in the L2. 335.It 0x26 L2_LINES_OUT 336Number of lines removed from the L2 for any reason. 337.It 0x27 L2_M_LINES_OUTM 338Number of modified lines removed from the L2 for any reason. 339.It 0x28 L2_IFETCH/mesi 340Number of L2 instruction fetches. 341.It 0x29 L2_LD/mesi 342Number of L2 data loads. 343.It 0x2a L2_ST/mesi 344Number of L2 data stores. 345.It 0x2e L2_RQSTS/mesi 346Number of L2 requests. 347.It 0x43 DATA_MEM_REFS 348All memory references, both cacheable and non-cacheable. 349.It 0x45 DCU_LINES_IN 350Total lines allocated in the DCU. 351.It 0x46 DCU_M_LINES_IN 352Number of M state lines allocated in the DCU. 353.It 0x47 DCU_M_LINES_OUT 354Number of M state lines evicted from the DCU. 355This includes evictions via snoop HITM, intervention or replacement 356.It 0x48 DCU_MISS_OUTSTANDING 357Weighted number of cycles while a DCU miss is outstanding. 358.It 0x60 BUS_REQ_OUTSTANDING 359Number of bus requests outstanding. 360.It 0x61 BUS_BNR_DRV 361Number of bus clock cycles during which the processor is driving the 362BNR pin. 363.It 0x62 BUS_DRDY_CLOCKS/a 364Number of clocks during which DRDY is asserted. 365.It 0x63 BUS_LOCK_CLOCKS/a 366Number of clocks during which LOCK is asserted. 367.It 0x64 BUS_DATA_RCV 368Number of bus clock cycles during which the processor is receiving 369data. 370.It 0x65 BUS_TRAN_BRD/a 371Number of burst read transactions. 372.It 0x66 BUS_TRAN_RFO/a 373Number of read for ownership transactions. 374.It 0x67 BUS_TRANS_WB/a 375Number of write back transactions. 376.It 0x68 BUS_TRAN_IFETCH/a 377Number of instruction fetch transactions. 378.It 0x69 BUS_TRAN_INVAL/a 379Number of invalidate transactions. 380.It 0x6a BUS_TRAN_PWR/a 381Number of partial write transactions. 382.It 0x6b BUS_TRANS_P/a 383Number of partial transactions. 384.It 0x6c BUS_TRANS_IO/a 385Number of I/O transactions. 386.It 0x6d BUS_TRAN_DEF/a 387Number of deferred transactions. 388.It 0x6e BUS_TRAN_BURST/a 389Number of burst transactions. 390.It 0x6f BUS_TRAN_MEM/a 391Number of memory transactions. 392.It 0x70 BUS_TRAN_ANY/a 393Number of all transactions. 394.It 0x79 CPU_CLK_UNHALTED 395Number of cycles during which the processor is not halted. 396.It 0x7a BUS_HIT_DRV 397Number of bus clock cycles during which the processor is driving the 398HIT pin. 399.It 0x7b BUS_HITM_DRV 400Number of bus clock cycles during which the processor is driving the 401HITM pin. 402.It 0x7e BUS_SNOOP_STALL 403Number of clock cycles during which the bus is snoop stalled. 404.It 0x80 IFU_IFETCH 405Number of instruction fetches, both cacheable and non-cacheable. 406.It 0x81 IFU_IFETCH_MISS 407Number of instruction fetch misses. 408.It 0x85 ITLB_MISS 409Number of ITLB misses. 410.It 0x86 IFU_MEM_STALL 411Number of cycles that the instruction fetch pipe stage is stalled, 412including cache mises, ITLB misses, ITLB faults, and victim cache 413evictions 414.It 0x87 ILD_STALL 415Number of cycles that the instruction length decoder is stalled 416.It 0xa2 RESOURCE_STALLS 417Number of cycles during which there are resource-related stalls. 418.It 0xc0 INST_RETIRED 419Number of instructions retired. 420.It 0xc1 FLOPS (ctr0 only) 421Number of computational floating-point operations retired. 422.It 0xc2 UOPS_RETIRED 423Number of UOPs retired. 424.It 0xc4 BR_INST_RETIRED 425Number of branch instructions retired. 426.It 0xc5 BR_MISS_PRED_RETIRED 427Number of mispredicted branches retired. 428.It 0xc6 CYCLES_INT_MASKED 429Number of processor cycles for which interrupts are disabled. 430.It 0xc7 CYCLES_INT_PENDING_AND_MASKED 431Number of processor cycles for which interrupts are disabled and 432interrupts are pending. 433.It 0xc8 HW_INT_RX 434Number of hardware interrupts received. 435.It 0xc9 BR_TAKEN_RETIRED 436Number of taken branches retired. 437.It 0xca BR_MISS_PRED_TAKEN_RET 438Number of taken mispredicted branches retired. 439.It 0xd0 INST_DECODER 440Number of instructions decoded. 441.It 0xd2 PARTIAL_RAT_STALLS 442Number of cycles or events for partial stalls. 443.It 0xe0 BR_INST_DECODED 444Number of branch instructions decoded. 445.It 0xe2 BTB_MISSES 446Number of branches that miss the BTB. 447.It 0xe4 BR_BOGUS 448Number of bogus branches. 449.It 0xe6 BACLEARS 450Number of times BACLEAR is asserted. 451.El 452.Pp 453Events marked /mesi require the 454.Dv P6CTR_UM_[MESI] 455bits in the 456.Em "unit mask" . \& 457Events marked /a can take the 458.Dv P6CTR_UM_A 459bit. 460.Pp 461Unlike the Pentium counters, the Pentium Pro counters can be read 462directly from user-mode without need to invoke the kernel. 463The macro 464.Fn rdpmc ctr 465takes 0 or 1 as an argument to specify a counter, and returns that 466counter's 40-bit value (which will be of type 467.Dv pctrval ) . 468This is generally preferable to making a system call as it introduces 469less distortion in measurements. 470However, you should be aware of the possibility of an interrupt between 471invocations of 472.Fn rdpmc 473and/or 474.Fn rdtsc . 475.Sh ERRORS 476.Bl -tag -width "[ENODEV]" 477.It Bq Er ENODEV 478An attempt was made to set the counter functions on a CPU that does 479not support counters. 480.It Bq Er EINVAL 481An invalid counter function was provided as an argument to the 482.Dv PCIOCS0 483or 484.Dv PCIOCS1 485.Em ioctl . 486.It Bq Er EPERM 487An attempt was made to set the counter functions, but the device was 488not open for writing. 489.El 490.Sh FILES 491.Bl -tag -width /dev/pctr -compact 492.It Pa /dev/pctr 493.El 494.Sh SEE ALSO 495.Xr pctr 1 496.Sh HISTORY 497A 498.Nm 499device first appeared in 500.Ox 2.0 . 501.Sh AUTHORS 502The 503.Nm 504device was written by David Mazieres 505.Aq dm@lcs.mit.edu . 506.Sh BUGS 507Not all counter functions are completely accurate. 508Some of the functions don't seem to make any sense at all. 509