1.\" 2.\" Copyright (c) 2009 Isilon Inc http://www.isilon.com/ 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice(s), this list of conditions and the following disclaimer as 9.\" the first lines of this file unmodified other than the possible 10.\" addition of one or more copyright notices. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice(s), this list of conditions and the following disclaimer in the 13.\" documentation and/or other materials provided with the distribution. 14.\" 15.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY 16.\" EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18.\" DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY 19.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 20.\" (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 21.\" SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 22.\" CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH 25.\" DAMAGE. 26.\" 27.\" $FreeBSD$ 28.\" 29.Dd May 10, 2009 30.Dt FAIL 9 31.Os 32.Sh NAME 33.Nm KFAIL_POINT_CODE , 34.Nm KFAIL_POINT_RETURN , 35.Nm KFAIL_POINT_RETURN_VOID , 36.Nm KFAIL_POINT_ERROR , 37.Nm KFAIL_POINT_GOTO , 38.Nm fail_point , 39.Nm DEBUG_FP 40.Nd fail points 41.Sh SYNOPSIS 42.In sys/fail.h 43.Fn KFAIL_POINT_CODE "parent" "name" "code" 44.Fn KFAIL_POINT_RETURN "parent" "name" 45.Fn KFAIL_POINT_RETURN_VOID "parent" "name" 46.Fn KFAIL_POINT_ERROR "parent" "name" "error_var" 47.Fn KFAIL_POINT_GOTO "parent" "name" "error_var" "label" 48.Sh DESCRIPTION 49Fail points are used to add code points where errors may be injected 50in a user controlled fashion. 51Fail points provide a convenient wrapper around user-provided error 52injection code, providing a 53.Xr sysctl 9 54MIB, and a parser for that MIB that describes how the error 55injection code should fire. 56.Pp 57The base fail point macro is 58.Fn KFAIL_POINT_CODE 59where 60.Fa parent 61is a sysctl tree (frequently 62.Sy DEBUG_FP 63for kernel fail points, but various subsystems may wish to provide 64their own fail point trees), and 65.Fa name 66is the name of the MIB in that tree, and 67.Fa code 68is the error injection code. 69The 70.Fa code 71argument does not require braces, but it is considered good style to 72use braces for any multi-line code arguments. 73Inside the 74.Fa code 75argument, the evaluation of 76.Sy RETURN_VALUE 77is derived from the 78.Fn return 79value set in the sysctl MIB. 80See 81.Sx SYSCTL VARIABLES 82below. 83.Pp 84The remaining 85.Fn KFAIL_POINT_* 86macros are wrappers around common error injection paths: 87.Bl -inset 88.It Fn KFAIL_POINT_RETURN parent name 89is the equivalent of 90.Sy KFAIL_POINT_CODE(..., return RETURN_VALUE) 91.It Fn KFAIL_POINT_RETURN_VOID parent name 92is the equivalent of 93.Sy KFAIL_POINT_CODE(..., return) 94.It Fn KFAIL_POINT_ERROR parent name error_var 95is the equivalent of 96.Sy KFAIL_POINT_CODE(..., error_var = RETURN_VALUE) 97.It Fn KFAIL_POINT_GOTO parent name error_var label 98is the equivalent of 99.Sy KFAIL_POINT_CODE(..., { error_var = RETURN_VALUE; goto label;}) 100.El 101.Sh SYSCTL VARIABLES 102The 103.Fn KFAIL_POINT_* 104macros add sysctl MIBs where specified. 105Many base kernel MIBs can be found in the 106.Sy debug.fail_point 107tree (referenced in code by 108.Sy DEBUG_FP ) . 109.Pp 110The sysctl variable may be set using the following grammar: 111.Bd -literal 112 <fail_point> :: 113 <term> ( "->" <term> )* 114 115 <term> :: 116 ( (<float> "%") | (<integer> "*" ) )* 117 <type> 118 [ "(" <integer> ")" ] 119 [ "[pid " <integer> "]" ] 120 121 <float> :: 122 <integer> [ "." <integer> ] | 123 "." <integer> 124 125 <type> :: 126 "off" | "return" | "sleep" | "panic" | "break" | "print" 127.Ed 128.Pp 129The <type> argument specifies which action to take: 130.Bl -tag -width ".Dv return" 131.It Sy off 132Take no action (does not trigger fail point code) 133.It Sy return 134Trigger fail point code with specified argument 135.It Sy sleep 136Sleep the specified number of milliseconds 137.It Sy panic 138Panic 139.It Sy break 140Break into the debugger, or trap if there is no debugger support 141.It Sy print 142Print that the fail point executed 143.El 144.Pp 145The <float>% and <integer>* modifiers prior to <type> control when 146<type> is executed. 147The <float>% form (e.g. "1.2%") can be used to specify a 148probability that <type> will execute. 149The <integer>* form (e.g. "5*") can be used to specify the number of 150times <type> should be executed before this <term> is disabled. 151Only the last probability and the last count are used if multiple 152are specified, i.e. "1.2%2%" is the same as "2%". 153When both a probability and a count are specified, the probability 154is evaluated before the count, i.e. "2%5*" means "2% of the time, 155but only 5 times total". 156.Pp 157The operator -> can be used to express cascading terms. 158If you specify <term1>-><term2>, it means that if <term1> does not 159.Ql execute , 160<term2> is evaluated. 161For the purpose of this operator, the return() and print() operators 162are the only types that cascade. 163A return() term only cascades if the code executes, and a print() 164term only cascades when passed a non-zero argument. 165A pid can optionally be specified. 166The fail point term is only executed when invoked by a process with a 167matching p_pid. 168.Sh EXAMPLES 169.Bl -tag -width Sy 170.It Sy sysctl debug.fail_point.foobar="2.1%return(5)" 17121/1000ths of the time, execute 172.Fa code 173with RETURN_VALUE set to 5. 174.It Sy sysctl debug.fail_point.foobar="2%return(5)->5%return(22)" 1752/100ths of the time, execute 176.Fa code 177with RETURN_VALUE set to 5. 178If that does not happen, 5% of the time execute 179.Fa code 180with RETURN_VALUE set to 22. 181.It Sy sysctl debug.fail_point.foobar="5*return(5)->0.1%return(22)" 182For 5 times, return 5. 183After that, 1/1000th of the time, return 22. 184.It Sy sysctl debug.fail_point.foobar="0.1%5*return(5)" 185Return 5 for 1 in 1000 executions, but only 5 times total. 186.It Sy sysctl debug.fail_point.foobar="1%*sleep(50)" 1871/100th of the time, sleep 50ms. 188.It Sy sysctl debug.fail_point.foobar="1*return(5)[pid 1234]" 189Return 5 once, when pid 1234 executes the fail point. 190.El 191.Sh AUTHORS 192.An -nosplit 193This manual page was written by 194.An Zach Loafman Aq zml@FreeBSD.org . 195.Sh CAVEATS 196It is easy to shoot yourself in the foot by setting fail points too 197aggressively or setting too many in combination. 198For example, forcing 199.Fn malloc 200to fail consistently is potentially harmful to uptime. 201.Pp 202The 203.Fn sleep 204sysctl setting may not be appropriate in all situations. 205Currently, 206.Fn fail_point_eval 207does not verify whether the context is appropriate for calling 208.Fn msleep . 209