1.\" $NetBSD: re_format.7,v 1.11 2015/08/22 14:04:54 wiz Exp $ 2.\" 3.\" Copyright (c) 1992, 1993, 1994 4.\" The Regents of the University of California. All rights reserved. 5.\" 6.\" This code is derived from software contributed to Berkeley by 7.\" Henry Spencer. 8.\" 9.\" Redistribution and use in source and binary forms, with or without 10.\" modification, are permitted provided that the following conditions 11.\" are met: 12.\" 1. Redistributions of source code must retain the above copyright 13.\" notice, this list of conditions and the following disclaimer. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 3. Neither the name of the University nor the names of its contributors 18.\" may be used to endorse or promote products derived from this software 19.\" without specific prior written permission. 20.\" 21.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND 22.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 23.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 24.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE 25.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 26.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 27.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 28.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 29.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 30.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 31.\" SUCH DAMAGE. 32.\" 33.\" Copyright (c) 1992, 1993, 1994 Henry Spencer. 34.\" 35.\" This code is derived from software contributed to Berkeley by 36.\" Henry Spencer. 37.\" 38.\" Redistribution and use in source and binary forms, with or without 39.\" modification, are permitted provided that the following conditions 40.\" are met: 41.\" 1. Redistributions of source code must retain the above copyright 42.\" notice, this list of conditions and the following disclaimer. 43.\" 2. Redistributions in binary form must reproduce the above copyright 44.\" notice, this list of conditions and the following disclaimer in the 45.\" documentation and/or other materials provided with the distribution. 46.\" 3. All advertising materials mentioning features or use of this software 47.\" must display the following acknowledgement: 48.\" This product includes software developed by the University of 49.\" California, Berkeley and its contributors. 50.\" 4. Neither the name of the University nor the names of its contributors 51.\" may be used to endorse or promote products derived from this software 52.\" without specific prior written permission. 53.\" 54.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND 55.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 56.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 57.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE 58.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 59.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 60.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 61.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 62.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 63.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 64.\" SUCH DAMAGE. 65.\" 66.\" @(#)re_format.7 8.3 (Berkeley) 3/20/94 67.\" 68.Dd March 20, 1994 69.Dt RE_FORMAT 7 70.Os 71.Sh NAME 72.Nm re_format 73.Nd POSIX 1003.2 regular expressions 74.Sh DESCRIPTION 75Regular expressions (``RE''s), 76as defined in POSIX 1003.2, come in two forms: 77modern REs (roughly those of 78.Xr egrep 1 ; 791003.2 calls these ``extended'' REs) 80and obsolete REs (roughly those of 81.Xr ed 1 ; 821003.2 ``basic'' REs). 83Obsolete REs mostly exist for backward compatibility in some old programs; 84they will be discussed at the end. 851003.2 leaves some aspects of RE syntax and semantics open; 86`(*)' marks decisions on these aspects that 87may not be fully portable to other 1003.2 implementations. 88.Pp 89A (modern) RE is one(*) or more non-empty(*) 90.Em branches , 91separated by `|'. 92It matches anything that matches one of the branches. 93.Pp 94A branch is one(*) or more 95.Em pieces , 96concatenated. 97It matches a match for the first, followed by a match for the second, etc. 98.Pp 99A piece is an 100.Em atom 101possibly followed 102by a single(*) `*', `+', `?', or 103.Em bound . 104An atom followed by `*' matches a sequence of 0 or more matches of the atom. 105An atom followed by `+' matches a sequence of 1 or more matches of the atom. 106An atom followed by `?' matches a sequence of 0 or 1 matches of the atom. 107.Pp 108A 109.Em bound 110is `{' followed by an unsigned decimal integer, possibly followed by `,' 111possibly followed by another unsigned decimal integer, 112always followed by `}'. 113The integers must lie between 0 and RE_DUP_MAX (255(*)) inclusive, 114and if there are two of them, the first may not exceed the second. 115An atom followed by a bound containing one integer 116.Em i 117and no comma matches a sequence of exactly 118.Em i 119matches of the atom. 120An atom followed by a bound containing one integer 121.Em i 122and a comma matches a sequence of 123.Em i 124or more matches of the atom. 125An atom followed by a bound containing two integers 126.Em i 127and 128.Em j 129matches a sequence of 130.Em i 131through 132.Em j 133(inclusive) matches of the atom. 134.Pp 135An atom is a regular expression enclosed in `()' (matching a match for the 136regular expression), an empty set of `()' (matching the null string)(*), a 137.Em bracket expression 138(see below), `.' (matching any single character), 139`^' (matching the null string at the beginning of a line), 140`$' (matching the null string at the end of a line), 141a `\e' followed by one of the characters `^.[$()|*+?{\e' 142(matching that character taken as an ordinary character), 143a `\e' followed by any other character(*) 144(matching that character taken as an ordinary character, 145as if the `\e' had not been present(*)), 146or a single character with no other significance (matching that character). 147A `{' followed by a character other than a digit is an ordinary 148character, not the beginning of a bound(*). 149It is illegal to end an RE with `\e'. 150.Pp 151A 152.Em bracket expression 153is a list of characters enclosed in `[]'. 154It normally matches any single character from the list (but see below). 155If the list begins with `^', 156it matches any single character (but see below) 157.Em not 158from the rest of the list. 159If two characters in the list are separated by `\-', this is shorthand 160for the full 161.Em range 162of characters between those two (inclusive) in the collating sequence, 163e.g. `[0-9]' in ASCII matches any decimal digit. 164It is illegal(*) for two ranges to share an endpoint, e.g. `a-c-e'. 165Ranges are very collating-sequence-dependent, 166and portable programs should avoid relying on them. 167.Pp 168To include a literal `]' in the list, make it the first character 169(following a possible `^'). 170To include a literal `\-', make it the first or last character, 171or the second endpoint of a range. 172To use a literal `\-' as the first endpoint of a range, 173enclose it in `[.' and `.]' to make it a collating element (see below). 174With the exception of these and some combinations using `[' (see next 175paragraphs), all other special characters, including `\e', lose their 176special significance within a bracket expression. 177.Pp 178Within a bracket expression, a collating element (a character, 179a multi-character sequence that collates as if it were a single character, 180or a collating-sequence name for either) 181enclosed in `[.' and `.]' stands for the 182sequence of characters of that collating element. 183The sequence is a single element of the bracket expression's list. 184A bracket expression containing a multi-character collating element 185can thus match more than one character, 186e.g. if the collating sequence includes a `ch' collating element, 187then the RE `[[.ch.]]*c' matches the first five characters 188of `chchcc'. 189.Pp 190Within a bracket expression, a collating element enclosed in `[=' and 191`=]' is an equivalence class, standing for the sequences of characters 192of all collating elements equivalent to that one, including itself. 193(If there are no other equivalent collating elements, 194the treatment is as if the enclosing delimiters were `[.' and `.]'.) 195For example, if o and '\(^o' are the members of an equivalence class, 196then `[[=o=]]', `[[=\(^o'=]]', and `[o\(^o']' are all synonymous. 197An equivalence class may not(*) be an endpoint 198of a range. 199.Pp 200Within a bracket expression, the name of a 201.Em character class 202enclosed in `[:' and `:]' stands for the list of all characters 203belonging to that class. 204Standard character class names are: 205.Bl -column "alnum" "digit" "xdigit" 206.It alnum digit punct 207.It alpha graph space 208.It blank lower upper 209.It cntrl print xdigit 210.El 211.Pp 212These stand for the character classes defined in 213.Xr ctype 3 . 214A locale may provide others. 215A character class may not be used as an endpoint of a range. 216.Pp 217There are two special cases(*) of bracket expressions: 218the bracket expressions `[[:\*[Lt]:]]' and `[[:\*[Gt]:]]' match 219the null string at the beginning and end of a word respectively. 220A word is defined as a sequence of word characters 221which is neither preceded nor followed by word characters. 222A word character is an 223.Em alnum 224character (as defined by 225.Xr ctype 3 ) 226or an underscore. 227This is an extension, compatible with but not specified by POSIX 1003.2, 228and should be used with caution in software intended to be portable 229to other systems. 230.Pp 231In the event that an RE could match more than one substring of a given 232string, the RE matches the one starting earliest in the string. 233If the RE could match more than one substring starting at that point, 234it matches the longest. 235Subexpressions also match the longest possible substrings, subject to 236the constraint that the whole match be as long as possible, 237with subexpressions starting earlier in the RE taking priority over 238ones starting later. 239Note that higher-level subexpressions thus take priority over 240their lower-level component subexpressions. 241.Pp 242Match lengths are measured in characters, not collating elements. 243A null string is considered longer than no match at all. 244For example, 245`bb*' matches the three middle characters of `abbbc', 246`(wee|week)(knights|nights)' matches all ten characters of `weeknights', 247when `(.*).*' is matched against `abc' the parenthesized subexpression 248matches all three characters, and 249when `(a*)*' is matched against `bc' both the whole RE and the parenthesized 250subexpression match the null string. 251.Pp 252If case-independent matching is specified, 253the effect is much as if all case distinctions had vanished from the 254alphabet. 255When an alphabetic that exists in multiple cases appears as an 256ordinary character outside a bracket expression, it is effectively 257transformed into a bracket expression containing both cases, 258e.g. `x' becomes `[xX]'. 259When it appears inside a bracket expression, all case counterparts 260of it are added to the bracket expression, so that (e.g.) `[x]' 261becomes `[xX]' and `[^x]' becomes `[^xX]'. 262.Pp 263No particular limit is imposed on the length of REs(*). 264Programs intended to be portable should not employ REs longer 265than 256 bytes, 266as an implementation can refuse to accept such REs and remain 267POSIX-compliant. 268.Pp 269Obsolete (``basic'') regular expressions differ in several respects. 270`|', `+', and `?' are ordinary characters and there is no equivalent 271for their functionality. 272The delimiters for bounds are `\e{' and `\e}', 273with `{' and `}' by themselves ordinary characters. 274The parentheses for nested subexpressions are `\e(' and `\e)', 275with `(' and `)' by themselves ordinary characters. 276`^' is an ordinary character except at the beginning of the 277RE or(*) the beginning of a parenthesized subexpression, 278`$' is an ordinary character except at the end of the 279RE or(*) the end of a parenthesized subexpression, 280and `*' is an ordinary character if it appears at the beginning of the 281RE or the beginning of a parenthesized subexpression 282(after a possible leading `^'). 283Finally, there is one new type of atom, a 284.Em back reference : 285`\e' followed by a non-zero decimal digit 286.Em d 287matches the same sequence of characters 288matched by the 289.Em d Ns th parenthesized subexpression 290(numbering subexpressions by the positions of their opening parentheses, 291left to right), 292so that (e.g.) `\e([bc]\e)\e1' matches `bb' or `cc' but not `bc'. 293.Sh SEE ALSO 294.Xr regex 3 295.Pp 296POSIX 1003.2, section 2.8 (Regular Expression Notation). 297.Sh BUGS 298Having two kinds of REs is a botch. 299.Pp 300The current 1003.2 spec says that `)' is an ordinary character in 301the absence of an unmatched `('; 302this was an unintentional result of a wording error, and change is likely. 303Avoid relying on it. 304.Pp 305Back references are a dreadful botch, 306posing major problems for efficient implementations. 307They are also somewhat vaguely defined 308(does `a\e(\e(b\e)*\e2\e)*d' match `abbbd'?). 309Avoid using them. 310.Pp 3111003.2's specification of case-independent matching is vague. 312The ``one case implies all cases'' definition given above 313is current consensus among implementors as to the right interpretation. 314.Pp 315The syntax for word boundaries is incredibly ugly. 316