1 /*! 2 This crate provides a library for parsing, compiling, and executing regular 3 expressions. Its syntax is similar to Perl-style regular expressions, but lacks 4 a few features like look around and backreferences. In exchange, all searches 5 execute in linear time with respect to the size of the regular expression and 6 search text. 7 8 This crate's documentation provides some simple examples, describes 9 [Unicode support](#unicode) and exhaustively lists the 10 [supported syntax](#syntax). 11 12 For more specific details on the API for regular expressions, please see the 13 documentation for the [`Regex`](struct.Regex.html) type. 14 15 # Usage 16 17 This crate is [on crates.io](https://crates.io/crates/regex) and can be 18 used by adding `regex` to your dependencies in your project's `Cargo.toml`. 19 20 ```toml 21 [dependencies] 22 regex = "1" 23 ``` 24 25 If you're using Rust 2015, then you'll also need to add it to your crate root: 26 27 ```rust 28 extern crate regex; 29 ``` 30 31 # Example: find a date 32 33 General use of regular expressions in this package involves compiling an 34 expression and then using it to search, split or replace text. For example, 35 to confirm that some text resembles a date: 36 37 ```rust 38 use regex::Regex; 39 let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap(); 40 assert!(re.is_match("2014-01-01")); 41 ``` 42 43 Notice the use of the `^` and `$` anchors. In this crate, every expression 44 is executed with an implicit `.*?` at the beginning and end, which allows 45 it to match anywhere in the text. Anchors can be used to ensure that the 46 full text matches an expression. 47 48 This example also demonstrates the utility of 49 [raw strings](https://doc.rust-lang.org/stable/reference/tokens.html#raw-string-literals) 50 in Rust, which 51 are just like regular strings except they are prefixed with an `r` and do 52 not process any escape sequences. For example, `"\\d"` is the same 53 expression as `r"\d"`. 54 55 # Example: Avoid compiling the same regex in a loop 56 57 It is an anti-pattern to compile the same regular expression in a loop 58 since compilation is typically expensive. (It takes anywhere from a few 59 microseconds to a few **milliseconds** depending on the size of the 60 regex.) Not only is compilation itself expensive, but this also prevents 61 optimizations that reuse allocations internally to the matching engines. 62 63 In Rust, it can sometimes be a pain to pass regular expressions around if 64 they're used from inside a helper function. Instead, we recommend using the 65 [`lazy_static`](https://crates.io/crates/lazy_static) crate to ensure that 66 regular expressions are compiled exactly once. 67 68 For example: 69 70 ```rust 71 #[macro_use] extern crate lazy_static; 72 extern crate regex; 73 74 use regex::Regex; 75 76 fn some_helper_function(text: &str) -> bool { 77 lazy_static! { 78 static ref RE: Regex = Regex::new("...").unwrap(); 79 } 80 RE.is_match(text) 81 } 82 83 fn main() {} 84 ``` 85 86 Specifically, in this example, the regex will be compiled when it is used for 87 the first time. On subsequent uses, it will reuse the previous compilation. 88 89 # Example: iterating over capture groups 90 91 This crate provides convenient iterators for matching an expression 92 repeatedly against a search string to find successive non-overlapping 93 matches. For example, to find all dates in a string and be able to access 94 them by their component pieces: 95 96 ```rust 97 # extern crate regex; use regex::Regex; 98 # fn main() { 99 let re = Regex::new(r"(\d{4})-(\d{2})-(\d{2})").unwrap(); 100 let text = "2012-03-14, 2013-01-01 and 2014-07-05"; 101 for cap in re.captures_iter(text) { 102 println!("Month: {} Day: {} Year: {}", &cap[2], &cap[3], &cap[1]); 103 } 104 // Output: 105 // Month: 03 Day: 14 Year: 2012 106 // Month: 01 Day: 01 Year: 2013 107 // Month: 07 Day: 05 Year: 2014 108 # } 109 ``` 110 111 Notice that the year is in the capture group indexed at `1`. This is 112 because the *entire match* is stored in the capture group at index `0`. 113 114 # Example: replacement with named capture groups 115 116 Building on the previous example, perhaps we'd like to rearrange the date 117 formats. This can be done with text replacement. But to make the code 118 clearer, we can *name* our capture groups and use those names as variables 119 in our replacement text: 120 121 ```rust 122 # extern crate regex; use regex::Regex; 123 # fn main() { 124 let re = Regex::new(r"(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})").unwrap(); 125 let before = "2012-03-14, 2013-01-01 and 2014-07-05"; 126 let after = re.replace_all(before, "$m/$d/$y"); 127 assert_eq!(after, "03/14/2012, 01/01/2013 and 07/05/2014"); 128 # } 129 ``` 130 131 The `replace` methods are actually polymorphic in the replacement, which 132 provides more flexibility than is seen here. (See the documentation for 133 `Regex::replace` for more details.) 134 135 Note that if your regex gets complicated, you can use the `x` flag to 136 enable insignificant whitespace mode, which also lets you write comments: 137 138 ```rust 139 # extern crate regex; use regex::Regex; 140 # fn main() { 141 let re = Regex::new(r"(?x) 142 (?P<y>\d{4}) # the year 143 - 144 (?P<m>\d{2}) # the month 145 - 146 (?P<d>\d{2}) # the day 147 ").unwrap(); 148 let before = "2012-03-14, 2013-01-01 and 2014-07-05"; 149 let after = re.replace_all(before, "$m/$d/$y"); 150 assert_eq!(after, "03/14/2012, 01/01/2013 and 07/05/2014"); 151 # } 152 ``` 153 154 If you wish to match against whitespace in this mode, you can still use `\s`, 155 `\n`, `\t`, etc. For escaping a single space character, you can use its hex 156 character code `\x20` or temporarily disable the `x` flag, e.g., `(?-x: )`. 157 158 # Example: match multiple regular expressions simultaneously 159 160 This demonstrates how to use a `RegexSet` to match multiple (possibly 161 overlapping) regular expressions in a single scan of the search text: 162 163 ```rust 164 use regex::RegexSet; 165 166 let set = RegexSet::new(&[ 167 r"\w+", 168 r"\d+", 169 r"\pL+", 170 r"foo", 171 r"bar", 172 r"barfoo", 173 r"foobar", 174 ]).unwrap(); 175 176 // Iterate over and collect all of the matches. 177 let matches: Vec<_> = set.matches("foobar").into_iter().collect(); 178 assert_eq!(matches, vec![0, 2, 3, 4, 6]); 179 180 // You can also test whether a particular regex matched: 181 let matches = set.matches("foobar"); 182 assert!(!matches.matched(5)); 183 assert!(matches.matched(6)); 184 ``` 185 186 # Pay for what you use 187 188 With respect to searching text with a regular expression, there are three 189 questions that can be asked: 190 191 1. Does the text match this expression? 192 2. If so, where does it match? 193 3. Where did the capturing groups match? 194 195 Generally speaking, this crate could provide a function to answer only #3, 196 which would subsume #1 and #2 automatically. However, it can be significantly 197 more expensive to compute the location of capturing group matches, so it's best 198 not to do it if you don't need to. 199 200 Therefore, only use what you need. For example, don't use `find` if you 201 only need to test if an expression matches a string. (Use `is_match` 202 instead.) 203 204 # Unicode 205 206 This implementation executes regular expressions **only** on valid UTF-8 207 while exposing match locations as byte indices into the search string. (To 208 relax this restriction, use the [`bytes`](bytes/index.html) sub-module.) 209 210 Only simple case folding is supported. Namely, when matching 211 case-insensitively, the characters are first mapped using the "simple" case 212 folding rules defined by Unicode. 213 214 Regular expressions themselves are **only** interpreted as a sequence of 215 Unicode scalar values. This means you can use Unicode characters directly 216 in your expression: 217 218 ```rust 219 # extern crate regex; use regex::Regex; 220 # fn main() { 221 let re = Regex::new(r"(?i)Δ+").unwrap(); 222 let mat = re.find("ΔδΔ").unwrap(); 223 assert_eq!((mat.start(), mat.end()), (0, 6)); 224 # } 225 ``` 226 227 Most features of the regular expressions in this crate are Unicode aware. Here 228 are some examples: 229 230 * `.` will match any valid UTF-8 encoded Unicode scalar value except for `\n`. 231 (To also match `\n`, enable the `s` flag, e.g., `(?s:.)`.) 232 * `\w`, `\d` and `\s` are Unicode aware. For example, `\s` will match all forms 233 of whitespace categorized by Unicode. 234 * `\b` matches a Unicode word boundary. 235 * Negated character classes like `[^a]` match all Unicode scalar values except 236 for `a`. 237 * `^` and `$` are **not** Unicode aware in multi-line mode. Namely, they only 238 recognize `\n` and not any of the other forms of line terminators defined 239 by Unicode. 240 241 Unicode general categories, scripts, script extensions, ages and a smattering 242 of boolean properties are available as character classes. For example, you can 243 match a sequence of numerals, Greek or Cherokee letters: 244 245 ```rust 246 # extern crate regex; use regex::Regex; 247 # fn main() { 248 let re = Regex::new(r"[\pN\p{Greek}\p{Cherokee}]+").unwrap(); 249 let mat = re.find("abcΔᎠβⅠᏴγδⅡxyz").unwrap(); 250 assert_eq!((mat.start(), mat.end()), (3, 23)); 251 # } 252 ``` 253 254 For a more detailed breakdown of Unicode support with respect to 255 [UTS#18](http://unicode.org/reports/tr18/), 256 please see the 257 [UNICODE](https://github.com/rust-lang/regex/blob/master/UNICODE.md) 258 document in the root of the regex repository. 259 260 # Opt out of Unicode support 261 262 The `bytes` sub-module provides a `Regex` type that can be used to match 263 on `&[u8]`. By default, text is interpreted as UTF-8 just like it is with 264 the main `Regex` type. However, this behavior can be disabled by turning 265 off the `u` flag, even if doing so could result in matching invalid UTF-8. 266 For example, when the `u` flag is disabled, `.` will match any byte instead 267 of any Unicode scalar value. 268 269 Disabling the `u` flag is also possible with the standard `&str`-based `Regex` 270 type, but it is only allowed where the UTF-8 invariant is maintained. For 271 example, `(?-u:\w)` is an ASCII-only `\w` character class and is legal in an 272 `&str`-based `Regex`, but `(?-u:\xFF)` will attempt to match the raw byte 273 `\xFF`, which is invalid UTF-8 and therefore is illegal in `&str`-based 274 regexes. 275 276 Finally, since Unicode support requires bundling large Unicode data 277 tables, this crate exposes knobs to disable the compilation of those 278 data tables, which can be useful for shrinking binary size and reducing 279 compilation times. For details on how to do that, see the section on [crate 280 features](#crate-features). 281 282 # Syntax 283 284 The syntax supported in this crate is documented below. 285 286 Note that the regular expression parser and abstract syntax are exposed in 287 a separate crate, [`regex-syntax`](https://docs.rs/regex-syntax). 288 289 ## Matching one character 290 291 <pre class="rust"> 292 . any character except new line (includes new line with s flag) 293 \d digit (\p{Nd}) 294 \D not digit 295 \pN One-letter name Unicode character class 296 \p{Greek} Unicode character class (general category or script) 297 \PN Negated one-letter name Unicode character class 298 \P{Greek} negated Unicode character class (general category or script) 299 </pre> 300 301 ### Character classes 302 303 <pre class="rust"> 304 [xyz] A character class matching either x, y or z (union). 305 [^xyz] A character class matching any character except x, y and z. 306 [a-z] A character class matching any character in range a-z. 307 [[:alpha:]] ASCII character class ([A-Za-z]) 308 [[:^alpha:]] Negated ASCII character class ([^A-Za-z]) 309 [x[^xyz]] Nested/grouping character class (matching any character except y and z) 310 [a-y&&xyz] Intersection (matching x or y) 311 [0-9&&[^4]] Subtraction using intersection and negation (matching 0-9 except 4) 312 [0-9--4] Direct subtraction (matching 0-9 except 4) 313 [a-g~~b-h] Symmetric difference (matching `a` and `h` only) 314 [\[\]] Escaping in character classes (matching [ or ]) 315 </pre> 316 317 Any named character class may appear inside a bracketed `[...]` character 318 class. For example, `[\p{Greek}[:digit:]]` matches any Greek or ASCII 319 digit. `[\p{Greek}&&\pL]` matches Greek letters. 320 321 Precedence in character classes, from most binding to least: 322 323 1. Ranges: `a-cd` == `[a-c]d` 324 2. Union: `ab&&bc` == `[ab]&&[bc]` 325 3. Intersection: `^a-z&&b` == `^[a-z&&b]` 326 4. Negation 327 328 ## Composites 329 330 <pre class="rust"> 331 xy concatenation (x followed by y) 332 x|y alternation (x or y, prefer x) 333 </pre> 334 335 ## Repetitions 336 337 <pre class="rust"> 338 x* zero or more of x (greedy) 339 x+ one or more of x (greedy) 340 x? zero or one of x (greedy) 341 x*? zero or more of x (ungreedy/lazy) 342 x+? one or more of x (ungreedy/lazy) 343 x?? zero or one of x (ungreedy/lazy) 344 x{n,m} at least n x and at most m x (greedy) 345 x{n,} at least n x (greedy) 346 x{n} exactly n x 347 x{n,m}? at least n x and at most m x (ungreedy/lazy) 348 x{n,}? at least n x (ungreedy/lazy) 349 x{n}? exactly n x 350 </pre> 351 352 ## Empty matches 353 354 <pre class="rust"> 355 ^ the beginning of text (or start-of-line with multi-line mode) 356 $ the end of text (or end-of-line with multi-line mode) 357 \A only the beginning of text (even with multi-line mode enabled) 358 \z only the end of text (even with multi-line mode enabled) 359 \b a Unicode word boundary (\w on one side and \W, \A, or \z on other) 360 \B not a Unicode word boundary 361 </pre> 362 363 ## Grouping and flags 364 365 <pre class="rust"> 366 (exp) numbered capture group (indexed by opening parenthesis) 367 (?P<name>exp) named (also numbered) capture group (allowed chars: [_0-9a-zA-Z]) 368 (?:exp) non-capturing group 369 (?flags) set flags within current group 370 (?flags:exp) set flags for exp (non-capturing) 371 </pre> 372 373 Flags are each a single character. For example, `(?x)` sets the flag `x` 374 and `(?-x)` clears the flag `x`. Multiple flags can be set or cleared at 375 the same time: `(?xy)` sets both the `x` and `y` flags and `(?x-y)` sets 376 the `x` flag and clears the `y` flag. 377 378 All flags are by default disabled unless stated otherwise. They are: 379 380 <pre class="rust"> 381 i case-insensitive: letters match both upper and lower case 382 m multi-line mode: ^ and $ match begin/end of line 383 s allow . to match \n 384 U swap the meaning of x* and x*? 385 u Unicode support (enabled by default) 386 x ignore whitespace and allow line comments (starting with `#`) 387 </pre> 388 389 Flags can be toggled within a pattern. Here's an example that matches 390 case-insensitively for the first part but case-sensitively for the second part: 391 392 ```rust 393 # extern crate regex; use regex::Regex; 394 # fn main() { 395 let re = Regex::new(r"(?i)a+(?-i)b+").unwrap(); 396 let cap = re.captures("AaAaAbbBBBb").unwrap(); 397 assert_eq!(&cap[0], "AaAaAbb"); 398 # } 399 ``` 400 401 Notice that the `a+` matches either `a` or `A`, but the `b+` only matches 402 `b`. 403 404 Multi-line mode means `^` and `$` no longer match just at the beginning/end of 405 the input, but at the beginning/end of lines: 406 407 ``` 408 # use regex::Regex; 409 let re = Regex::new(r"(?m)^line \d+").unwrap(); 410 let m = re.find("line one\nline 2\n").unwrap(); 411 assert_eq!(m.as_str(), "line 2"); 412 ``` 413 414 Note that `^` matches after new lines, even at the end of input: 415 416 ``` 417 # use regex::Regex; 418 let re = Regex::new(r"(?m)^").unwrap(); 419 let m = re.find_iter("test\n").last().unwrap(); 420 assert_eq!((m.start(), m.end()), (5, 5)); 421 ``` 422 423 Here is an example that uses an ASCII word boundary instead of a Unicode 424 word boundary: 425 426 ```rust 427 # extern crate regex; use regex::Regex; 428 # fn main() { 429 let re = Regex::new(r"(?-u:\b).+(?-u:\b)").unwrap(); 430 let cap = re.captures("$$abc$$").unwrap(); 431 assert_eq!(&cap[0], "abc"); 432 # } 433 ``` 434 435 ## Escape sequences 436 437 <pre class="rust"> 438 \* literal *, works for any punctuation character: \.+*?()|[]{}^$ 439 \a bell (\x07) 440 \f form feed (\x0C) 441 \t horizontal tab 442 \n new line 443 \r carriage return 444 \v vertical tab (\x0B) 445 \123 octal character code (up to three digits) (when enabled) 446 \x7F hex character code (exactly two digits) 447 \x{10FFFF} any hex character code corresponding to a Unicode code point 448 \u007F hex character code (exactly four digits) 449 \u{7F} any hex character code corresponding to a Unicode code point 450 \U0000007F hex character code (exactly eight digits) 451 \U{7F} any hex character code corresponding to a Unicode code point 452 </pre> 453 454 ## Perl character classes (Unicode friendly) 455 456 These classes are based on the definitions provided in 457 [UTS#18](http://www.unicode.org/reports/tr18/#Compatibility_Properties): 458 459 <pre class="rust"> 460 \d digit (\p{Nd}) 461 \D not digit 462 \s whitespace (\p{White_Space}) 463 \S not whitespace 464 \w word character (\p{Alphabetic} + \p{M} + \d + \p{Pc} + \p{Join_Control}) 465 \W not word character 466 </pre> 467 468 ## ASCII character classes 469 470 <pre class="rust"> 471 [[:alnum:]] alphanumeric ([0-9A-Za-z]) 472 [[:alpha:]] alphabetic ([A-Za-z]) 473 [[:ascii:]] ASCII ([\x00-\x7F]) 474 [[:blank:]] blank ([\t ]) 475 [[:cntrl:]] control ([\x00-\x1F\x7F]) 476 [[:digit:]] digits ([0-9]) 477 [[:graph:]] graphical ([!-~]) 478 [[:lower:]] lower case ([a-z]) 479 [[:print:]] printable ([ -~]) 480 [[:punct:]] punctuation ([!-/:-@\[-`{-~]) 481 [[:space:]] whitespace ([\t\n\v\f\r ]) 482 [[:upper:]] upper case ([A-Z]) 483 [[:word:]] word characters ([0-9A-Za-z_]) 484 [[:xdigit:]] hex digit ([0-9A-Fa-f]) 485 </pre> 486 487 # Crate features 488 489 By default, this crate tries pretty hard to make regex matching both as fast 490 as possible and as correct as it can be, within reason. This means that there 491 is a lot of code dedicated to performance, the handling of Unicode data and the 492 Unicode data itself. Overall, this leads to more dependencies, larger binaries 493 and longer compile times. This trade off may not be appropriate in all cases, 494 and indeed, even when all Unicode and performance features are disabled, one 495 is still left with a perfectly serviceable regex engine that will work well 496 in many cases. 497 498 This crate exposes a number of features for controlling that trade off. Some 499 of these features are strictly performance oriented, such that disabling them 500 won't result in a loss of functionality, but may result in worse performance. 501 Other features, such as the ones controlling the presence or absence of Unicode 502 data, can result in a loss of functionality. For example, if one disables the 503 `unicode-case` feature (described below), then compiling the regex `(?i)a` 504 will fail since Unicode case insensitivity is enabled by default. Instead, 505 callers must use `(?i-u)a` instead to disable Unicode case folding. Stated 506 differently, enabling or disabling any of the features below can only add or 507 subtract from the total set of valid regular expressions. Enabling or disabling 508 a feature will never modify the match semantics of a regular expression. 509 510 All features below are enabled by default. 511 512 ### Ecosystem features 513 514 * **std** - 515 When enabled, this will cause `regex` to use the standard library. Currently, 516 disabling this feature will always result in a compilation error. It is 517 intended to add `alloc`-only support to regex in the future. 518 519 ### Performance features 520 521 * **perf** - 522 Enables all performance related features. This feature is enabled by default 523 and will always cover all features that improve performance, even if more 524 are added in the future. 525 * **perf-cache** - 526 Enables the use of very fast thread safe caching for internal match state. 527 When this is disabled, caching is still used, but with a slower and simpler 528 implementation. Disabling this drops the `thread_local` and `lazy_static` 529 dependencies. 530 * **perf-dfa** - 531 Enables the use of a lazy DFA for matching. The lazy DFA is used to compile 532 portions of a regex to a very fast DFA on an as-needed basis. This can 533 result in substantial speedups, usually by an order of magnitude on large 534 haystacks. The lazy DFA does not bring in any new dependencies, but it can 535 make compile times longer. 536 * **perf-inline** - 537 Enables the use of aggressive inlining inside match routines. This reduces 538 the overhead of each match. The aggressive inlining, however, increases 539 compile times and binary size. 540 * **perf-literal** - 541 Enables the use of literal optimizations for speeding up matches. In some 542 cases, literal optimizations can result in speedups of _several_ orders of 543 magnitude. Disabling this drops the `aho-corasick` and `memchr` dependencies. 544 545 ### Unicode features 546 547 * **unicode** - 548 Enables all Unicode features. This feature is enabled by default, and will 549 always cover all Unicode features, even if more are added in the future. 550 * **unicode-age** - 551 Provide the data for the 552 [Unicode `Age` property](https://www.unicode.org/reports/tr44/tr44-24.html#Character_Age). 553 This makes it possible to use classes like `\p{Age:6.0}` to refer to all 554 codepoints first introduced in Unicode 6.0 555 * **unicode-bool** - 556 Provide the data for numerous Unicode boolean properties. The full list 557 is not included here, but contains properties like `Alphabetic`, `Emoji`, 558 `Lowercase`, `Math`, `Uppercase` and `White_Space`. 559 * **unicode-case** - 560 Provide the data for case insensitive matching using 561 [Unicode's "simple loose matches" specification](https://www.unicode.org/reports/tr18/#Simple_Loose_Matches). 562 * **unicode-gencat** - 563 Provide the data for 564 [Uncode general categories](https://www.unicode.org/reports/tr44/tr44-24.html#General_Category_Values). 565 This includes, but is not limited to, `Decimal_Number`, `Letter`, 566 `Math_Symbol`, `Number` and `Punctuation`. 567 * **unicode-perl** - 568 Provide the data for supporting the Unicode-aware Perl character classes, 569 corresponding to `\w`, `\s` and `\d`. This is also necessary for using 570 Unicode-aware word boundary assertions. Note that if this feature is 571 disabled, the `\s` and `\d` character classes are still available if the 572 `unicode-bool` and `unicode-gencat` features are enabled, respectively. 573 * **unicode-script** - 574 Provide the data for 575 [Unicode scripts and script extensions](https://www.unicode.org/reports/tr24/). 576 This includes, but is not limited to, `Arabic`, `Cyrillic`, `Hebrew`, 577 `Latin` and `Thai`. 578 * **unicode-segment** - 579 Provide the data necessary to provide the properties used to implement the 580 [Unicode text segmentation algorithms](https://www.unicode.org/reports/tr29/). 581 This enables using classes like `\p{gcb=Extend}`, `\p{wb=Katakana}` and 582 `\p{sb=ATerm}`. 583 584 585 # Untrusted input 586 587 This crate can handle both untrusted regular expressions and untrusted 588 search text. 589 590 Untrusted regular expressions are handled by capping the size of a compiled 591 regular expression. 592 (See [`RegexBuilder::size_limit`](struct.RegexBuilder.html#method.size_limit).) 593 Without this, it would be trivial for an attacker to exhaust your system's 594 memory with expressions like `a{100}{100}{100}`. 595 596 Untrusted search text is allowed because the matching engine(s) in this 597 crate have time complexity `O(mn)` (with `m ~ regex` and `n ~ search 598 text`), which means there's no way to cause exponential blow-up like with 599 some other regular expression engines. (We pay for this by disallowing 600 features like arbitrary look-ahead and backreferences.) 601 602 When a DFA is used, pathological cases with exponential state blow-up are 603 avoided by constructing the DFA lazily or in an "online" manner. Therefore, 604 at most one new state can be created for each byte of input. This satisfies 605 our time complexity guarantees, but can lead to memory growth 606 proportional to the size of the input. As a stopgap, the DFA is only 607 allowed to store a fixed number of states. When the limit is reached, its 608 states are wiped and continues on, possibly duplicating previous work. If 609 the limit is reached too frequently, it gives up and hands control off to 610 another matching engine with fixed memory requirements. 611 (The DFA size limit can also be tweaked. See 612 [`RegexBuilder::dfa_size_limit`](struct.RegexBuilder.html#method.dfa_size_limit).) 613 */ 614 615 #![deny(missing_docs)] 616 #![cfg_attr(test, deny(warnings))] 617 #![cfg_attr(feature = "pattern", feature(pattern))] 618 619 #[cfg(not(feature = "std"))] 620 compile_error!("`std` feature is currently required to build this crate"); 621 622 #[cfg(feature = "perf-literal")] 623 extern crate aho_corasick; 624 #[cfg(test)] 625 extern crate doc_comment; 626 #[cfg(feature = "perf-literal")] 627 extern crate memchr; 628 #[cfg(test)] 629 #[cfg_attr(feature = "perf-literal", macro_use)] 630 extern crate quickcheck; 631 extern crate regex_syntax as syntax; 632 #[cfg(feature = "perf-cache")] 633 extern crate thread_local; 634 635 #[cfg(test)] 636 doc_comment::doctest!("../README.md"); 637 638 #[cfg(feature = "std")] 639 pub use error::Error; 640 #[cfg(feature = "std")] 641 pub use re_builder::set_unicode::*; 642 #[cfg(feature = "std")] 643 pub use re_builder::unicode::*; 644 #[cfg(feature = "std")] 645 pub use re_set::unicode::*; 646 #[cfg(feature = "std")] 647 #[cfg(feature = "std")] 648 pub use re_unicode::{ 649 escape, CaptureLocations, CaptureMatches, CaptureNames, Captures, 650 Locations, Match, Matches, NoExpand, Regex, Replacer, ReplacerRef, Split, 651 SplitN, SubCaptureMatches, 652 }; 653 654 /** 655 Match regular expressions on arbitrary bytes. 656 657 This module provides a nearly identical API to the one found in the 658 top-level of this crate. There are two important differences: 659 660 1. Matching is done on `&[u8]` instead of `&str`. Additionally, `Vec<u8>` 661 is used where `String` would have been used. 662 2. Unicode support can be disabled even when disabling it would result in 663 matching invalid UTF-8 bytes. 664 665 # Example: match null terminated string 666 667 This shows how to find all null-terminated strings in a slice of bytes: 668 669 ```rust 670 # use regex::bytes::Regex; 671 let re = Regex::new(r"(?-u)(?P<cstr>[^\x00]+)\x00").unwrap(); 672 let text = b"foo\x00bar\x00baz\x00"; 673 674 // Extract all of the strings without the null terminator from each match. 675 // The unwrap is OK here since a match requires the `cstr` capture to match. 676 let cstrs: Vec<&[u8]> = 677 re.captures_iter(text) 678 .map(|c| c.name("cstr").unwrap().as_bytes()) 679 .collect(); 680 assert_eq!(vec![&b"foo"[..], &b"bar"[..], &b"baz"[..]], cstrs); 681 ``` 682 683 # Example: selectively enable Unicode support 684 685 This shows how to match an arbitrary byte pattern followed by a UTF-8 encoded 686 string (e.g., to extract a title from a Matroska file): 687 688 ```rust 689 # use std::str; 690 # use regex::bytes::Regex; 691 let re = Regex::new( 692 r"(?-u)\x7b\xa9(?:[\x80-\xfe]|[\x40-\xff].)(?u:(.*))" 693 ).unwrap(); 694 let text = b"\x12\xd0\x3b\x5f\x7b\xa9\x85\xe2\x98\x83\x80\x98\x54\x76\x68\x65"; 695 let caps = re.captures(text).unwrap(); 696 697 // Notice that despite the `.*` at the end, it will only match valid UTF-8 698 // because Unicode mode was enabled with the `u` flag. Without the `u` flag, 699 // the `.*` would match the rest of the bytes. 700 let mat = caps.get(1).unwrap(); 701 assert_eq!((7, 10), (mat.start(), mat.end())); 702 703 // If there was a match, Unicode mode guarantees that `title` is valid UTF-8. 704 let title = str::from_utf8(&caps[1]).unwrap(); 705 assert_eq!("☃", title); 706 ``` 707 708 In general, if the Unicode flag is enabled in a capture group and that capture 709 is part of the overall match, then the capture is *guaranteed* to be valid 710 UTF-8. 711 712 # Syntax 713 714 The supported syntax is pretty much the same as the syntax for Unicode 715 regular expressions with a few changes that make sense for matching arbitrary 716 bytes: 717 718 1. The `u` flag can be disabled even when disabling it might cause the regex to 719 match invalid UTF-8. When the `u` flag is disabled, the regex is said to be in 720 "ASCII compatible" mode. 721 2. In ASCII compatible mode, neither Unicode scalar values nor Unicode 722 character classes are allowed. 723 3. In ASCII compatible mode, Perl character classes (`\w`, `\d` and `\s`) 724 revert to their typical ASCII definition. `\w` maps to `[[:word:]]`, `\d` maps 725 to `[[:digit:]]` and `\s` maps to `[[:space:]]`. 726 4. In ASCII compatible mode, word boundaries use the ASCII compatible `\w` to 727 determine whether a byte is a word byte or not. 728 5. Hexadecimal notation can be used to specify arbitrary bytes instead of 729 Unicode codepoints. For example, in ASCII compatible mode, `\xFF` matches the 730 literal byte `\xFF`, while in Unicode mode, `\xFF` is a Unicode codepoint that 731 matches its UTF-8 encoding of `\xC3\xBF`. Similarly for octal notation when 732 enabled. 733 6. `.` matches any *byte* except for `\n` instead of any Unicode scalar value. 734 When the `s` flag is enabled, `.` matches any byte. 735 736 # Performance 737 738 In general, one should expect performance on `&[u8]` to be roughly similar to 739 performance on `&str`. 740 */ 741 #[cfg(feature = "std")] 742 pub mod bytes { 743 pub use re_builder::bytes::*; 744 pub use re_builder::set_bytes::*; 745 pub use re_bytes::*; 746 pub use re_set::bytes::*; 747 } 748 749 mod backtrack; 750 mod cache; 751 mod compile; 752 #[cfg(feature = "perf-dfa")] 753 mod dfa; 754 mod error; 755 mod exec; 756 mod expand; 757 mod find_byte; 758 #[cfg(feature = "perf-literal")] 759 mod freqs; 760 mod input; 761 mod literal; 762 #[cfg(feature = "pattern")] 763 mod pattern; 764 mod pikevm; 765 mod prog; 766 mod re_builder; 767 mod re_bytes; 768 mod re_set; 769 mod re_trait; 770 mod re_unicode; 771 mod sparse; 772 mod utf8; 773 774 /// The `internal` module exists to support suspicious activity, such as 775 /// testing different matching engines and supporting the `regex-debug` CLI 776 /// utility. 777 #[doc(hidden)] 778 #[cfg(feature = "std")] 779 pub mod internal { 780 pub use compile::Compiler; 781 pub use exec::{Exec, ExecBuilder}; 782 pub use input::{Char, CharInput, Input, InputAt}; 783 pub use literal::LiteralSearcher; 784 pub use prog::{EmptyLook, Inst, InstRanges, Program}; 785 } 786