1.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! 2 3.. _borg_patterns: 4 5borg help patterns 6~~~~~~~~~~~~~~~~~~ 7 8 9The path/filenames used as input for the pattern matching start from the 10currently active recursion root. You usually give the recursion root(s) 11when invoking borg and these can be either relative or absolute paths. 12 13So, when you give `relative/` as root, the paths going into the matcher 14will look like `relative/.../file.ext`. When you give `/absolute/` as 15root, they will look like `/absolute/.../file.ext`. This is meant when 16we talk about "full path" below. 17 18File paths in Borg archives are always stored normalized and relative. 19This means that e.g. ``borg create /path/to/repo ../some/path`` will 20store all files as `some/path/.../file.ext` and ``borg create 21/path/to/repo /home/user`` will store all files as 22`home/user/.../file.ext`. Therefore, always use relative paths in your 23patterns when matching archive content in commands like ``extract`` or 24``mount``. Starting with Borg 1.2 this behaviour will be changed to 25accept both absolute and relative paths. 26 27File patterns support these styles: fnmatch, shell, regular expressions, 28path prefixes and path full-matches. By default, fnmatch is used for 29``--exclude`` patterns and shell-style is used for the experimental 30``--pattern`` option. 31 32If followed by a colon (':') the first two characters of a pattern are 33used as a style selector. Explicit style selection is necessary when a 34non-default style is desired or when the desired pattern starts with 35two alphanumeric characters followed by a colon (i.e. `aa:something/*`). 36 37`Fnmatch <https://docs.python.org/3/library/fnmatch.html>`_, selector `fm:` 38 This is the default style for ``--exclude`` and ``--exclude-from``. 39 These patterns use a variant of shell pattern syntax, with '\*' matching 40 any number of characters, '?' matching any single character, '[...]' 41 matching any single character specified, including ranges, and '[!...]' 42 matching any character not specified. For the purpose of these patterns, 43 the path separator (backslash for Windows and '/' on other systems) is not 44 treated specially. Wrap meta-characters in brackets for a literal 45 match (i.e. `[?]` to match the literal character `?`). For a path 46 to match a pattern, the full path must match, or it must match 47 from the start of the full path to just before a path separator. Except 48 for the root path, paths will never end in the path separator when 49 matching is attempted. Thus, if a given pattern ends in a path 50 separator, a '\*' is appended before matching is attempted. 51 52Shell-style patterns, selector `sh:` 53 This is the default style for ``--pattern`` and ``--patterns-from``. 54 Like fnmatch patterns these are similar to shell patterns. The difference 55 is that the pattern may include `**/` for matching zero or more directory 56 levels, `*` for matching zero or more arbitrary characters with the 57 exception of any path separator. 58 59Regular expressions, selector `re:` 60 Regular expressions similar to those found in Perl are supported. Unlike 61 shell patterns regular expressions are not required to match the full 62 path and any substring match is sufficient. It is strongly recommended to 63 anchor patterns to the start ('^'), to the end ('$') or both. Path 64 separators (backslash for Windows and '/' on other systems) in paths are 65 always normalized to a forward slash ('/') before applying a pattern. The 66 regular expression syntax is described in the `Python documentation for 67 the re module <https://docs.python.org/3/library/re.html>`_. 68 69Path prefix, selector `pp:` 70 This pattern style is useful to match whole sub-directories. The pattern 71 `pp:root/somedir` matches `root/somedir` and everything therein. 72 73Path full-match, selector `pf:` 74 This pattern style is (only) useful to match full paths. 75 This is kind of a pseudo pattern as it can not have any variable or 76 unspecified parts - the full path must be given. 77 `pf:root/file.ext` matches `root/file.ext` only. 78 79 Implementation note: this is implemented via very time-efficient O(1) 80 hashtable lookups (this means you can have huge amounts of such patterns 81 without impacting performance much). 82 Due to that, this kind of pattern does not respect any context or order. 83 If you use such a pattern to include a file, it will always be included 84 (if the directory recursion encounters it). 85 Other include/exclude patterns that would normally match will be ignored. 86 Same logic applies for exclude. 87 88.. note:: 89 90 `re:`, `sh:` and `fm:` patterns are all implemented on top of the Python SRE 91 engine. It is very easy to formulate patterns for each of these types which 92 requires an inordinate amount of time to match paths. If untrusted users 93 are able to supply patterns, ensure they cannot supply `re:` patterns. 94 Further, ensure that `sh:` and `fm:` patterns only contain a handful of 95 wildcards at most. 96 97Exclusions can be passed via the command line option ``--exclude``. When used 98from within a shell, the patterns should be quoted to protect them from 99expansion. 100 101The ``--exclude-from`` option permits loading exclusion patterns from a text 102file with one pattern per line. Lines empty or starting with the number sign 103('#') after removing whitespace on both ends are ignored. The optional style 104selector prefix is also supported for patterns loaded from a file. Due to 105whitespace removal, paths with whitespace at the beginning or end can only be 106excluded using regular expressions. 107 108To test your exclusion patterns without performing an actual backup you can 109run ``borg create --list --dry-run ...``. 110 111Examples:: 112 113 # Exclude '/home/user/file.o' but not '/home/user/file.odt': 114 $ borg create -e '*.o' backup / 115 116 # Exclude '/home/user/junk' and '/home/user/subdir/junk' but 117 # not '/home/user/importantjunk' or '/etc/junk': 118 $ borg create -e '/home/*/junk' backup / 119 120 # Exclude the contents of '/home/user/cache' but not the directory itself: 121 $ borg create -e /home/user/cache/ backup / 122 123 # The file '/home/user/cache/important' is *not* backed up: 124 $ borg create -e /home/user/cache/ backup / /home/user/cache/important 125 126 # The contents of directories in '/home' are not backed up when their name 127 # ends in '.tmp' 128 $ borg create --exclude 're:^/home/[^/]+\.tmp/' backup / 129 130 # Load exclusions from file 131 $ cat >exclude.txt <<EOF 132 # Comment line 133 /home/*/junk 134 *.tmp 135 fm:aa:something/* 136 re:^/home/[^/]+\.tmp/ 137 sh:/home/*/.thumbnails 138 # Example with spaces, no need to escape as it is processed by borg 139 some file with spaces.txt 140 EOF 141 $ borg create --exclude-from exclude.txt backup / 142 143.. container:: experimental 144 145 A more general and easier to use way to define filename matching patterns exists 146 with the experimental ``--pattern`` and ``--patterns-from`` options. Using these, you 147 may specify the backup roots (starting points) and patterns for inclusion/exclusion. 148 A root path starts with the prefix `R`, followed by a path (a plain path, not a 149 file pattern). An include rule starts with the prefix +, an exclude rule starts 150 with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern. 151 152 .. note:: 153 154 Via ``--pattern`` or ``--patterns-from`` you can define BOTH inclusion and exclusion 155 of files using pattern prefixes ``+`` and ``-``. With ``--exclude`` and 156 ``--exclude-from`` ONLY excludes are defined. 157 158 Inclusion patterns are useful to include paths that are contained in an excluded 159 path. The first matching pattern is used so if an include pattern matches before 160 an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches 161 a directory, it won't recurse into it and won't discover any potential matches for 162 include rules below that directory. 163 164 Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is 165 shell style (`sh:`), so those patterns behave similar to rsync include/exclude 166 patterns. The pattern style can be set via the `P` prefix. 167 168 Patterns (``--pattern``) and excludes (``--exclude``) from the command line are 169 considered first (in the order of appearance). Then patterns from ``--patterns-from`` 170 are added. Exclusion patterns from ``--exclude-from`` files are appended last. 171 172 Examples:: 173 174 # backup pics, but not the ones from 2018, except the good ones: 175 # note: using = is essential to avoid cmdline argument parsing issues. 176 borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics 177 178 # use a file with patterns: 179 borg create --patterns-from patterns.lst repo::arch 180 181 The patterns.lst file could look like that:: 182 183 # "sh:" pattern style is the default, so the following line is not needed: 184 P sh 185 R / 186 # can be rebuild 187 - /home/*/.cache 188 # they're downloads for a reason 189 - /home/*/Downloads 190 # susan is a nice person 191 # include susans home 192 + /home/susan 193 # don't backup the other home directories 194 - /home/* 195 # don't even look in /proc 196 ! /proc 197 198.. _borg_placeholders: 199 200borg help placeholders 201~~~~~~~~~~~~~~~~~~~~~~ 202 203 204Repository (or Archive) URLs, ``--prefix``, ``--glob-archives``, ``--comment`` 205and ``--remote-path`` values support these placeholders: 206 207{hostname} 208 The (short) hostname of the machine. 209 210{fqdn} 211 The full name of the machine. 212 213{reverse-fqdn} 214 The full name of the machine in reverse domain name notation. 215 216{now} 217 The current local date and time, by default in ISO-8601 format. 218 You can also supply your own `format string <https://docs.python.org/3.7/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {now:%Y-%m-%d_%H:%M:%S} 219 220{utcnow} 221 The current UTC date and time, by default in ISO-8601 format. 222 You can also supply your own `format string <https://docs.python.org/3.7/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {utcnow:%Y-%m-%d_%H:%M:%S} 223 224{user} 225 The user name (or UID, if no name is available) of the user running borg. 226 227{pid} 228 The current process ID. 229 230{borgversion} 231 The version of borg, e.g.: 1.0.8rc1 232 233{borgmajor} 234 The version of borg, only the major version, e.g.: 1 235 236{borgminor} 237 The version of borg, only major and minor version, e.g.: 1.0 238 239{borgpatch} 240 The version of borg, only major, minor and patch version, e.g.: 1.0.8 241 242If literal curly braces need to be used, double them for escaping:: 243 244 borg create /path/to/repo::{{literal_text}} 245 246Examples:: 247 248 borg create /path/to/repo::{hostname}-{user}-{utcnow} ... 249 borg create /path/to/repo::{hostname}-{now:%Y-%m-%d_%H:%M:%S} ... 250 borg prune --prefix '{hostname}-' ... 251 252.. note:: 253 systemd uses a difficult, non-standard syntax for command lines in unit files (refer to 254 the `systemd.unit(5)` manual page). 255 256 When invoking borg from unit files, pay particular attention to escaping, 257 especially when using the now/utcnow placeholders, since systemd performs its own 258 %-based variable replacement even in quoted text. To avoid interference from systemd, 259 double all percent signs (``{hostname}-{now:%Y-%m-%d_%H:%M:%S}`` 260 becomes ``{hostname}-{now:%%Y-%%m-%%d_%%H:%%M:%%S}``). 261 262.. _borg_compression: 263 264borg help compression 265~~~~~~~~~~~~~~~~~~~~~ 266 267 268It is no problem to mix different compression methods in one repo, 269deduplication is done on the source data chunks (not on the compressed 270or encrypted data). 271 272If some specific chunk was once compressed and stored into the repo, creating 273another backup that also uses this chunk will not change the stored chunk. 274So if you use different compression specs for the backups, whichever stores a 275chunk first determines its compression. See also borg recreate. 276 277Compression is lz4 by default. If you want something else, you have to specify what you want. 278 279Valid compression specifiers are: 280 281none 282 Do not compress. 283 284lz4 285 Use lz4 compression. Very high speed, very low compression. (default) 286 287zstd[,L] 288 Use zstd ("zstandard") compression, a modern wide-range algorithm. 289 If you do not explicitely give the compression level L (ranging from 1 290 to 22), it will use level 3. 291 Archives compressed with zstd are not compatible with borg < 1.1.4. 292 293zlib[,L] 294 Use zlib ("gz") compression. Medium speed, medium compression. 295 If you do not explicitely give the compression level L (ranging from 0 296 to 9), it will use level 6. 297 Giving level 0 (means "no compression", but still has zlib protocol 298 overhead) is usually pointless, you better use "none" compression. 299 300lzma[,L] 301 Use lzma ("xz") compression. Low speed, high compression. 302 If you do not explicitely give the compression level L (ranging from 0 303 to 9), it will use level 6. 304 Giving levels above 6 is pointless and counterproductive because it does 305 not compress better due to the buffer size used by borg - but it wastes 306 lots of CPU cycles and RAM. 307 308auto,C[,L] 309 Use a built-in heuristic to decide per chunk whether to compress or not. 310 The heuristic tries with lz4 whether the data is compressible. 311 For incompressible data, it will not use compression (uses "none"). 312 For compressible data, it uses the given C[,L] compression - with C[,L] 313 being any valid compression specifier. 314 315Examples:: 316 317 borg create --compression lz4 REPO::ARCHIVE data 318 borg create --compression zstd REPO::ARCHIVE data 319 borg create --compression zstd,10 REPO::ARCHIVE data 320 borg create --compression zlib REPO::ARCHIVE data 321 borg create --compression zlib,1 REPO::ARCHIVE data 322 borg create --compression auto,lzma,6 REPO::ARCHIVE data 323 borg create --compression auto,lzma ... 324 325