1fdm ============================================================================ 2 3*** Introduction 4 5fdm is a program to fetch mail and deliver it in various ways depending on a 6user-supplied ruleset. Mail may be fetched from stdin, IMAP or POP3 servers, or 7from local maildirs, and filtered based on whether it matches a regexp, its 8size or age, or the output of a shell command. It can be rewritten by an 9external process, dropped, left on the server or delivered into maildirs, 10mboxes, to a file or pipe, or any combination. 11 12fdm is designed to be lightweight but powerful, with a compact but clear 13configuration syntax. It is primarily designed for single-user uses but may 14also be configured to deliver mail in a multi-user setup. In this case, it uses 15privilege separation to minimise the amount of code running as the root user. 16 17*** Table of contents 18 19## Installation 20## Quick start 21## The configuration file 22%% Including other files 23%% Macros 24%% Testing macros 25%% Shell commands 26## Invoking fdm 27%% Temporary files 28%% Command line arguments 29%% Running from cron 30%% The lock file 31%% Testing and debugging 32## Fetching mail 33%% Mail tags 34%% POP3 and POP3S 35%% SSL certificate verification 36%% The .netrc file 37%% IMAP and IMAPS 38%% IMAP or POP3 over a pipe or ssh 39%% stdin and local mail 40%% From maildirs and mboxes 41%% Using NNTP and NNTPS 42%% New or old mail only 43## Defining actions 44%% Drop and keep 45%% Maildirs 46%% Mboxes 47%% IMAP and IMAPS 48%% SMTP 49%% Write, pipe, exec and append 50%% stdout 51%% Rewriting mail 52%% Adding or removing headers 53%% Tagging 54%% Compound actions 55%% Chained actions 56## Filtering mail 57%% Nesting rules 58%% Lambda actions 59%% The all condition 60%% Matched and unmatched 61%% Matching by account 62%% Matching a regexp 63%% Matching bracket expressions 64%% Matching by age or size 65%% Using a shell command 66%% Attachments 67%% Matching tags 68%% Using caches 69%% Cache commands 70## Setting options 71## Archiving and searching mail 72## Using fdm behind a proxy 73## Bug reports and queries 74## Frequently asked questions 75 76### Installation 77 78fdm depends on the Trivial Database library (TDB), available at: 79 80 https://tdb.samba.org 81 82Ensure it is installed, then download the source tarball and build fdm with: 83 84 $ tar -zxvf fdm-?.?.tar.gz 85 $ cd fdm-?.? 86 $ ./configure && make 87 88Then run 'make install' to install fdm to the default location under 89/usr/local. The --prefix argument may be set to specify an alternative 90installation location: 91 92 $ ./configure --prefix=/opt/fdm && make 93 $ sudo make install 94 95If being run as root, fdm requires a user named "_fdm" to exist. It will drop 96privileges to this user and its primary group. The user may be added on 97OpenBSD with, for example: 98 99 # useradd -u 999 -s /bin/nologin -d /var/empty -g=uid _fdm 100 101It is not necessary to add a user if fdm is always started by a non-root user. 102 103fdm can be built to use PCRE rather than standard regexps. To do so, add -DPCRE 104to the make command: 105 106 $ make -DPCRE 107 108Or PCRE=1 if using GNU make: 109 110 $ make PCRE=1 111 112### Quick start 113 114A simple ~/.fdm.conf file for a single user fetching from POP3, POP3S and IMAP 115accounts and delivering to one maildir may look similar to: 116 117 # Set the maximum size of mail. 118 set maximum-size 128M 119 120 # An action to save to the maildir ~/mail/inbox. 121 action "inbox" maildir "%h/mail/inbox" 122 123 # Accounts: POP3, POP3S and IMAP. Note the double escaping of the '\' 124 # character in the password. If the port is omitted, the default 125 # ("pop3", "pop3s", "imap" or "imaps" in the services(5) db) is used. 126 account "pop3" pop3 server "my.pop3.server" 127 user "my-username" pass "my-password-with-a-\\-in-it" 128 account "pop3s" pop3s server "pop.googlemail.com" port 995 129 user "my-account@gmail.com" pass "my-password" 130 # If the 'folder "my-folder"' argument is omitted, fdm will fetch mail 131 # from the inbox. 132 account "imap" imap server "my.imap.server" 133 user "my-username" pass "my-password" folder "my-folder" 134 135 # Discard mail from Bob Idiot. Note that the regexp is an extended 136 # regexp, and case-insensitive by default. This action is a "lambda" or 137 # unnamed action, it is defined inline as part of the match rule. 138 match "^From:.*bob@idiot\\.net" in headers action drop 139 140 # Match all other mail and deliver using the 'inbox' action. 141 match all action "inbox" 142 143A simple initial configuration file without filtering, perhaps to replace 144fetchmail or getmail delivering to maildrop, may look similar to: 145 146 # Set the maximum size of mail. 147 set maximum-size 128M 148 149 # Action to pipe directly to maildrop. 150 action "maildrop" pipe "/usr/local/bin/maildrop" 151 152 # Account definitions. 153 account .... 154 155 # Send all mail to maildrop. 156 match all action "maildrop" 157 158To run fdm every half hour from cron, add something like this: 159 160*/30 * * * * /usr/local/bin/fdm -l fetch 161 162See the fdm.conf(5) man page or the rest of this manual for more detail of the 163configuration file format. 164 165### The configuration file 166 167fdm is controlled by its configuration file. It first searches for a .fdm.conf 168file in the invoking user's home directory. If that fails, fdm attempts to use 169/usr/local/etc/fdm.conf. The configuration file may also be specified using the '-f' 170command line option, see the section on that subject below. 171 172This section gives an overview of the configuration file syntax. Further 173details of syntax, and specific keywords, are covered in later sections. 174 175The configuration file has the following general rules: 176 177- Keywords are specified as unadorned lowercase words: match, action, all. 178- Strings are enclosed in double quotes (") or single quotes ('). In double 179 quoted strings, double quotes may be included by escaping them using the 180 backslash character (\). Backslashes must also be escaped ("\\") - this 181 applies to all such strings, including regexps and passwords. The special 182 sequence '\t' is replaced by a tab character. In single quoted strings no 183 escaping is necessary, but it is not possible to include a literal ' or a 184 tab character. 185- Comments are prefixed by the hash character (#) and continue to the end of 186 the line. 187- Whitespace is largely ignored. Lines may generally be split, concatenated 188 or indented as preferred. 189- Lists are usually specified as 'singular item' or 'plural { item item }', for 190 example: 'user "nicholas"', 'users { "nicholas" "bob" }'. The singular/plural 191 distinction is not required, it is recommended only to aid readability: 192 'user { "nicholas "bob" }' is also accepted. 193- Regexps are specified as normal strings without additional adornment other 194 than the "s (not wrapped in /s). All regexps are extended regexps. They are 195 case insensitive by default but may be prefixed with the 'case' keyword to 196 indicate case sensitivity is required. 197- Strings may be concatenated using plus: "a" + "b" is the same as "ab". This 198 is most useful to wrap strings across multiple lines. 199 200Definition/option lines generally follow the following basic form: 201 202 <keyword> <name or command> <parameters> 203 204Example lines that may appear in a configuration file are: 205 206 # This is a comment. 207 208 set lock-types flock 209 210 account "stdin" disabled stdin 211 212 action "strip-full-disclosure" 213 rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'" 214 215 match "^X-Mailing-List:.*linux-kernel@vger.kernel.org" in headers 216 or "^(To:|Cc:):.*@vger.kernel.org" in headers 217 action "linux-kernel" 218 219%%% Including other files 220 221The fdm configuration may be split into several files. Additional files may 222be referenced using the 'include' keyword: 223 224 include "my-include-file.conf" 225 226 include "/usr/local/etc/fdm.d/shared-conf-1.conf" 227 228%%% Macros 229 230Macros may be defined and used in the configuration file. fdm makes a 231distinction between macros which may hold a number (numeric macros) and 232those that hold a string (string macros). Numeric macros are prefixed with 233the percentage sign (%) and string by the dollar sign ($). Macros are 234defined using the equals operator (=): 235 236 %nummacro = 123 237 238 $strmacro = "a string" 239 240Macros may then be referenced in either a standalone fashion anywhere a string 241or number is expected, depending on the type of macro: 242 243 $myfile = "a-file" 244 include $myfile 245 246 %theage = 12 247 match age < %theage action "old-mail" 248 249Or embedded in a string by enclosing the macro name in {}s: 250 251 $myfile2 = "a-file2" 252 include "/usr/local/etc/${myfile2}" 253 254 %anum = 57 255 include "/usr/local/etc/file-number-%{anum}" 256 257Macros are not substituted in strings specified using single-quotes. 258 259%%% Testing macros 260 261The 'ifdef', 'ifndef' and 'endif' keywords may be used to include or omit 262sections of the configuration file depending on whether a macro is defined. An 263'ifdef' is followed by a macro name (including $ or % type specifier) and if 264that macro exists, all following statements up until the next endif are 265evaluated (accounts created, rules added, and so on), otherwise they are 266skipped. 'ifndef' is the inverse: if the macro exists, the statements are 267skipped, otherwise they are included. An example is: 268 269 ifdef $dropeverything 270 match all action drop 271 endif 272 273These keywords are particularly useful in conjunction with the '-D' command line 274option. Any statements between 'ifdef'/'ifndef' and 'endif' must still be valid 275syntax. 276 277%%% Shell commands 278 279The value of a shell command may be used at any point in the configuration file 280where fdm expects a string or number. Shell commands are invoked by enclosing 281them in $() or %(). They are executed when the configuration file is parsed 282and if $() is used, any output to stdout is treated as a literal string (as 283if the output was inserted directly in the file enclosed in double quotes); %() 284attempts to convert the output to a number. For example: 285 286 $mytz = $(date +%Z) 287 288 %two = %(expr 1 + 1) 289 290 $astring = "abc" + $(echo def) 291 292Parts of the command within double quotes (") are subject to tag and macro 293replacement as normal (so it is necessary to use %% if a literal % is required, 294see the section on tags below); parts outside double quotes or inside single 295quotes are not. 296 297### Invoking fdm 298 299fdm accepts a number of command line arguments and may be invoked as needed 300from the command line or by a mail transfer agent, such as sendmail, or at 301regular times using a program such as cron(8). 302 303%%% Temporary files 304 305As each mail is being processed, it is stored in a temporary file in /tmp, or 306if the TMPDIR environment variable exists in the directory it points to. 307 308fdm tries to queue a number of mails simultaneously, so that older can be 309delivered while waiting for the server to provide the next. The maximum length 310of the queue for each account is set by the 'queue-high' option (the default is 311two) and the maximum mail size accepted by the 'maximum-size' option (the 312default is 32 MB). In addition, the 'rewrite' action requires an additional 313temporary mail. Although fdm will fail rather than dropping mail if the disk 314becomes full, users should bear in mind the possibility and set the size of the 315temporary directory and the fdm options according to their needs. 316 317%%% Command line arguments 318 319The fdm command has the following synopsis: 320 321 fdm [-klmnqv] [-f conffile] [-u user] [-a account] [-x account] 322 [-D name=value] [fetch | poll | cache ...] 323 324The meaning of the flags are covered in the fdm(1) man page, but a brief 325description is given below. The flags are also mentioned at relevant points 326in the rest of this document. 327 328Flag Meaning 329-k Keep all mail (do not delete it from the server). This is useful for 330 testing delivery rules without risking mail ending up permanently 331 in the wrong place. 332-l Log to syslog(3) using the 'mail' facility rather than outputting to 333 stderr. 334-m Ignore the lock file. 335-n Run a syntax check on the configuration file and exit without fetching 336 any mail. 337-q Quiet mode. Don't print anything except errors. 338-v Print verbose debugging output. This option may be specified multiple 339 times for increasing levels of verbosity. Useful levels are -vv to 340 display the result of parsing the configuration file, and -vvvv to copy 341 all traffic to and from POP3 or IMAP servers to stdout (note that -l 342 disables this behaviour). 343-f conffile 344 Specify the path of the configuration file. 345-u user 346 Use 'user' as the default user for delivering mail when started as 347 root. 348-a account 349 Process only accounts with a name matching the given pattern. Note that 350 fnmatch(3) wildcards may be used to match multiple accounts with one 351 option, and that the option may be specified multiple times. 352-x account 353 Process all accounts except those that match the given pattern. Again, 354 fnmatch(3) wildcards may be used, and the -x option may be specified 355 multiple times. 356-D name=value 357 Define a macro. The macro name must be prefixed with '$' or '%' to 358 indicate if it is a string or numeric macro. Macros defined on the 359 command line override any macros with the same name defined in the 360 configuration file. 361 362If -n is not specified, the flags must be followed by one of the keywords 363'fetch' or 'poll' or 'cache'. The 'fetch' keyword will fetch and deliver mail, 364the 'poll' keyword print an indication of how many mails are present in each 365account, and the 'cache' keyword is followed by one of a set of cache commands 366used to manipulate caches from the command-line (see the sections on caches 367below). 'fetch' or 'poll' or 'cache' may be abbreviated. 368 369Examples: 370 371 $ fdm -v poll 372 373 $ fdm -vvnf /usr/local/etc/my-fdm.conf 374 375 $ fdm -lm -a pop3\* fetch 376 377 $ fdm -x stdinacct fetch 378 379 # fdm -u nicholas -vv f 380 381%%% Running from cron 382 383To fetch mail regularly, fdm must be run from cron. This line in a crontab(5) 384will run fdm every 30 minutes: 385 386*/30 * * * * /usr/local/bin/fdm -l fetch 387 388The '-l' option sends fdm's output to syslog(3) rather than having cron mail 389it. To keep a closer eye, adding '-v' options and removing '-l' will have 390debugging output mailed by cron, or, using a line such as: 391 392*/30 * * * * fdm -vvvv fetch >>/home/user/.fdm.log 2>&1 393 394Will append extremely verbose fdm output to the ~/.fdm.log file. Note that this 395log file can become pretty large, so another cronjob may be required to remove 396it occasionally! 397 398%%% The lock file 399 400fdm makes use of a lock file to prevent two instances running simultaneously. 401By default, this lock file is .fdm.lock in the home directory of the user who 402runs fdm, or /var/db/fdm.lock for root. This default may be overridden in 403the configuration file with the 'set lock-file' command: 404 405 set lock-file "/path/to/my/lock-file" 406 407Or disabled altogether by being set to the empty string: 408 409 set lock-file "" 410 411The '-m' command line option may be used to force fdm to ignore the lock file 412and run regardless of its existence and without attempting to create it. 413 414%%% Testing and debugging 415 416fdm has some features to assist with testing and debugging a ruleset: 417 418The '-n' command line option. This is particularly useful in conjunction with 419'-vv', for example: 420 421 $ cat test.conf 422 account "pop3" pop3 server "s" user "u" pass "p" 423 action "rw" rewrite "sed 's/\\(Subject:.*\\)\\[XYZ\\]/\1/'" 424 action "mbox" mbox "%h/INBOX" 425 match all actions { "rw" "mbox" } 426 $ fdm -vvnf test.conf 427 version is: fdm 0.6 (20061204-1433) 428 starting at: Tue Dec 5 15:45:41 2006 429 user is: nicholas, home is: /home2/nicholas 430 loading configuration from test.conf 431 added account: name=pop3 fetch=pop3 server "s" port pop3 user "u" 432 added action: name=rw deliver=rewrite "sed 's/\(Subject:.*\)\[XYZ\]/1/'" 433 added action: name=mbox deliver=mbox "%h/INBOX" 434 finished file test.conf 435 added rule: actions="rw" "mbox" matches=all 436 configuration loaded 437 locking using: flock 438 headers are: "to" "cc" 439 domains are: "yelena" 440 using tmp directory: /tmp 441 442Looking at the output, the parsed strings used by fdm can be seen, and it is 443possible to spot that an escape character has been missed in the command. 444 445If '-vvvv' is used, fdm will print all data sent to and received from remote 446servers to stdout. Note that this is disabled if the '-l' option is given, and 447includes passwords, usernames and hostnames unmodified. The 'fdm-sanitize' 448script provided with fdm may be used to remove passwords and usernames from 449this output, either while it is being collected: 450 451 fdm -vvvv -a testacct f 2>&1|./fdm-sanitize|tee my-output 452 453Or afterwards: 454 455 ./fdm-sanitize <vvvv-output >my-output 456 457Since fdm fetches multiple accounts simultaneously, which may intersperse 458debugging output, it is recommended to fetch each account seperately if running 459the output through fdm-sanitize. If this is not done, it may not be able to 460detect all usernames or passwords. 461 462The '-k' command line option (and the 'keep' keywords on actions and accounts, 463covered later) prevent fdm from deleting any mail after delivery. This may be 464used to perform any number of test deliveries without risk of losing mail. 465 466### Fetching mail 467 468fdm fetches mail from a set of 'accounts', defined using the 'account' 469keyword. Each account has a name, a type, a number of account specific 470parameters and a couple of optional flags. The general form is: 471 472 account <name> [<users>] [disabled] <type> [<parameters>] [keep] 473 474The <name> item is a string by which the account is referred in filtering 475rules, log output and for the '-a' and '-x' command line options. 476 477The <users> portion specifies the default users to use when delivering mail 478fetched from this account as root. It has the same syntax as discussed in 479detail in the section below on defining actions. 480 481If the optional 'disabled' keyword is present, fdm ignores the account unless 482it is specified on the command line using the '-a' flag. 483 484The optional 'keep' keyword instructs fdm to keep all mail from this account 485(not delete it from the server) regardless of the result of the filtering 486rules. 487 488The <type> item may be one of: 'pop3', 'pop3s', 'imap', 'imaps', 'stdin', 489'maildir' or 'maildirs'. 490 491%%% Mail tags 492 493As mail is processed by fdm, it is tagged with a number of name/value pairs. 494Some tags are added automatically, and mail may also be tagged explicitly by 495the user (see the later tagging section). Tags may be inserted in strings in a 496similar manner to macros, except tags are processed when the string is used 497rather than always as the configuration file is parsed. A tag's value is 498inserted by wrapping its name in %[], for example: 499 500 match string "%[account]" to "myacct" action "myacctact" 501 502Most of the default tags have a single-letter shorthand which removes the needs 503for the []s: 504 505 match string "%a" to "myacct" action "myacctact" 506 507Including a nonexistent tag in a string is equivalent to including a tag with 508an empty value, so "abc%[nonexistent]def" will be translated to "abcdef". 509 510The automatically added tags are: 511 512Name Shorthand Replaced with 513account %a The name of the account from which the mail was 514 fetched. 515home %h The delivery user's home directory. 516uid %n The delivery user's uid. 517action %t The name of the action the mail has matched. 518user %u The delivery user's username. 519hour %H The current hour (00-23). 520minute %M The current minute (00-59). 521second %S The current second (00-59). 522day %d The current day of the month (00-31). 523month %m The current month (01-12). 524year %y The current year as four digits. 525year2 The current year as two digits. 526dayofweek %W The current day of the week (0-6, Sunday is 0). 527dayofyear %Y The current day of the year (000-365). 528quarter %Q The current quarter (1-4). 529rfc822date The current time in RFC822 date format. 530mail_hour The hour from the mail's date header, converted 531 to local time, if it exists and is valid, 532 otherwise the current time. 533mail_minute The minute from the mail's date header. 534mail_second The second from the mail's date header. 535mail_day The day from the mail's date header. 536mail_month The month from the mail's date header. 537mail_year The year from the mail's date header as four 538 digits. 539mail_year2 The same as two digits. 540mail_rfc822date The mail date in RFC822 format. 541hostname The local hostname. 542 543In addition, the shorthand %% is replaced with a literal %, and %1 to %9 are 544replaced with the result of any bracket expressions in the last regexp (see 545later section on regexps). A leading ~ or ~user is expanded in strings where a 546path or command is expected. 547 548Some accounts add additional tags, discussed below. 549 550Tags are replaced in almost all strings (including those in single-quotes!), 551some when the configuration file is parsed and some when the string is used. 552 553%%% POP3 and POP3S 554 555Mail may be fetched from a POP3 account. A POP3 account is defined by 556specifying the following parameters: the server host and optionally port, and 557optionally the user name and password. If the port is not specified, the 558default port ('pop3' in the services(5) database) is used. If the user name, 559password, or both is omitted, fdm attempts to look it up the .netrc file, see 560the next section for details. Optionally fdm can read the password from a 561command line program, see below for details. 562 563Examples of a POP3 account definition are: 564 565 account "pop3acct" pop3 server "pop.isp.com" user "bob" pass "pass" 566 567 account "gmx" pop3 server "pop.gmx.net" port 110 user "jim" pass "pass" 568 569 account "acct" pop3 server "10.0.0.1" port "pop3" 570 user "nicholas" keep 571 572 account "lycos" disabled pop3 server $localserver port 10110 573 pass "password" 574 575Note that the server string is enclosed in double quotes even if it is an IP, 576and don't forget to escape any " and \ characters in passwords! 577 578fdm will attempt to use APOP to obscure the password, if the server offers it. 579If the server advertises itself as supporting APOP but subsequently refuses 580to accept it, fdm will not retry with a cleartext password. Use of APOP can be 581disabled for an account using the 'no-apop' flag, for example: 582 583 account "acct" pop3 server "server" user "bob" pass "pass" no-apop 584 585The 'starttls' keyword may be added to a POP3 account to attemp STARTTLS after 586connection. 587 588POP3S is specified in exactly the same way, except using the 'pop3s' keyword 589for the type, and the default port is 'pop3s' rather than 'pop3': 590 591 account "pop3sacct" pop3s server "pop.isp.com" user "bob" pass "pass" 592 593POP3 accounts automatically tag mail with 'server' and 'port' tags, with the 594value of the server and port attributes exactly as specified in the account 595definition. A 'server_uid' tag is also added with the server unique id (UIDL). 596 597POP3 adds 'lines', 'body_lines' and 'header_lines' tags with the number of 598lines in the complete mail and its body and header. These tags are not updated 599to reflect any changes made to the mail by fdm rules. 600 601%%% SSL certificate verification 602 603fdm can verify SSL certificates before collecting mail from an SSL server. This 604is enabled globally with the 'verify-certificates' option: 605 606 set verify-certificates 607 608And may be disabled per-account using the 'no-verify' keyword (this applies to 609both POP3S and IMAPS accounts): 610 611 account "pop3sacct" pop3s server "pop.isp.com" no-verify 612 613For an introduction to SSL, see: 614 615 http://httpd.apache.org/docs/2.0/ssl/ssl_intro.html 616 617A cert bundle is required to verify SSL certificate chains. For more information 618see: 619 620 http://lynx.isc.org/current/README.sslcerts 621 622A pregenerated bundle is available courtesy of the MirOS project: 623 624 http://cvs.mirbsd.de/src/usr/local/etc/ssl.certs.shar 625 626%%% The .netrc file 627 628If the user name or password is omitted in POP3 or IMAP account definitions, 629fdm will attempt to look it up in the .netrc file in the invoking user's home 630directory. 631 632The .netrc file format is shared with ftp(1) and some other programs. It 633consists of a number of 'machine' sections and optionally one 'default' section 634containing a username ('login') and password for that host. fdm accepts entries 635only if the machine name matches the POP3 or IMAP server string exactly. If no 636matches are found and a 'default' section exists, it is used. 637 638An example .netrc file is: 639 640 machine "my.mail-server.com" 641 login "nicholas" 642 password "abcdef" 643 644 machine "pop.googlemail.com" 645 password "pass1" 646 647 default 648 login "bob" 649 password "moo" 650 651fdm will abort if the .netrc file is world-writable or world-readable. 652 653%%% Passwords from a command 654 655fdm can read the password from a command by using command substitution 656with $(). For example: 657 658 user "fdm@example.com" pass $(gpg --quiet --decrypt ~/.password.gpg) 659 660%%% IMAP and IMAPS 661 662IMAP and IMAPS accounts are defined using exactly the same syntax as for POP3 663and POP3S, aside from using the 'imap' or 'imaps' keywords and that the default 664port is 'imap' or 'imaps'. There is also an additional, optional 'folders' 665option to specify the folders from which mail should be fetched. If omitted, 666fdm defaults to the inbox. 667 668Note that with IMAP and IMAPS, mail is still removed from the server unless the 669'keep' option is given, or the '-k' command line option used. 670 671Examples of IMAP and IMAPS accounts include: 672 673 account "imapacct" imap server "imap.server.ca" user "bob" pass "pass" 674 675 account "oldimap" disabled imaps server "192.168.0.1" port 10993 676 user "nicholas" pass "pass" folders { "Saved" "MyStuff" } 677 678 account "workspam" disabled imap server "my-work.ath.cx" 679 user "Nicholas" folder "Junk" 680 681By default, fdm prefers the CRAM-MD5 authentication method, since no passwords 682are sent in the clear. If the server does not advertise CRAM-MD5 capability, 683the older LOGIN method is used. For IMAPS connections (which use SSL), the 684LOGIN method is just as secure. Either of these methods may be disabled with 685the 'no-cram-md5' and 'no-login' options. 686 687The 'starttls' keyword may be added to an IMAP account to attemp STARTTLS after 688connection. 689 690As with POP3, IMAP adds the 'server', 'port', 'server_uid' and the three line 691count tags to mail. 692 693%%% IMAP or POP3 over a pipe or ssh 694 695Mail may be fetched using IMAP or POP3 via a pipe. This is particularly useful 696for fetching mail over ssh using public keys. 697 698For IMAP, a user and password may be supplied, but fdm will only use them if 699the server asks. If the connection is preauthenticated, the user and password 700are unnecessary. For POP3, a user and password must be supplied as usual: due 701to the lack of server name, it cannot be read from the .netrc file. 702 703Communication takes place via the pipe program's stdin and stdout. If any 704output is found on stderr, fdm will print it (or log it with '-l'). 705 706Examples are: 707 708 account "imapssh" imap pipe "ssh jim@myhost /usr/local/libexec/imapd" 709 710 account "imapssh2" imap pipe "/usr/bin/whatever" user "bob" pass "bah" 711 712 account "pop3local" pop3 713 pipe "/usr/local/bin/ipop3d" user "me" pass "foo" 714 715%%% stdin and local mail 716 717fdm may be configured to fetch mail from stdin, by specifying an account of 718type 'stdin', for example: 719 720 account "stdin" disabled stdin 721 722This is most useful to have fdm behave as a mail delivery agent. To configure 723it for single-user use with sendmail, the simplest method it to add: 724 725 "|/usr/local/bin/fdm -m -a stdin fetch" 726 727To the user's ~/.forward file (including the double quotes). Note the use of 728'-m' to prevent stdin delivery from interfering with any normal cronjob, and 729'-a' to specify that only the disabled "stdin" account should be fetched. 730 731stdin accounts add the three line count tags described in the POP3 section. 732 733%%% From maildirs and mboxes 734 735Fetching from maildirs allows fdm to be used to filter mail on the local 736machine. This is covered more detail in the later section on archiving and 737searching. 738 739Maildir accounts are specified as follows: 740 741 account "mymaildir" maildir "/path/to/dir" 742 743 account "mymaildirs" maildirs { "/path/to/dir1" "/path/to/dir2" } 744 745Shell glob wildcards may be included in the path names to match multiple 746maildirs, but every directory found must be a valid maildir. 747 748Maildir accounts tag mail with a 'maildir' tag which is the basename of the 749maildir. 750 751Fetching from mboxes is similar: 752 753 account "mybox" mbox "/path/to/mbox" 754 755 account "mymboxes" mboxes { "/path/to/mbox1" "/path/to/mbox2" } 756 757Note that if an mbox is modified (mail is dropped from it), sufficient disk 758space is required to create a temporary copy of the entire mbox. 759 760%%% Using NNTP and NNTPS 761 762fdm can fetch news messages from a news server using NNTP or NNTPS. News 763accounts are specified like so: 764 765 account "news1" nntp server "news.server.sk" port 119 766 group "comp.unix.bsd.openbsd.misc" 767 cache "%h/.fdm.cache/%[group]" 768 769 account "mynews" nntps server "ssl.news.server" port "nntps" 770 user "myuser" pass "mypass" 771 groups { "alt.test" "alt.humor.best-of-usenet" } 772 cache "%h/.fdm.cache" 773 774The cache is a file used to store details of the last article fetched. If only 775one group is supplied in the account definition, %[group] tags are replaced by 776the name of the group in the cache path. If multiple groups are provided, 777%[group] is removed. 778 779Note that whether a message is kept or deleted is irrelevent to NNTP, articles 780are always left on the server. The index and message-id of the last article 781is recorded in the cache file so that older articles are skipped when the a 782newsgroup is again fetched. This happens regardless of any 'keep' keywords or 783the '-k' command line option. 784 785As with POP3 and IMAP, NNTP accounts add the 'server' and 'port' tags to mail. 786In addition, a 'group' tag is added with the group name. This can ensure 787articles are matched purely on the group they are fetched from (trying to do 788this using headers is unreliable with cross-posted articles). For example: 789 790match account "news" { 791 match string "%[group]" to "comp.lang.c" action "news-%[group]" 792 match string "%[group]" to "comp.std.c" action "news-%[group]" 793 match all action drop 794} 795 796%%% New or old mail only 797 798With POP3 and IMAP, fdm can be set up to fetch only new or old mail. For POP3 799this is achieved by recording the current state of the server in a cache file, 800which is updated as each mail is fetched. For IMAP it makes use of the 'seen' 801server flag which is updated by the server after each mail is fetched. 802 803These options are specified as in the following examples. For POP3: 804 805 account "name" pop3 server "blah" new-only cache "~/.fdm-pop3-cache" 806 807 account "acct" pop3s server "my-server" user "bob" 808 new-only cache "my-server-pop3-cache" no-apop 809 810And for IMAP: 811 812 account "imap" imap server "blah" new-only 813 814 account "sslimap" imaps server "imaps.somewhere" 815 user "user" pass "pass" old-only no-verify 816 817Note that currently, when using this with IMAP, the server is permitted to flag 818the mail as 'seen' before fdm has successfully delivered it, so there is no 819guarantee that mail so marked has been delivered, only that it has been 820fetched. 821 822### Defining actions 823 824An action is a particular command to execute on a mail when it matches a 825filtering rule (see the next section on filtering mail). Actions are named, 826similar to accounts, and have a similar form: 827 828 action <name> [<users>] <type> <parameters> 829 830The <users> item may be either: 831 832- the keyword 'user' followed by a single username string or uid, such as: 833 834 user "nicholas" 835 836 user "1000" 837 838- the keyword 'users' followed by a list of users in {}s, for example: 839 840 users { "1001" "nicholas" } 841 842If users are specified, the action will be run once for each user, with fdm 843changing to that user before executing the action. Note that fdm will execute 844the action once for each user even when not started as root, but will not be 845able to change to the user. The user keyword is primarily of use in multiuser 846configurations. If users are present on an action, they override any specified 847by the account definition. 848 849If running as root and no user is specified on either the action or on the 850filtering rule (see the section on filtering below), the default user is 851used, see the '-u' command line option and the 'default-user' option in the 852setting options section 853 854%%% Drop and keep 855 856The simplest actions are the 'drop' and 'keep' actions. They have no parameters 857and are specified like this: 858 859 action "mydropaction" drop 860 861 action "mykeepaction" keep 862 863The 'drop' action arranges for mail to be dropped when rule evaluation is 864complete. Note that using 'drop' does not stop further evaluation if the 865filtering rule contains a 'continue' keyword, and it may be overridden by a 866'keep' option on the account or by the '-k' flag on the command line. 867 868The 'keep' action is similar to 'drop', but it arranges for the mail to be 869kept once rule evaluation is complete, rather than dropped. 870 871%%% Maildirs 872 873Mails may be saved to a maildir through a 'maildir' action, defined like so: 874 875 action "mymaildiraction" maildir "/path/to/maildir" 876 877If any component of the maildir path does not exist, it is created, unless the 878no-create option is specified. Mails saved to a maildir are tagged with a 879'mail_file' tag containing the full path to the file in which they were saved. 880 881%%% Mboxes 882 883An action to deliver to an mbox is defined in the same way as for a maildir: 884 885 action "mymboxaction" mbox "/path/to/mbox" 886 887The same % tokens are replaced in the path. If the mbox does not exist, it 888is created. Mboxes may optionally be gzip compressed by adding the 'compress' 889keyword: 890 891 action "mymboxaction" mbox "/path/to/mbox" compress 892 893fdm will append .gz to the mbox path (if it is not already present) and append 894compressed data. If the mbox exists but is not already compressed, uncompressed 895data will be appended. 896 897As with maildirs, if any component of the mbox path does not exist, it is 898created, unless the no-create option is set. Mails saved to an mbox are tagged 899with an 'mbox_file' tag with the path of the mbox. 900 901%%% IMAP and IMAPS 902 903An action may be defined to store mail in an IMAP folder. The specification is 904similar to the IMAP account definition. A server host and optionally port 905(default 'imap' or 'imaps') must be specified. A username and password may be 906supplied; if they are omitted, fdm will attempt to find a .netrc 907entry. Examples include: 908 909 action "myimapaction" imap server "imap.server" 910 911 action "myimapaction" imaps server "imap.server" 912 port "8993" user "user" pass "pass" folder "folder" 913 914 action "myimapaction" imaps server "imap.server" 915 user "user" pass "pass" no-verify no-login 916 917%%% SMTP 918 919An action may be defined to pass mail on over SMTP. The server host must be 920specified and optionally the port and string to pass to the server with the 921RCPT TO and MAIL FROM commands. If the port is not specified it defaults to 922"smtp". Examples include: 923 924 action "mysmtpaction" smtp server "smtp.server" 925 926 action "mysmtpaction" smtp server "smtp.server" port 587 927 928 action "mysmtpaction" smtp 929 server "smtp.server" port "submission" from "bob@server.com" 930 931 action "mysmtpaction" smtp server "smtp.server" to "me@somewhere" 932 933%%% Write, pipe, exec and append 934 935Actions may be defined to write or append a mail to a file, to pipe it to a 936shell command, or merely to execute a shell command. The append action appends 937to and write overwrites the file. % tokens are replaced in the file or command 938as for maildir and mbox actions. 939 940Examples are: 941 942 action "mywriteaction" write "/tmp/file" 943 944 action "myappendaction" append "/tmp/file" 945 946 action "mypipeaction" pipe "cat > /dev/null" 947 948 action "domaildirexec" exec "~/.fdm.d/my-special-script %[mail_file]" 949 950Pipe and exec commands are run as the command user (by default the user who 951invoked fdm). 952 953%%% stdout 954 955fdm can write mails directly to stdout, using the 'stdout' action: 956 957 action "so" stdout 958 959%%% Rewriting mail 960 961Mail may be altered by passing it to a rewrite action. This is similar to 962the pipe action, but the output of the shell command to stdout is reread by fdm 963and saved as a new mail. This is useful for such things as passing mail 964through a spam filter or removing or altering headers with sed. Note that 965rewrite only makes sense on filtering rules where the continue keyword is 966specified, or where multiple actions are used (see the next section for details 967of this). Possible rewrite action definitions are: 968 969 action "myspamaction" rewrite "bmf -p" 970 971 action "mysedaction" rewrite "sed 's/x/y/'" 972 973%%% Adding or removing headers 974 975Simple actions are provided to add a header to a mail: 976 977 action "lines" add-header "Lines" value "%[lines]" 978 979Or to remove all instances of a header from mail: 980 981 action "del-ua" remove-header "user-agent" 982 983 action "rmhdr" remove-header "x-stupid-header" 984 985 action "remove-headers" remove-headers { "X-*" "Another-Header" } 986 987%%% Tagging 988 989Mails may be assigned one of more tags manually using the tag action type. For 990example, 991 992 match account "my*" action tag "myaccts" 993 994 match "^User-Agent:[ \t]*(.*)" action tag "user-agent" value "%1" 995 996The tag is attached to the mail with the specified value, or no value if none 997is provided. 998 999%%% Compound actions 1000 1001Compound actions may be defined which perform multiple single actions. They 1002are similar to standard single actions but multiple actions are provided using 1003{}. For example, 1004 1005 action "multiple" { 1006 add-header "X-My-Header" value "Yay!" 1007 mbox "mbox2" 1008 } 1009 1010 action "myaction" users { "bob" "jim" } { 1011 rewrite "rev" 1012 maildir "%h/%u's maildir" 1013 } 1014 1015Compound action are executed from top-to-bottom, once for each user. Note that 1016the effects are cumulative: the second example above would deliver a mail 1017rewritten once to 'bob' and rewritten again (ie, twice) to 'jim'. If this is 1018not desired, seperate actions must be used. 1019 1020%%% Chained actions 1021 1022An action may call other named actions by reusing the 'action' keyword: 1023 1024 action "abc" action "def" 1025 1026 action "an_action" { 1027 rewrite "rev" 1028 action "another_action" 1029 action "yet_more_actions" 1030 } 1031 1032There is a hard limit of five chained actions in a sequence to prevent infinite 1033loops. 1034 1035### Filtering mail 1036 1037Mail is filtered by defining a set of filtering rules. These rules tie together 1038mail fetched from an account and passed to one or more actions. Rules are 1039evaluated from top-to-bottom of the file, and evaluation stops at the first 1040matching rule (unless the continue keyword is specified). 1041 1042The general form of a filtering rule is: 1043 1044 match <conditions> [<users>] <actions> [continue] 1045 1046The optional <users> item is specified as for an action definition. If users 1047are specified on a filtering (match) rule, they override any specified on the 1048action or account. 1049 1050The <conditions> item is set of conditions against which the match may be 1051specified, each condition returns true or false. Conditions are described in 1052the next few sections. Aside from the 'all' condition, which is a special case, 1053conditions may be chained as an expression using 'and' and 'or', in which case 1054they are evaluated from left to right at the same precedence, or prepended with 1055'not' to invert their outcome. 1056 1057The <actions> item is a list of actions to execute when this rule matches. 1058It is in the same list format: 'action "name"' or 'actions { "name1" "name2" }'. 1059It may also be a lambda (inline) action, see the section below. 1060 1061If a rule with the 'continue' keyword matches, evaluation does not stop after 1062the actions are executed, instead subsequent rules are matched. 1063 1064%%% Nesting rules 1065 1066Filtering rules may be nested by using the special form: 1067 1068 match <conditions> [<accounts>] { 1069 match ... 1070 } 1071 1072If the conditions on the outer rule match, the inner rules are evaluated. If 1073none of the inner rules match (or they all specify the 'continue' keyword) 1074evaluation continues outside to rules following the nested rule, otherwise it 1075stops. 1076 1077%%% Lambda actions 1078 1079Lambda actions are unnamed actions included inline as part of the filtering 1080rule. This can be convenient for actions which do not need to be used multiple 1081times. Lambda actions are specified as a combination of the rule and an action 1082definition. For example: 1083 1084 match all action maildir "mymaildir" 1085 1086 match all actions { 1087 rewrite "rev" 1088 tag "reversed" 1089 } continue 1090 1091%%% The all condition 1092 1093The all condition matches all mail. 1094 1095Examples include: 1096 1097 match all action "default" 1098 1099 match all rewrite "rewaction" continue 1100 1101%%% Matched and unmatched 1102 1103The matched and unmatched conditions are used to match mail that has matched 1104or has not matched previous rules and been passed on with the 'continue' 1105keyword. For example, 1106 1107 match "myregexp" action "act1" continue 1108 # This rule will match only mails that also matched the first. 1109 match matched action "act2" 1110 # This rule will match only mails that matched neither of the first two. 1111 match unmatched action "act3" 1112 1113%%% Matching by account 1114 1115The account condition matches a list of accounts from which the mail was 1116fetched. It is specified as either a single account ('account "name"') or a list 1117of accounts ('accounts { "name1" "name2" }'). fnmatch(3) wildcards may also be 1118used. Examples include: 1119 1120 match "blah" accounts { "pop3" "imap" } action "go!" 1121 1122 match matched and account "myacc" action drop 1123 1124%%% Matching a regexp 1125 1126Matching against a regexp is the most common form of condition. It takes the 1127following syntax: 1128 1129 [case] <regexp> [in headers|in body] 1130 1131The 'case' keyword instructs fdm to match the regexp case sensitively rather 1132than the default of case insensitivity. The 'in headers' or 'in body' keywords 1133make fdm search only the headers or body of each mail, the default is to match 1134the regexp against the entire mail. Any multiline headers are unwrapped onto 1135a single line before matching takes place and the process reversed afterwards. 1136The regexp itself is an extended regexp specified as a simple string, but care 1137must be taken to escape \s and "s properly. 1138 1139Examples include: 1140 1141 match "^From:.*bob@bobland\\.bob" in headers and 1142 account "pop3" action "act" 1143 1144 match ".*YAHOO.*BOOTER.*" in body action "junk" 1145 1146%%% Matching bracket expressions 1147 1148The results of any bracket expressions within the last regexp match are 1149remembered, and may be made use of using the 'string' condition, or used to 1150construct an action name, maildir or mbox path, etc. The bracket expressions 1151may be substituted using the %0 to %9 tokens. For example, 1152 1153 match "^From:.*[ \t]([a-z]*)@domain" in headers action "all" continue 1154 match string "%1" to "bob.*" action "bobmail" 1155 1156 match "^From:.*[ \t]([a-z]*)@domain" in headers action "all" continue 1157 match all action "%1mail" 1158 1159This is particularly useful in combination with nested rules (see later): 1160bracket expressions in a regexp on the outer rule may be compared on inner 1161rules. 1162 1163Note that %0 to %9 are used only for 'regexp' rules. Regexps that are part of 1164'command' rules use the 'command0' to 'command9' tags. 1165 1166%%% Matching by age or size 1167 1168Mail may be matched based on its age or size. An age condition is specified as 1169follows: 1170 1171 age [<|>] <age> [hours|minutes|seconds|days|months|years] 1172 1173If '<' is used, mail is matched if it is younger than the specified age. If '>', 1174if it is older. The <age> item may be a simple number of seconds, or suffixed 1175with a unit. Examples are: 1176 1177 match age < 3 months actions { "act1" "act2" } 1178 1179 match age > 100 hours action "tooold" 1180 1181The age is extracted from the 'Date' header, if possible. To match mails for 1182which the header was invalid, the following form may be used: 1183 1184 match age invalid action "baddate" 1185 1186The size condition is similar: 1187 1188 size [<|>] <size> [K|KB|kilobytes...] 1189 1190Where <size> is a simple number in bytes, or suffixed with 'K', 'M' or 'G' to 1191specify a size in kilobytes, megabytes or gigabytes, such as: 1192 1193 match size < 1K action "small" 1194 1195 match size > 2G action "whoa" 1196 1197%%% Using a shell command 1198 1199Mail may be matched using the result of a shell command. This condition follows 1200the form: 1201 1202 [exec|pipe] <command> returns (<return code>, [case] <stdout regexp>) 1203 1204If 'exec' is used, the command is executed. If 'pipe', the mail is piped to the 1205command's stdin. The <command> is a simple string. % tokens are replaced as 1206normal. 1207 1208Any of the <return code> or <stdout regexp> or both may be specified. The 1209<return code> is a simple number which is compared against the return code from 1210the command, the <stdout regexp> is a regexp that is matched case insensitively 1211against each line output by the command on stdout. The result of any bracket 1212expressions in the stdout regexp are saved as 'command0' to 'command9' tags on 1213the mail. 1214 1215Any output on stderr is logged by fdm, so 2>&1 must be included in the command 1216in order to apply the regexp to it. 1217 1218Examples: 1219 1220 match exec "true" (0, ) action "act" 1221 1222 match not pipe "grep Bob" (1, ) action "act" 1223 1224 match pipe "myprogram" (, "failed") actions { "act1" "act2" } 1225 1226 match exec "blah" (12, "^Out") action "meep" 1227 1228%%% Attachments 1229 1230There are five conditions available to filter based on the size, quantity, type 1231and name of attachments. They are all prefixed with the 'attachment' 1232keyword. Two compare the overall number of attachments: 1233 1234The 'attachment count' conditions matches if the number of attachments is 1235equal to, not equal to, less than or greater than the specified number: 1236 1237 match attachment count == 0 action "action" 1238 1239 match attachment count != 10 action "action" 1240 1241 match attachment count < 2 action "action" 1242 1243 match attachment count > 7 action "action" 1244 1245The 'attachment total-size' condition is similar, but compares the total 1246size of all the attachments in a mail: 1247 1248 match attachment total-size < 4096 kilobytes action "action" 1249 1250 match attachment total-size > 1M action "action" 1251 1252There are also three conditions which matches if any individual attachment 1253fulfils the condition: 'any-size' to match if any attachment is less than or 1254greater than the given size, and 'any-type' and 'any-name' which compare the 1255attachment MIME type and name attribute (if any) using fnmatch(3): 1256 1257 match attachment any-size < 2K action "action" 1258 1259 match attachment any-type "*/pdf" action "action" 1260 1261 match attachment any-name "*.doc" action "action" 1262 1263%%% Matching tags 1264 1265The existence of a tag may be tested for using the 'tagged' condition: 1266 1267 match tagged "mytag" action "a" 1268 1269 match tagged "ohno" and size >1K action drop 1270 1271Or the tags value matched using the 'string' match type (in a similar way to 1272matching bracket expressions): 1273 1274 match string "%[group]" to "comp.lang.c" action "clc" 1275 1276 match string "%u" to "bob" action "bob" 1277 1278%%% Using caches 1279 1280fdm has builtin support for maintaining a cache of string keys, including 1281appending to a cache, checking if a key is present in a cache, and expiring 1282keys from a cache once they reach a certain age. 1283 1284These caches should not be confused with the NNTP cache file. Key caches are 1285referenced by filename and must be declared before use: 1286 1287 cache "%h/path/to/cache" 1288 1289 cache "~/.fdm.db" expire 1 month 1290 1291If the expiry time is not specified, items are never expired from the cache. 1292 1293Once declared, keys may be added to the cache with the 'add-to-cache' action: 1294 1295 match all action add-to-cache "~/my-cache" key "%[message_id]" 1296 1297Or removed with the 'remove-from-cache' action: 1298 1299 match all action remove-from-cache "~/my-cache" key "%[message_id]" 1300 1301And the existence of a key in the cache may be tested for using the 'in-cache' 1302condition: 1303 1304 match in-cache "~/my-cache" key "%[message_id]" action "foundincache" 1305 1306Any string may be used as key, but the message-id is most often useful. Note 1307that the key may not be empty, so care must be taken with messages without 1308message-id (such as news posts fetched with NNTP). 1309 1310Caches may be used to elimate duplicate messages using rules similar to those 1311above: 1312 1313 $db = "~/.fdm-duplicates.db" 1314 $key = "%[message_id]" 1315 cache $db expire 2 weeks 1316 1317 match not string $key to "" { 1318 match in-cache $db key $key action maildir "%h/mail/duplicates" 1319 match all action add-to-cache $db key $key continue 1320 } 1321 1322%%% Cache commands 1323 1324fdm includes a number of commands to manipulate caches from the command-line. 1325These are invoked with the 'cache' keyword followed by a command. The 1326following commands are supported: 1327 1328cache add <path> <string> 1329cache remove <path> <string> 1330 These add or remove <string> as a key in the cache <path>. 1331 1332cache list [<path>] 1333 This lists the number of keys in a cache, or in all caches declared 1334 in the configuration file if <path> is omitted. 1335 1336cache dump <path> 1337 This dumps the contents of the cache <path> to stdout. Each key is 1338 printed followed by a space and the timestamp as Unix time. 1339 1340cache clear <path> 1341 Delete all keys from a cache. 1342 1343Examples: 1344 1345 $ fdm cache list 1346 /export/home/nicholas/.fdm.d/duplicates: 4206 keys 1347 1348 $ touch my-cache 1349 $ fdm cache dump my-cache 1350 $ fdm cache add my-cache test 1351 $ fdm cache dump my-cache 1352 test 1195072403 1353 1354### Setting options 1355 1356fdm has a number of options that control its behaviour. These are defined 1357using the set command: 1358 1359 set <option> [<value>] 1360 1361In addition to the options below, some environment variables may be used to 1362control fdm. If TMPDIR is present, its value will be used instead of /tmp for 1363saving temporary files. 1364 1365The possible options are: 1366 1367- maximum-size <size> 1368 1369This specifies the maximum size of mail that fdm will accept. The default is 32 1370MB. Note that fdm may be storing a number of mails simultaneously, up to the 1371'queue-high' setting (doubled if rewrite is used) for each account, so care 1372should be taken when increasing this option. 1373 1374If mail over the maximum size is encountered, fdm will abort with an error, 1375without deleting the mail from the server (unless 'delete-oversized' is set). 1376 1377- delete-oversized 1378 1379If this option is set, fdm will delete oversized mail from the server rather 1380than leaving it for the user to sort out. 1381 1382- queue-high <number> 1383 1384This sets the number of mails fdm will queue simultaneously. The default is 1385two. Once this limit is reached, fdm will cease fetching mail until the number 1386queued drops to the value of the 'queue-low' setting. 1387 1388fdm queues mails so that they may be processed while waiting for data from the 1389server. Changing this option can increase (or decrease) performance, it's 1390usefulness varies wildly with the ruleset, speed of connection and remote host, 1391and the local hardware. 1392 1393The maximum possible value is currently 50. To ensure fdm fetches and delivers 1394mail sequentially, set this option to one. 1395 1396- queue-low <number> 1397 1398This sets the number of mails fdm will wait for the queue to drop to before 1399restarting fetching after the 'queue-high' limit has been reached. The default 1400is three-quarters of the 'queue-high' setting. 1401 1402- allow-multiple 1403 1404This option makes fdm ignore the lock file, similar to the '-m' command line 1405option. 1406 1407- lock-file <path> 1408 1409This option specifies a string to use as the path of the lock file. For example: 1410 1411 set lock-file "/tmp/my-lock-file" 1412 1413- command-user <user> 1414 1415This specifies the user used to run exec and pipe actions. By default it is 1416the user who invoked fdm. 1417 1418- default-user <user> 1419 1420This specifies the default user to use if run as root and no users are specified 1421on the action or filtering rule. This option may be overriden with the '-u' 1422flag on the command line. 1423 1424- lock-types [fcntl] [flock] [dotlock] 1425 1426These specify the type of locking to use when writing to mboxes. fcntl and 1427flock locking are mutually exclusive, but dotlock may be used with either. 1428Some NFS servers do not support fcntl. The default is flock only. Example: 1429 1430 set lock-types fcntl dotlock 1431 1432- proxy <url> 1433 1434This specifies a URL to proxy outgoing connections through. See the section 1435on proxying below. 1436 1437- unmatched-mail [drop|keep] 1438 1439This option controls how fdm should deal with mail that reaches the end of 1440the ruleset (doesn't match any rules, or only rules with the 'continue' 1441keyword). If 'drop' is specified, such mail is dropped. If 'keep', it is 1442kept. The default is to keep the mail, and issue a warning. 1443 1444- purge-after <message count> 1445 1446The 'purge-after' option instructs fdm to attempt to purge deleted mail after 1447the specified number of mails has been fetched. This is useful to limit the 1448number of mails refetched on the next run if the connection fails. It can have 1449a large effect on performance, particularly if the message count is set to a 1450low number. 1451 1452Note that for POP3, purging deleted mail involves disconnecting and 1453reconnecting from the server; some POP3 servers refuse reconnections if too 1454many are made too quickly, so this option should be used with care. 1455 1456- no-received 1457 1458If this option is present, fdm will not insert a 'Received' header into fetched 1459mail. 1460 1461- no-create 1462 1463When this option is specified, fdm will not attempt to create maildir and 1464mboxes or directories above them. 1465 1466- file-umask [user|<umask>] 1467 1468This specifies the umask to use when creating files. 'user' means to use the 1469umask set when fdm is started, or it may be specified as a three-digit octal 1470number. The default is 077. 1471 1472- file-group [user|<group>] 1473 1474This option allows the default group ownership of files and directories created 1475by fdm to be specified. 'group' may be a group name string or a numeric gid. If 1476'user' is used, or if this option is not set in the configuration file, fdm 1477does not attempt to set the group of new files and directories. 1478 1479- timeout <time> 1480 1481This controls the maximum time to wait for a server to send data before closing 1482a connection. The default is 900 seconds. 1483 1484- verify-certificates 1485 1486This instructs fdm to verify SSL certificates for all SSL connections, see the 1487previous section on SSL certificate verification. 1488 1489- ignore-errors 1490 1491By default, fdm will stop processing mail and exit if any delivery fails; if 1492this option is set, it will instead continue and try the next mail when 1493delivery of one fails. 1494 1495### Archiving and searching mail 1496 1497As fdm can fetch from maildirs, it can be used to filter mail from one maildir 1498into another, for example to archive older mail (with drop) or to search for 1499mail matching a particular pattern (with keep). The 'age' condition, and the 1500%[maildir] and time/date tags are particularly useful for archiving. For 1501example, to archive all mail older than 30 days by quarter, something like this 1502may suffice (the account restriction can be dropped if being used in a 1503single-purpose configuration file): 1504 1505 account "archive" disabled maildir "source-maildir" 1506 match account "archive" and age > 30 days action mbox "%[maildir]q%Q" 1507 match account "archive" action keep 1508 1509Then, fdm may be run to move the mail: 1510 1511 fdm -vaarchive fetch 1512 1513To search mail, similar rules may be used, but all mail should be kept, in this 1514example by marking the account with 'keep' so that mail is kept no matter what 1515rules it matches: 1516 1517 account "search" disabled maildir "source-maildir" keep 1518 action "found" mbox "search-results" 1519 match "^From.*Bob" in headers account "search" action "found" 1520 match account "search" action keep 1521 1522All mail matching the regexp will be copied to the target mbox. There are 1523several other ways to write this ruleset. 1524 1525### Using fdm behind a proxy 1526 1527fdm may be used behind a proxy by specifying the proxy URL using the 'proxy' 1528option. HTTP and SOCKS5 proxies are supported: 1529 1530 set proxy "http://proxy.server/" 1531 1532 set proxy "http://proxy.server:port/" 1533 1534 set proxy "socks5://proxy.server/" 1535 1536 set proxy "socks5://proxy.server:port/" 1537 1538 set proxy "socks5://user@proxy.server/" 1539 1540 set proxy "socks5://user:pass@proxy.server/" 1541 1542Authentication is not supported for HTTP proxies. 1543 1544### Bug reports and queries 1545 1546Bug reports, queries, suggestions and code are best sent by email to: 1547 1548 nicholas.marriott@gmail.com 1549 1550Bug reports may also be registered on GitHub. 1551 1552### Frequently asked questions 1553 1554%%% Why? 1555 1556Two main reasons: 1557 15581. I didn't like the existing tools. That is not to say that other tools are 1559 bad or aren't useful, merely that fdm meets my needs better. 1560 1561And more importantly: 1562 15632. I disliked the fact that as a home user with a relatively simple setup I 1564 had to have five programs to deal with mail: sendmail, fetchmail, procmail, 1565 archivemail, mutt; all with different, variously broken configuration file 1566 syntaxes, and weird quirks. I now have three programs: sendmail, fdm and 1567 mutt, and fewer weird configurations to learn and potential problems. I can 1568 also do some quite complex things without reaching for additional tools like 1569 formail and reformail. 1570 1571%%% How do I write regexps? 1572 1573See the re_format(7) man page. It is online here (for OpenBSD): 1574 1575 http://www.openbsd.org/cgi-bin/man.cgi?re_format 1576 1577There are also a number of books and probably websites on the subject. 1578 1579%%% Keep doesn't work with GMail! 1580 1581This is because GMail is broken, see: 1582 1583 http://pyropus.ca/software/getmail/faq.html#faq-notabug-gmail-bug 1584 1585In addition, GMail IMAP doesn't set the \Seen flag when mail is fetched with 1586the UID FETCH BODY[] command (RFC3501 says that the "\Seen flag is implicitly 1587set"). fdm works around this bug by setting the flag explicitly when mail is 1588kept. 1589 1590%%% Why doesn't fdm run as a daemon? 1591 1592Because that is what cron is for: it comes as standard with all sensible 1593operating systems, and by using it I avoid a lot of code to deal with signals, 1594configuration reload, extra configuration options and general other crap 1595related to running all the time. Daemonising is for servers, for stuff that 1596needs to run periodically, use cron. 1597 1598%%% Why does fdm fork child processes when not running as root? 1599 1600Because one design is much cleaner and easier to work with than two. And it 1601has a negligible practical effect on performance in any case. 1602 1603%%% Can fdm get rid of that crap in []s some mailing lists add to subjects? 1604 1605Yes! Rewrite the mail using sed to remove the crap, for example: 1606 1607action "full-disclosure" { 1608 rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'" 1609 maildir "%h/mail/full-disclosure" 1610} 1611match "^List-Id:.*<full-disclosure\\.lists\\.grok\\.org\\.uk>" in headers 1612 action "full-disclosure" 1613 1614%%% I'm building on Linux and it complains about REG_STARTEND. 1615 1616Older versions (pre-2005) of glibc don't support the REG_STARTEND flag to 1617regexec(3). Either upgrade to a later version of glibc, or use PCRE. 1618 1619%%% Should I use PCRE or standard POSIX regexps? What's the difference? 1620 1621PCRE is a library providing "perl-compatible" regexps. These are broadly 1622compatible with POSIX extended regexps, but with a number of extensions based 1623on perl regexps. 1624 1625Unless you want or need some of the extensions, there is generally no 1626compelling reason to choose one over the other. PCRE is faster than some 1627POSIX regexp implementations, but few rulesets will include sufficient 1628regexps for this to make any difference. PCRE has had some security 1629problems, but most regexp implementations pose a similar risk for their 1630complexity if nothing else; fdm performs regexp matching as non-root in 1631any case. 1632 1633The Debian package and FreeBSD port both use PCRE by default. 1634 1635Because I hate maintaining and testing two sets of code, there is a strong 1636possibility that PCRE may become the fdm default at some point. 1637 1638================================================================================ 1639$Id$ 1640 1641