1<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN" 2 "http://www.w3.org/TR/html4/loose.dtd"> 3 4<html> 5 6<head> 7 8<title>Postfix Lookup Table Overview</title> 9 10<meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> 11 12</head> 13 14<body> 15 16<h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix 17Lookup Table Overview</h1> 18 19<hr> 20 21<h2>Overview </h2> 22 23This document covers the following topics: 24 25<ul> 26 27<li><a href="#intro">The Postfix lookup table model</a> 28 29<li><a href="#lists">Postfix lists versus tables </a> 30 31<li><a href="#preparing">Preparing Postfix for LDAP or SQL lookups</a> 32 33<li><a href="#detect">Maintaining Postfix lookup table files</a> 34 35<li><a href="#safe_db">Updating Berkeley DB files safely</a> 36 37<li><a href="#types">Postfix lookup table types</a> 38 39</ul> 40 41<h2><a name="intro">The Postfix lookup table model</a></h2> 42 43<p> Postfix uses lookup tables to store and look up information 44for access control, address rewriting and even for content filtering. 45All Postfix lookup tables are specified as "<a href="DATABASE_README.html">type:table</a>", where 46"type" is one of the database types described under "<a 47href="#types">Postfix lookup table types</a>" at the end of this 48document, and where "table" is the lookup table name. The Postfix 49documentation uses the terms "database" and "lookup table" for the 50same thing. </p> 51 52<p> Examples of lookup tables that appear often in the Postfix 53documentation: </p> 54 55<blockquote> 56<pre> 57/etc/postfix/<a href="postconf.5.html">main.cf</a>: 58 <a href="postconf.5.html#alias_maps">alias_maps</a> = hash:/etc/postfix/aliases (local aliasing) 59 <a href="postconf.5.html#header_checks">header_checks</a> = <a href="regexp_table.5.html">regexp</a>:/etc/postfix/header_checks (content filtering) 60 <a href="postconf.5.html#transport_maps">transport_maps</a> = hash:/etc/postfix/transport (routing table) 61 <a href="postconf.5.html#virtual_alias_maps">virtual_alias_maps</a> = hash:/etc/postfix/virtual (address rewriting) 62</pre> 63</blockquote> 64 65<p> All Postfix lookup tables store information as (key, value) 66pairs. This interface may seem simplistic at first, but it turns 67out to be very powerful. The (key, value) query interface completely 68hides the complexities of LDAP or SQL from Postfix. This is a good 69example of connecting complex systems with simple interfaces. </p> 70 71<p> Benefits of the Postfix (key, value) query interface:</p> 72 73<ul> 74 75<li> You can implement Postfix lookup tables first with local 76Berkeley DB files and then switch to LDAP or MySQL without any 77impact on the Postfix configuration itself, as described under "<a 78href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>" 79below. 80 81<li> You can use Berkeley DB files with fixed lookup strings for 82simple address rewriting operations and you can use regular expression 83tables for the more complicated work. In other words, you don't 84have to put everything into the same table. 85 86</ul> 87 88<h2><a name="lists">Postfix lists versus tables </a></h2> 89 90<p> Most Postfix lookup tables are used to look up information. 91Examples are address rewriting (the lookup string is the old address, 92and the result is the new address) or access control (the lookup 93string is the client, sender or recipient, and the result is an 94action such as "reject"). </p> 95 96<p> With some tables, however, Postfix needs to know only if the 97lookup key exists. The lookup result itself is not used. Examples 98are the <a href="postconf.5.html#local_recipient_maps">local_recipient_maps</a> that determine what local recipients 99Postfix accepts in mail from the network, the <a href="postconf.5.html#mydestination">mydestination</a> parameter 100that specifies what domains Postfix delivers locally, or the 101<a href="postconf.5.html#mynetworks">mynetworks</a> parameter that specifies the IP addresses of trusted 102clients or client networks. Technically, these are lists, not 103tables. Despite the difference, Postfix lists are described here 104because they use the same underlying infrastructure as Postfix 105lookup tables. </p> 106 107<h2><a name="preparing">Preparing Postfix for LDAP or SQL lookups</a> 108</h2> 109 110<p> LDAP and SQL are complex systems. Trying to set up both Postfix 111and LDAP or SQL at the same time is definitely not a good idea. 112You can save yourself a lot of time by implementing Postfix first 113with local files such as Berkeley DB. Local files have few surprises, 114and are easy to debug with the <a href="postmap.1.html">postmap(1)</a> command: </p> 115 116<blockquote> 117<pre> 118% <b>postmap -q info@example.com hash:/etc/postfix/virtual </b> 119</pre> 120</blockquote> 121 122<p> Once you have local files working properly you can follow the 123instructions in <a href="ldap_table.5.html">ldap_table(5)</a>, <a href="mysql_table.5.html">mysql_table(5)</a> or <a href="pgsql_table.5.html">pgsql_table(5)</a> 124and replace local file lookups with LDAP or SQL lookups. When you 125do this, you should use the <a href="postmap.1.html">postmap(1)</a> command again, to verify 126that database lookups still produce the exact same results as local 127file lookup: </p> 128 129<blockquote> 130<pre> 131% <b>postmap -q info@example.com <a href="ldap_table.5.html">ldap</a>:/etc/postfix/virtual.cf </b> 132</pre> 133</blockquote> 134 135<p> Be sure to exercise all the partial address or parent domain 136queries that are documented under "table search order" in the 137relevant manual page: <a href="access.5.html">access(5)</a>, <a href="canonical.5.html">canonical(5)</a>, <a href="virtual.5.html">virtual(5)</a>, 138<a href="transport.5.html">transport(5)</a>, or under the relevant configuration parameter: 139<a href="postconf.5.html#mynetworks">mynetworks</a>, <a href="postconf.5.html#relay_domains">relay_domains</a>, <a href="postconf.5.html#parent_domain_matches_subdomains">parent_domain_matches_subdomains</a>. </p> 140 141<h2><a name="detect">Maintaining Postfix lookup table files</a></h2> 142 143<p> When you make changes to a database while the mail system is 144running, it would be desirable if Postfix avoids reading information 145while that information is being changed. It would also be nice if 146you can change a database without having to execute "postfix reload", 147in order to force Postfix to use the new information. Each time 148you do "postfix reload" Postfix loses a lot of performance. 149</p> 150 151<ul> 152 153<li> <p> If you change a network database such as LDAP, NIS or 154SQL, there is no need to execute "postfix reload". The LDAP, NIS 155or SQL server takes care of read/write access conflicts and gives 156the new data to Postfix once that data is available. </p> 157 158<li> <p> If you change a <a href="regexp_table.5.html">regexp</a>: or <a href="pcre_table.5.html">pcre</a>: file then Postfix may or 159may not pick up the file changes immediately. This is because a 160Postfix process reads the entire file into memory once and never 161examines the file again. </p> 162 163<ul> 164 165<li> <p> If the file is used by a short-running process such as 166<a href="smtpd.8.html">smtpd(8)</a>, <a href="cleanup.8.html">cleanup(8)</a> or <a href="local.8.html">local(8)</a>, there is no need to execute 167"postfix reload" after making a change. </p> 168 169<li> <p> If the file is being used by a long-running process such 170as <a href="trivial-rewrite.8.html">trivial-rewrite(8)</a> on a busy server it may be necessary to 171execute "postfix reload". </p> 172 173</ul> 174 175<li> <p> If you change a local file based database such as DBM or 176Berkeley DB, there is no need to execute "postfix reload". Postfix 177uses file locking to avoid read/write access conflicts, and whenever 178a Postfix daemon process notices that a file has changed it will 179terminate before handling the next client request, so that a new 180process can initialize with the new database. </p> 181 182</ul> 183 184<h2><a name="safe_db">Updating Berkeley DB files safely</a></h2> 185 186<p> Although Postfix uses file locking to avoid access conflicts 187while updating Berkeley DB or other local database files, you still 188have a problem when the update fails because the disk is full or 189because something else happens. This is because commands such as 190<a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> overwrite existing files. If the update 191fails in the middle then you have no usable database, and Postfix 192will stop working. This is not an issue with the CDB database type 193available with Postfix 2.2 and later: <a href="CDB_README.html">CDB</a> 194creates a new file, and renames the file upon successful completion. 195</p> 196 197<p> With multi-file databases such as DBM, there is no simple 198solution. With Berkeley DB and other "one file" databases, it is 199possible to add some extra robustness by using "mv" to REPLACE an 200existing database file instead of overwriting it: </p> 201 202<blockquote> 203<pre> 204# <b>postmap access.in && mv access.in.db access.db</b> 205</pre> 206</blockquote> 207 208<p> This converts the input file "access.in" into the output file 209"access.in.db", and replaces the file "access.db" only when the 210<a href="postmap.1.html">postmap(1)</a> command was successful. Of course typing such commands 211becomes boring quickly, and this is why people use "make" instead, 212as shown below. User input is shown in bold font. </p> 213 214<blockquote> 215<pre> 216# <b>cat Makefile</b> 217all: aliases.db access.db virtual.db ...etcetera... 218 219# Note 1: commands are specified after a TAB character. 220# Note 2: use <a href="postalias.1.html">postalias(1)</a> for local aliases, <a href="postmap.1.html">postmap(1)</a> for the rest. 221aliases.db: aliases.in 222 postalias aliases.in 223 mv aliases.in.db aliases.db 224 225access.db: access.in 226 postmap access.in 227 mv access.in.db access.db 228 229virtual.db: virtual.in 230 postmap virtual.in 231 mv virtual.in.db virtual.db 232 233...etcetera... 234# <b>vi access.in</b> 235...editing session not shown... 236# <b>make</b> 237postmap access.in 238mv access.in.db access.db 239# 240</pre> 241</blockquote> 242 243<p> The "make" command updates only the files that have changed. 244In case of error, the "make" command will stop and will not invoke 245the "mv" command, so that Postfix will keep using the existing 246database file as if nothing happened. </p> 247 248<h2><a name="types">Postfix lookup table types</a> </h2> 249 250<p> To find out what database types your Postfix system supports, 251use the "<b>postconf -m</b>" command. Here is a list of database types 252that are often supported: </p> 253 254<blockquote> 255 256<dl> 257 258<dt> <b>btree</b> </dt> 259 260<dd> A sorted, balanced tree structure. This is available only on 261systems with support for Berkeley DB databases. Database files are 262created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The lookup 263table name as used in "btree:table" is the database file name 264without the ".db" suffix. </dd> 265 266<dt> <b>cdb</b> </dt> 267 268<dd> A read-optimized structure with no support for incremental updates. 269Database files are created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. 270The lookup table name as used in "<a href="CDB_README.html">cdb</a>:table" is the database file name 271without the ".cdb" suffix. This feature is available with Postfix 2.2 272and later. </dd> 273 274<dt> <b>cidr</b> </dt> 275 276<dd> A table that associates values with Classless Inter-Domain 277Routing (CIDR) patterns. The table format is described in <a href="cidr_table.5.html">cidr_table(5)</a>. 278</dd> 279 280<dt> <b>dbm</b> </dt> 281 282<dd> An indexed file type based on hashing. This is available only 283on systems with support for DBM databases. Database files are 284created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The lookup 285table name as used in "dbm:table" is the database file name without 286the ".dir" or ".pag" suffix. </dd> 287 288<dt> <b>environ</b> </dt> 289 290<dd> The UNIX process environment array. The lookup key is the 291variable name. The lookup table name in "environ:table" is ignored. 292</dd> 293 294<dt> <b>hash</b> </dt> 295 296<dd> An indexed file type based on hashing. This is available only 297on systems with support for Berkeley DB databases. Database files are 298created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The database 299name as used in "hash:table" is the database file name without the 300".db" suffix. </dd> 301 302<dt> <b>internal</b> </dt> 303 304<dd> A non-shared, in-memory hash table. Its content are lost when 305a process terminates. </dd> 306 307<dt> <b>ldap</b> (read-only) </dt> 308 309<dd> Perform lookups using the LDAP protocol. Configuration details 310are given in the <a href="ldap_table.5.html">ldap_table(5)</a>. </dd> 311 312<dt> <b>mysql</b> (read-only) </dt> 313 314<dd> Perform MySQL database lookups. Configuration details are given 315in <a href="mysql_table.5.html">mysql_table(5)</a>. </dd> 316 317<dt> <b>netinfo</b> (read-only) </dt> 318 319<dd> Perform Netinfo database lookups. </dd> 320 321<dt> <b>nis</b> (read-only) </dt> 322 323<dd> Perform NIS database lookups. </dd> 324 325<dt> <b>nisplus</b> (read-only) </dt> 326 327<dd> Perform NIS+ database lookups. Configuration details are given 328in <a href="nisplus_table.5.html">nisplus_table(5)</a>. </dd> 329 330<dt> <b>pcre</b> (read-only) </dt> 331 332<dd> A lookup table based on Perl Compatible Regular Expressions. 333The file format is described in <a href="pcre_table.5.html">pcre_table(5)</a>. The lookup table 334name as used in "<a href="pcre_table.5.html">pcre</a>:table" is the name of the regular expression 335file. </dd> 336 337<dt> <b>pgsql</b> (read-only) </dt> 338 339<dd> Perform PostgreSQL database lookups. Configuration details 340are given in <a href="pgsql_table.5.html">pgsql_table(5)</a>. </dd> 341 342<dt> <b>proxy</b> (read-only) </dt> 343 344<dd> Access information via the Postfix <a href="proxymap.8.html">proxymap(8)</a> service. The 345lookup table name syntax is "<a href="proxymap.8.html">proxy</a>:<a href="DATABASE_README.html">type:table</a>". </dd> 346 347<dt> <b>regexp</b> (read-only) </dt> 348 349<dd> A lookup table based on regular expressions. The file format 350is described in <a href="regexp_table.5.html">regexp_table(5)</a>. The lookup table name as used in 351"<a href="regexp_table.5.html">regexp</a>:table" is the name of the regular expression file. </dd> 352 353<dt> <b>sdbm</b> </dt> 354 355<dd> An indexed file type based on hashing. This is available only 356on systems with support for SDBM databases. Database files are 357created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The lookup 358table name as used in "sdbm:table" is the database file name without 359the ".dir" or ".pag" suffix. </dd> 360 361<dt> <b>static</b> (read-only) </dt> 362 363<dd> Always returns its lookup table name as lookup result. For 364example, the lookup table "static:foobar" always returns the string 365"foobar" as lookup result. </dd> 366 367<dt> <b>tcp</b> </dt> 368 369<dd> Access information through a TCP/IP server. The protocol is 370described in <a href="tcp_table.5.html">tcp_table(5)</a>. The lookup table name is "<a href="tcp_table.5.html">tcp</a>:host:port" 371where "host" specifies a symbolic hostname or a numeric IP address, 372and "port" specifies a symbolic service name or a numeric port 373number. 374</dd> 375 376<dt> <b>unix</b> (read-only) </dt> 377 378<dd> A limited way to query the UNIX authentication database. The 379following tables are implemented: 380 381<dl> 382 383<dt> <b>unix:passwd.byname</b> </dt> 384 385<dd>The table is the UNIX password database. The key is a login 386name. The result is a password file entry in passwd(5) format. 387</dd> 388 389<dt> <b>unix:group.byname</b> </dt> 390 391<dd> The table is the UNIX group database. The key is a group name. 392The result is a group file entry in group(5) format. </dd> 393 394</dl> </dd> 395 396</dl> 397 398</blockquote> 399 400<p> Other lookup table types may be available depending on how 401Postfix was built. With some Postfix distributions the list is 402dynamically extensible as support for lookup tables is dynamically 403linked into Postfix. </p> 404 405</body> 406 407</html> 408