1.\" Copyright (c) 2001 Matthew Dillon. Terms and conditions are those of 2.\" the BSD Copyright as specified in the file "/usr/src/COPYRIGHT" in 3.\" the source tree. 4.\" 5.\" $FreeBSD: src/share/man/man7/tuning.7,v 1.1.2.30 2002/12/17 19:32:08 dillon Exp $ 6.\" $DragonFly: src/share/man/man7/tuning.7,v 1.21 2008/10/17 11:30:24 swildner Exp $ 7.\" 8.Dd March 4, 2007 9.Dt TUNING 7 10.Os 11.Sh NAME 12.Nm tuning 13.Nd performance tuning under DragonFly 14.Sh SYSTEM SETUP - DISKLABEL, NEWFS, TUNEFS, SWAP 15When using 16.Xr disklabel 8 17or the 18.Dx 19installer 20to lay out your filesystems on a hard disk it is important to remember 21that hard drives can transfer data much more quickly from outer tracks 22than they can from inner tracks. 23To take advantage of this you should 24try to pack your smaller filesystems and swap closer to the outer tracks, 25follow with the larger filesystems, and end with the largest filesystems. 26It is also important to size system standard filesystems such that you 27will not be forced to resize them later as you scale the machine up. 28I usually create, in order, a 128M root, 1G swap, 128M 29.Pa /var , 30128M 31.Pa /var/tmp , 323G 33.Pa /usr , 34and use any remaining space for 35.Pa /home . 36.Pp 37You should typically size your swap space to approximately 2x main memory. 38If you do not have a lot of RAM, though, you will generally want a lot 39more swap. 40It is not recommended that you configure any less than 41256M of swap on a system and you should keep in mind future memory 42expansion when sizing the swap partition. 43The kernel's VM paging algorithms are tuned to perform best when there is 44at least 2x swap versus main memory. 45Configuring too little swap can lead 46to inefficiencies in the VM page scanning code as well as create issues 47later on if you add more memory to your machine. 48Finally, on larger systems 49with multiple SCSI disks (or multiple IDE disks operating on different 50controllers), we strongly recommend that you configure swap on each drive 51(up to four drives). 52The swap partitions on the drives should be approximately the same size. 53The kernel can handle arbitrary sizes but 54internal data structures scale to 4 times the largest swap partition. 55Keeping 56the swap partitions near the same size will allow the kernel to optimally 57stripe swap space across the N disks. 58Do not worry about overdoing it a 59little, swap space is the saving grace of 60.Ux 61and even if you do not normally use much swap, it can give you more time to 62recover from a runaway program before being forced to reboot. 63.Pp 64How you size your 65.Pa /var 66partition depends heavily on what you intend to use the machine for. 67This 68partition is primarily used to hold mailboxes, the print spool, and log 69files. 70Some people even make 71.Pa /var/log 72its own partition (but except for extreme cases it is not worth the waste 73of a partition ID). 74If your machine is intended to act as a mail 75or print server, 76or you are running a heavily visited web server, you should consider 77creating a much larger partition \(en perhaps a gig or more. 78It is very easy 79to underestimate log file storage requirements. 80.Pp 81Sizing 82.Pa /var/tmp 83depends on the kind of temporary file usage you think you will need. 84128M is 85the minimum we recommend. 86Also note that the 87.Dx 88installer will create a 89.Pa /tmp 90directory. 91Dedicating a partition for temporary file storage is important for 92two reasons: first, it reduces the possibility of filesystem corruption 93in a crash, and second it reduces the chance of a runaway process that 94fills up 95.Oo Pa /var Oc Ns Pa /tmp 96from blowing up more critical subsystems (mail, 97logging, etc). 98Filling up 99.Oo Pa /var Oc Ns Pa /tmp 100is a very common problem to have. 101.Pp 102In the old days there were differences between 103.Pa /tmp 104and 105.Pa /var/tmp , 106but the introduction of 107.Pa /var 108(and 109.Pa /var/tmp ) 110led to massive confusion 111by program writers so today programs haphazardly use one or the 112other and thus no real distinction can be made between the two. 113So it makes sense to have just one temporary directory and 114softlink to it from the other tmp directory locations. 115However you handle 116.Pa /tmp , 117the one thing you do not want to do is leave it sitting 118on the root partition where it might cause root to fill up or possibly 119corrupt root in a crash/reboot situation. 120.Pp 121The 122.Pa /usr 123partition holds the bulk of the files required to support the system and 124a subdirectory within it called 125.Pa /usr/pkg 126holds the bulk of the files installed from the 127.Xr pkgsrc 7 128collection. 129If you do not use 130.Xr pkgsrc 7 131all that much and do not intend to keep system source 132.Pq Pa /usr/src 133on the machine, you can get away with 134a 1 gigabyte 135.Pa /usr 136partition. 137However, if you install a lot of packages 138(especially window managers and Linux-emulated binaries), we recommend 139at least a 2 gigabyte 140.Pa /usr 141and if you also intend to keep system source 142on the machine, we recommend a 3 gigabyte 143.Pa /usr . 144Do not underestimate the 145amount of space you will need in this partition, it can creep up and 146surprise you! 147.Pp 148The 149.Pa /home 150partition is typically used to hold user-specific data. 151I usually size it to the remainder of the disk. 152.Pp 153Why partition at all? 154Why not create one big 155.Pa / 156partition and be done with it? 157Then I do not have to worry about undersizing things! 158Well, there are several reasons this is not a good idea. 159First, 160each partition has different operational characteristics and separating them 161allows the filesystem to tune itself to those characteristics. 162For example, 163the root and 164.Pa /usr 165partitions are read-mostly, with very little writing, while 166a lot of reading and writing could occur in 167.Pa /var 168and 169.Pa /var/tmp . 170By properly 171partitioning your system fragmentation introduced in the smaller more 172heavily write-loaded partitions will not bleed over into the mostly-read 173partitions. 174Additionally, keeping the write-loaded partitions closer to 175the edge of the disk (i.e. before the really big partitions instead of after 176in the partition table) will increase I/O performance in the partitions 177where you need it the most. 178Now it is true that you might also need I/O 179performance in the larger partitions, but they are so large that shifting 180them more towards the edge of the disk will not lead to a significant 181performance improvement whereas moving 182.Pa /var 183to the edge can have a huge impact. 184Finally, there are safety concerns. 185Having a small neat root partition that 186is essentially read-only gives it a greater chance of surviving a bad crash 187intact. 188.Pp 189Properly partitioning your system also allows you to tune 190.Xr newfs 8 , 191and 192.Xr tunefs 8 193parameters. 194Tuning 195.Xr newfs 8 196requires more experience but can lead to significant improvements in 197performance. 198There are three parameters that are relatively safe to tune: 199.Em blocksize , bytes/i-node , 200and 201.Em cylinders/group . 202.Pp 203.Dx 204performs best when using 8K or 16K filesystem block sizes. 205The default filesystem block size is 16K, 206which provides best performance for most applications, 207with the exception of those that perform random access on large files 208(such as database server software). 209Such applications tend to perform better with a smaller block size, 210although modern disk characteristics are such that the performance 211gain from using a smaller block size may not be worth consideration. 212Using a block size larger than 16K 213can cause fragmentation of the buffer cache and 214lead to lower performance. 215.Pp 216The defaults may be unsuitable 217for a filesystem that requires a very large number of i-nodes 218or is intended to hold a large number of very small files. 219Such a filesystem should be created with an 8K or 4K block size. 220This also requires you to specify a smaller 221fragment size. 222We recommend always using a fragment size that is \(18 223the block size (less testing has been done on other fragment size factors). 224The 225.Xr newfs 8 226options for this would be 227.Dq Li "newfs -f 1024 -b 8192 ..." . 228.Pp 229If a large partition is intended to be used to hold fewer, larger files, such 230as database files, you can increase the 231.Em bytes/i-node 232ratio which reduces the number of i-nodes (maximum number of files and 233directories that can be created) for that partition. 234Decreasing the number 235of i-nodes in a filesystem can greatly reduce 236.Xr fsck 8 237recovery times after a crash. 238Do not use this option 239unless you are actually storing large files on the partition, because if you 240overcompensate you can wind up with a filesystem that has lots of free 241space remaining but cannot accommodate any more files. 242Using 32768, 65536, or 262144 bytes/i-node is recommended. 243You can go higher but 244it will have only incremental effects on 245.Xr fsck 8 246recovery times. 247For example, 248.Dq Li "newfs -i 32768 ..." . 249.Pp 250.Xr tunefs 8 251may be used to further tune a filesystem. 252This command can be run in 253single-user mode without having to reformat the filesystem. 254However, this is possibly the most abused program in the system. 255Many people attempt to 256increase available filesystem space by setting the min-free percentage to 0. 257This can lead to severe filesystem fragmentation and we do not recommend 258that you do this. 259Really the only 260.Xr tunefs 8 261option worthwhile here is turning on 262.Em softupdates 263with 264.Dq Li "tunefs -n enable /filesystem" . 265(Note: in 266.Dx , 267softupdates can be turned on using the 268.Fl U 269option to 270.Xr newfs 8 , 271and 272.Dx 273installer will typically enable softupdates automatically for 274non-root filesystems). 275Softupdates drastically improves meta-data performance, mainly file 276creation and deletion. 277We recommend enabling softupdates on most filesystems; however, there 278are two limitations to softupdates that you should be aware of when 279determining whether to use it on a filesystem. 280First, softupdates guarantees filesystem consistency in the 281case of a crash but could very easily be several seconds (even a minute!) 282behind on pending writes to the physical disk. 283If you crash you may lose more work 284than otherwise. 285Secondly, softupdates delays the freeing of filesystem 286blocks. 287If you have a filesystem (such as the root filesystem) which is 288close to full, doing a major update of it, e.g.\& 289.Dq Li "make installworld" , 290can run it out of space and cause the update to fail. 291For this reason, softupdates will not be enabled on the root filesystem 292during a typical install. There is no loss of performance since the root 293filesystem is rarely written to. 294.Pp 295A number of run-time 296.Xr mount 8 297options exist that can help you tune the system. 298The most obvious and most dangerous one is 299.Cm async . 300Do not ever use it; it is far too dangerous. 301A less dangerous and more 302useful 303.Xr mount 8 304option is called 305.Cm noatime . 306.Ux 307filesystems normally update the last-accessed time of a file or 308directory whenever it is accessed. 309This operation is handled in 310.Dx 311with a delayed write and normally does not create a burden on the system. 312However, if your system is accessing a huge number of files on a continuing 313basis the buffer cache can wind up getting polluted with atime updates, 314creating a burden on the system. 315For example, if you are running a heavily 316loaded web site, or a news server with lots of readers, you might want to 317consider turning off atime updates on your larger partitions with this 318.Xr mount 8 319option. 320However, you should not gratuitously turn off atime 321updates everywhere. 322For example, the 323.Pa /var 324filesystem customarily 325holds mailboxes, and atime (in combination with mtime) is used to 326determine whether a mailbox has new mail. 327You might as well leave 328atime turned on for mostly read-only partitions such as 329.Pa / 330and 331.Pa /usr 332as well. 333This is especially useful for 334.Pa / 335since some system utilities 336use the atime field for reporting. 337.Sh STRIPING DISKS 338In larger systems you can stripe partitions from several drives together 339to create a much larger overall partition. 340Striping can also improve 341the performance of a filesystem by splitting I/O operations across two 342or more disks. 343The 344.Xr vinum 8 345and 346.Xr ccdconfig 8 347utilities may be used to create simple striped filesystems. 348Generally 349speaking, striping smaller partitions such as the root and 350.Pa /var/tmp , 351or essentially read-only partitions such as 352.Pa /usr 353is a complete waste of time. 354You should only stripe partitions that require serious I/O performance, 355typically 356.Pa /var , /home , 357or custom partitions used to hold databases and web pages. 358Choosing the proper stripe size is also 359important. 360Filesystems tend to store meta-data on power-of-2 boundaries 361and you usually want to reduce seeking rather than increase seeking. 362This 363means you want to use a large off-center stripe size such as 1152 sectors 364so sequential I/O does not seek both disks and so meta-data is distributed 365across both disks rather than concentrated on a single disk. 366If 367you really need to get sophisticated, we recommend using a real hardware 368RAID controller from the list of 369.Dx 370supported controllers. 371.Sh SYSCTL TUNING 372.Xr sysctl 8 373variables permit system behavior to be monitored and controlled at 374run-time. 375Some sysctls simply report on the behavior of the system; others allow 376the system behavior to be modified; 377some may be set at boot time using 378.Xr rc.conf 5 , 379but most will be set via 380.Xr sysctl.conf 5 . 381There are several hundred sysctls in the system, including many that appear 382to be candidates for tuning but actually are not. 383In this document we will only cover the ones that have the greatest effect 384on the system. 385.Pp 386The 387.Va kern.ipc.shm_use_phys 388sysctl defaults to 0 (off) and may be set to 0 (off) or 1 (on). 389Setting 390this parameter to 1 will cause all System V shared memory segments to be 391mapped to unpageable physical RAM. 392This feature only has an effect if you 393are either (A) mapping small amounts of shared memory across many (hundreds) 394of processes, or (B) mapping large amounts of shared memory across any 395number of processes. 396This feature allows the kernel to remove a great deal 397of internal memory management page-tracking overhead at the cost of wiring 398the shared memory into core, making it unswappable. 399.Pp 400The 401.Va vfs.write_behind 402sysctl defaults to 1 (on). This tells the filesystem to issue media 403writes as full clusters are collected, which typically occurs when writing 404large sequential files. The idea is to avoid saturating the buffer 405cache with dirty buffers when it would not benefit I/O performance. However, 406this may stall processes and under certain circumstances you may wish to turn 407it off. 408.Pp 409The 410.Va vfs.hirunningspace 411sysctl determines how much outstanding write I/O may be queued to 412disk controllers system wide at any given instance. The default is 413usually sufficient but on machines with lots of disks you may want to bump 414it up to four or five megabytes. Note that setting too high a value 415(exceeding the buffer cache's write threshold) can lead to extremely 416bad clustering performance. Do not set this value arbitrarily high! Also, 417higher write queueing values may add latency to reads occurring at the same 418time. 419.Pp 420There are various other buffer-cache and VM page cache related sysctls. 421We do not recommend modifying these values. 422As of 423.Fx 4.3 , 424the VM system does an extremely good job tuning itself. 425.Pp 426The 427.Va net.inet.tcp.sendspace 428and 429.Va net.inet.tcp.recvspace 430sysctls are of particular interest if you are running network intensive 431applications. 432They control the amount of send and receive buffer space 433allowed for any given TCP connection. 434The default sending buffer is 32K; the default receiving buffer 435is 64K. 436You can often 437improve bandwidth utilization by increasing the default at the cost of 438eating up more kernel memory for each connection. 439We do not recommend 440increasing the defaults if you are serving hundreds or thousands of 441simultaneous connections because it is possible to quickly run the system 442out of memory due to stalled connections building up. 443But if you need 444high bandwidth over a fewer number of connections, especially if you have 445gigabit Ethernet, increasing these defaults can make a huge difference. 446You can adjust the buffer size for incoming and outgoing data separately. 447For example, if your machine is primarily doing web serving you may want 448to decrease the recvspace in order to be able to increase the 449sendspace without eating too much kernel memory. 450Note that the routing table (see 451.Xr route 8 ) 452can be used to introduce route-specific send and receive buffer size 453defaults. 454.Pp 455As an additional management tool you can use pipes in your 456firewall rules (see 457.Xr ipfw 8 ) 458to limit the bandwidth going to or from particular IP blocks or ports. 459For example, if you have a T1 you might want to limit your web traffic 460to 70% of the T1's bandwidth in order to leave the remainder available 461for mail and interactive use. 462Normally a heavily loaded web server 463will not introduce significant latencies into other services even if 464the network link is maxed out, but enforcing a limit can smooth things 465out and lead to longer term stability. 466Many people also enforce artificial 467bandwidth limitations in order to ensure that they are not charged for 468using too much bandwidth. 469.Pp 470Setting the send or receive TCP buffer to values larger than 65535 will result 471in a marginal performance improvement unless both hosts support the window 472scaling extension of the TCP protocol, which is controlled by the 473.Va net.inet.tcp.rfc1323 474sysctl. 475These extensions should be enabled and the TCP buffer size should be set 476to a value larger than 65536 in order to obtain good performance from 477certain types of network links; specifically, gigabit WAN links and 478high-latency satellite links. 479RFC 1323 support is enabled by default. 480.Pp 481The 482.Va net.inet.tcp.always_keepalive 483sysctl determines whether or not the TCP implementation should attempt 484to detect dead TCP connections by intermittently delivering 485.Dq keepalives 486on the connection. 487By default, this is disabled for all applications, only applications 488that specifically request keepalives will use them. 489In most environments, TCP keepalives will improve the management of 490system state by expiring dead TCP connections, particularly for 491systems serving dialup users who may not always terminate individual 492TCP connections before disconnecting from the network. 493However, in some environments, temporary network outages may be 494incorrectly identified as dead sessions, resulting in unexpectedly 495terminated TCP connections. 496In such environments, setting the sysctl to 0 may reduce the occurrence of 497TCP session disconnections. 498.Pp 499The 500.Va net.inet.tcp.delayed_ack 501TCP feature is largely misunderstood. Historically speaking this feature 502was designed to allow the acknowledgement to transmitted data to be returned 503along with the response. For example, when you type over a remote shell 504the acknowledgement to the character you send can be returned along with the 505data representing the echo of the character. With delayed acks turned off 506the acknowledgement may be sent in its own packet before the remote service 507has a chance to echo the data it just received. This same concept also 508applies to any interactive protocol (e.g. SMTP, WWW, POP3) and can cut the 509number of tiny packets flowing across the network in half. The 510.Dx 511delayed-ack implementation also follows the TCP protocol rule that 512at least every other packet be acknowledged even if the standard 100ms 513timeout has not yet passed. Normally the worst a delayed ack can do is 514slightly delay the teardown of a connection, or slightly delay the ramp-up 515of a slow-start TCP connection. While we aren't sure we believe that 516the several FAQs related to packages such as SAMBA and SQUID which advise 517turning off delayed acks may be referring to the slow-start issue. 518.Pp 519The 520.Va net.inet.tcp.inflight_enable 521sysctl turns on bandwidth delay product limiting for all TCP connections. 522The system will attempt to calculate the bandwidth delay product for each 523connection and limit the amount of data queued to the network to just the 524amount required to maintain optimum throughput. This feature is useful 525if you are serving data over modems, GigE, or high speed WAN links (or 526any other link with a high bandwidth*delay product), especially if you are 527also using window scaling or have configured a large send window. If 528you enable this option you should also be sure to set 529.Va net.inet.tcp.inflight_debug 530to 0 (disable debugging), and for production use setting 531.Va net.inet.tcp.inflight_min 532to at least 6144 may be beneficial. Note, however, that setting high 533minimums may effectively disable bandwidth limiting depending on the link. 534The limiting feature reduces the amount of data built up in intermediate 535router and switch packet queues as well as reduces the amount of data built 536up in the local host's interface queue. With fewer packets queued up, 537interactive connections, especially over slow modems, will also be able 538to operate with lower round trip times. However, note that this feature 539only affects data transmission (uploading / server-side). It does not 540affect data reception (downloading). 541.Pp 542Adjusting 543.Va net.inet.tcp.inflight_stab 544is not recommended. 545This parameter defaults to 20, representing 2 maximal packets added 546to the bandwidth delay product window calculation. The additional 547window is required to stabilize the algorithm and improve responsiveness 548to changing conditions, but it can also result in higher ping times 549over slow links (though still much lower than you would get without 550the inflight algorithm). In such cases you may 551wish to try reducing this parameter to 15, 10, or 5, and you may also 552have to reduce 553.Va net.inet.tcp.inflight_min 554(for example, to 3500) to get the desired effect. Reducing these parameters 555should be done as a last resort only. 556.Pp 557The 558.Va net.inet.ip.portrange.* 559sysctls control the port number ranges automatically bound to TCP and UDP 560sockets. There are three ranges: A low range, a default range, and a 561high range, selectable via an IP_PORTRANGE setsockopt() call. Most 562network programs use the default range which is controlled by 563.Va net.inet.ip.portrange.first 564and 565.Va net.inet.ip.portrange.last , 566which defaults to 1024 and 5000 respectively. Bound port ranges are 567used for outgoing connections and it is possible to run the system out 568of ports under certain circumstances. This most commonly occurs when you are 569running a heavily loaded web proxy. The port range is not an issue 570when running serves which handle mainly incoming connections such as a 571normal web server, or has a limited number of outgoing connections such 572as a mail relay. For situations where you may run yourself out of 573ports we recommend increasing 574.Va net.inet.ip.portrange.last 575modestly. A value of 10000 or 20000 or 30000 may be reasonable. You should 576also consider firewall effects when changing the port range. Some firewalls 577may block large ranges of ports (usually low-numbered ports) and expect systems 578to use higher ranges of ports for outgoing connections. For this reason 579we do not recommend that 580.Va net.inet.ip.portrange.first 581be lowered. 582.Pp 583The 584.Va kern.ipc.somaxconn 585sysctl limits the size of the listen queue for accepting new TCP connections. 586The default value of 128 is typically too low for robust handling of new 587connections in a heavily loaded web server environment. 588For such environments, 589we recommend increasing this value to 1024 or higher. 590The service daemon 591may itself limit the listen queue size (e.g.\& 592.Xr sendmail 8 , 593apache) but will 594often have a directive in its configuration file to adjust the queue size up. 595Larger listen queues also do a better job of fending off denial of service 596attacks. 597.Pp 598The 599.Va kern.maxfiles 600sysctl determines how many open files the system supports. 601The default is 602typically a few thousand but you may need to bump this up to ten or twenty 603thousand if you are running databases or large descriptor-heavy daemons. 604The read-only 605.Va kern.openfiles 606sysctl may be interrogated to determine the current number of open files 607on the system. 608.Pp 609The 610.Va vm.swap_idle_enabled 611sysctl is useful in large multi-user systems where you have lots of users 612entering and leaving the system and lots of idle processes. 613Such systems 614tend to generate a great deal of continuous pressure on free memory reserves. 615Turning this feature on and adjusting the swapout hysteresis (in idle 616seconds) via 617.Va vm.swap_idle_threshold1 618and 619.Va vm.swap_idle_threshold2 620allows you to depress the priority of pages associated with idle processes 621more quickly than the normal pageout algorithm. 622This gives a helping hand 623to the pageout daemon. 624Do not turn this option on unless you need it, 625because the tradeoff you are making is to essentially pre-page memory sooner 626rather than later, eating more swap and disk bandwidth. 627In a small system 628this option will have a detrimental effect but in a large system that is 629already doing moderate paging this option allows the VM system to stage 630whole processes into and out of memory more easily. 631.Sh LOADER TUNABLES 632Some aspects of the system behavior may not be tunable at runtime because 633memory allocations they perform must occur early in the boot process. 634To change loader tunables, you must set their values in 635.Xr loader.conf 5 636and reboot the system. 637.Pp 638.Va kern.maxusers 639controls the scaling of a number of static system tables, including defaults 640for the maximum number of open files, sizing of network memory resources, etc. 641On 642.Dx , 643.Va kern.maxusers 644is automatically sized at boot based on the amount of memory available in 645the system, and may be determined at run-time by inspecting the value of the 646read-only 647.Va kern.maxusers 648sysctl. 649Some sites will require larger or smaller values of 650.Va kern.maxusers 651and may set it as a loader tunable; values of 64, 128, and 256 are not 652uncommon. 653We do not recommend going above 256 unless you need a huge number 654of file descriptors; many of the tunable values set to their defaults by 655.Va kern.maxusers 656may be individually overridden at boot-time or run-time as described 657elsewhere in this document. 658.Pp 659The 660.Va kern.dfldsiz 661and 662.Va kern.dflssiz 663tunables set the default soft limits for process data and stack size 664respectively. 665Processes may increase these up to the hard limits by calling 666.Xr setrlimit 2 . 667The 668.Va kern.maxdsiz , 669.Va kern.maxssiz , 670and 671.Va kern.maxtsiz 672tunables set the hard limits for process data, stack, and text size 673respectively; processes may not exceed these limits. 674The 675.Va kern.sgrowsiz 676tunable controls how much the stack segment will grow when a process 677needs to allocate more stack. 678.Pp 679.Va kern.ipc.nmbclusters 680may be adjusted to increase the number of network mbufs the system is 681willing to allocate. 682Each cluster represents approximately 2K of memory, 683so a value of 1024 represents 2M of kernel memory reserved for network 684buffers. 685You can do a simple calculation to figure out how many you need. 686If you have a web server which maxes out at 1000 simultaneous connections, 687and each connection eats a 16K receive and 16K send buffer, you need 688approximately 32MB worth of network buffers to deal with it. 689A good rule of 690thumb is to multiply by 2, so 32MBx2 = 64MB/2K = 32768. 691So for this case 692you would want to set 693.Va kern.ipc.nmbclusters 694to 32768. 695We recommend values between 6961024 and 4096 for machines with moderates amount of memory, and between 4096 697and 32768 for machines with greater amounts of memory. 698Under no circumstances 699should you specify an arbitrarily high value for this parameter, it could 700lead to a boot-time crash. 701The 702.Fl m 703option to 704.Xr netstat 1 705may be used to observe network cluster use. 706.Pp 707More and more programs are using the 708.Xr sendfile 2 709system call to transmit files over the network. 710The 711.Va kern.ipc.nsfbufs 712sysctl controls the number of filesystem buffers 713.Xr sendfile 2 714is allowed to use to perform its work. 715This parameter nominally scales 716with 717.Va kern.maxusers 718so you should not need to modify this parameter except under extreme 719circumstances. 720.Sh KERNEL CONFIG TUNING 721There are a number of kernel options that you may have to fiddle with in 722a large-scale system. 723In order to change these options you need to be 724able to compile a new kernel from source. 725The 726.Xr config 8 727manual page and the handbook are good starting points for learning how to 728do this. 729Generally the first thing you do when creating your own custom 730kernel is to strip out all the drivers and services you do not use. 731Removing things like 732.Dv INET6 733and drivers you do not have will reduce the size of your kernel, sometimes 734by a megabyte or more, leaving more memory available for applications. 735.Pp 736.Dv SCSI_DELAY 737may be used to reduce system boot times. 738The default is fairly high and 739can be responsible for 15+ seconds of delay in the boot process. 740Reducing 741.Dv SCSI_DELAY 742to 5 seconds usually works (especially with modern drives). 743.Pp 744There are a number of 745.Dv *_CPU 746options that can be commented out. 747If you only want the kernel to run 748on a Pentium class CPU, you can easily remove 749.Dv I486_CPU , 750but only remove 751.Dv I586_CPU 752if you are sure your CPU is being recognized as a Pentium II or better. 753Some clones may be recognized as a Pentium and not be able to boot 754without those options. 755If it works, great! 756The operating system 757will be able to better-use higher-end CPU features for MMU, task switching, 758timebase, and even device operations. 759Additionally, higher-end CPUs support 7604MB MMU pages, which the kernel uses to map the kernel itself into memory, 761increasing its efficiency under heavy syscall loads. 762.Sh IDE WRITE CACHING 763.Fx 4.3 764flirted with turning off IDE write caching. 765This reduced write bandwidth 766to IDE disks but was considered necessary due to serious data consistency 767issues introduced by hard drive vendors. 768Basically the problem is that 769IDE drives lie about when a write completes. 770With IDE write caching turned 771on, IDE hard drives will not only write data to disk out of order, they 772will sometimes delay some of the blocks indefinitely under heavy disk 773load. 774A crash or power failure can result in serious filesystem 775corruption. 776So our default was changed to be safe. 777Unfortunately, the 778result was such a huge loss in performance that we caved in and changed the 779default back to on after the release. 780You should check the default on 781your system by observing the 782.Va hw.ata.wc 783sysctl variable. 784If IDE write caching is turned off, you can turn it back 785on by setting the 786.Va hw.ata.wc 787loader tunable to 1. 788More information on tuning the ATA driver system may be found in the 789.Xr nata 4 790man page. 791.Pp 792There is a new experimental feature for IDE hard drives called 793.Va hw.ata.tags 794(you also set this in the boot loader) which allows write caching to be safely 795turned on. 796This brings SCSI tagging features to IDE drives. 797As of this 798writing only IBM DPTA and DTLA drives support the feature. 799Warning! 800These 801drives apparently have quality control problems and I do not recommend 802purchasing them at this time. 803If you need performance, go with SCSI. 804.Sh CPU, MEMORY, DISK, NETWORK 805The type of tuning you do depends heavily on where your system begins to 806bottleneck as load increases. 807If your system runs out of CPU (idle times 808are perpetually 0%) then you need to consider upgrading the CPU or moving to 809an SMP motherboard (multiple CPU's), or perhaps you need to revisit the 810programs that are causing the load and try to optimize them. 811If your system 812is paging to swap a lot you need to consider adding more memory. 813If your 814system is saturating the disk you typically see high CPU idle times and 815total disk saturation. 816.Xr systat 1 817can be used to monitor this. 818There are many solutions to saturated disks: 819increasing memory for caching, mirroring disks, distributing operations across 820several machines, and so forth. 821If disk performance is an issue and you 822are using IDE drives, switching to SCSI can help a great deal. 823While modern 824IDE drives compare with SCSI in raw sequential bandwidth, the moment you 825start seeking around the disk SCSI drives usually win. 826.Pp 827Finally, you might run out of network suds. 828The first line of defense for 829improving network performance is to make sure you are using switches instead 830of hubs, especially these days where switches are almost as cheap. 831Hubs 832have severe problems under heavy loads due to collision backoff and one bad 833host can severely degrade the entire LAN. 834Second, optimize the network path 835as much as possible. 836For example, in 837.Xr firewall 7 838we describe a firewall protecting internal hosts with a topology where 839the externally visible hosts are not routed through it. 840Use 100BaseT rather 841than 10BaseT, or use 1000BaseT rather than 100BaseT, depending on your needs. 842Most bottlenecks occur at the WAN link (e.g.\& 843modem, T1, DSL, whatever). 844If expanding the link is not an option it may be possible to use the 845.Xr dummynet 4 846feature to implement peak shaving or other forms of traffic shaping to 847prevent the overloaded service (such as web services) from affecting other 848services (such as email), or vice versa. 849In home installations this could 850be used to give interactive traffic (your browser, 851.Xr ssh 1 852logins) priority 853over services you export from your box (web services, email). 854.Sh SEE ALSO 855.Xr netstat 1 , 856.Xr systat 1 , 857.Xr nata 4 , 858.Xr dummynet 4 , 859.Xr login.conf 5 , 860.Xr rc.conf 5 , 861.Xr sysctl.conf 5 , 862.Xr firewall 7 , 863.Xr hier 7 , 864.Xr boot 8 , 865.Xr ccdconfig 8 , 866.Xr config 8 , 867.Xr disklabel 8 , 868.Xr fsck 8 , 869.Xr ifconfig 8 , 870.Xr ipfw 8 , 871.Xr loader 8 , 872.Xr mount 8 , 873.Xr newfs 8 , 874.Xr route 8 , 875.Xr sysctl 8 , 876.Xr tunefs 8 , 877.Xr vinum 8 878.Sh HISTORY 879The 880.Nm 881manual page was originally written by 882.An Matthew Dillon 883and first appeared 884in 885.Fx 4.3 , 886May 2001. 887