1=head1 NAME 2 3makepp_perl_performance -- How to make Perl faster 4 5=for vc $Id: perl_performance.pod,v 1.10 2012/02/07 22:26:15 pfeiffer Exp $ 6 7=head1 DESCRIPTION 8 9The biggest tuning gains will usually come from algorithmic improvements. But 10while these can be hard to find, there is also a lot you can do mechanically. 11 12Makepp is a big heavy-duty program, where speed is a must. A lot of effort 13has been put into optimizing it. This documents some general things we have 14found. Currently the concrete tests leading to these results have mostly been 15discarded, but I plan to gradually add them. 16 17If you are looking at how to speedup makepp (beyond the Perl code you put into 18your makefiles), look at L<makepp_speedup>. This page is completely 19independent of makepp, only intended to make our results available to the Perl 20community. Some of these measures are common sence, but you sometimes forget 21them. Others need measuring to believe them, so: 22 23 24 25=head2 Measure, don't guess 26 27=over 28 29=item Profile your program 30 31Makepp comes with a module F<profiler.pm> in its cvs repository. This is 32first run as a program on a copy(!) of your code, which it instruments. Then 33you run your copy and get configurable statistics per interval and a final 34total on the most frequently called functions and on the most time spent in 35functions (minus subcalls). Both are provided absolutely and in caller-callee 36pairs. (Documentation within.) 37 38This tells you which functions are the most promising candidates for tuning. 39It also gives you a hint where your algorithm might be wrong, either within 40surprisingly expensive functions, or through surprisingly frequent calls. 41 42=item Time your solution 43 44Either one of 45 46 perl -Mstrict -MBenchmark -we 'my <initialization>; timethis -10, sub { <code> }' 47 time perl -Mstrict -we 'my <initialization>; for( 0..999_999 ) { <code> }' 48 49when run on different variants of code you can think of, can give surprising 50results. Even small modifications can matter a lot. Be careful not to 51"measure" code that can get optimized away, because you discard the result, or 52because it depends on constants. 53 54Depending on your system, this will tell you in kb how fat Perl got: 55 56 perl -Mstrict -we '<build huge data>; system "ps -ovsz $$"' 57 58Below we only show the code within the C<-e> option as one liners. 59 60=back 61 62 63 64=head2 Regexps 65 66=over 67 68=item Use simple regexps 69 70Several matches combined with C<||> are faster than a big one with C<|>. 71 72=item Use precompiled regexps 73 74Instead of interpolating strings into regexps (except if the string will never 75change and you use the C<o> modifier), precompile the regexp with C<qr//> and 76interpolate that. 77 78=item Use (?:...) 79 80If you don't use what the grouping matches, don't make Perl save it with 81C<(...)>. 82 83=item Anchor at beginning of string 84 85Don't make Perl look through your whole string, if you want a match only at 86the beginning. 87 88=item Don't anchor at end after greedy 89 90If you have a C<*> or C<+> that will match till the end of string, don't put a 91C<$> after it. 92 93=item Use tr/// 94 95This is twice as fast as s/// when it is applicable. 96 97=back 98 99 100 101=head2 Functions 102 103=over 104 105=item Avoid object orientation 106 107Dynamic method lookup is slower in any language, and Perl, being loosely 108typed, can never do it at compile time. Don't use it, unless you need the 109benefit of polymorphism through inheritance. The following call methods are 110ordered from slowest to fastest: 111 112 $o->method( ... ); # searched in class of $o and its @ISA 113 Class::method( $o, ... ); # static function, new stack 114 Class::method $o, ...; # static function, new stack, checked at compile time 115 &Class::method; # static function, reuse stack 116 117This last form always possible if method (or normal function) takes no 118arguments. If it does take arguments, watch out that you don't inadvertently 119supply any optional ones! If you use this form a lot, it is best to keep 120track of the minimum and maximum number of arguments each function can take. 121Reusing a stack with extra arguments is no problem, they'll get ignored. 122 123=item Don't modify stack 124 125The following sin is frequently found even in the Perl doc: 126 127 my $self = shift; 128 129Unless you have a pertinent reason for this, use this: 130 131 my( $self, $x, $y, @z ) = @_; 132 133=item Use few functions and modules 134 135Every function (and that alas includes constants) takes up over 1kb for it's 136mere existence. With each module requiring other ones, most of which you 137never need, that can add up. Don't pull in a big module, just to replace two 138lines of Perl code with a single more elegant looking function call. 139 140If you have a function only called in one place, and the two combined would 141still be reasonably short, merge them with due comments. 142 143Don't have one function only call another with the same arguments. Alias it 144instead: 145 146 *alias = \&function; 147 148=item Group calls to print 149 150Individual calls to print, or print with separate arguments are very 151expensive. Build up the string in memory and print it in one go. If you can 152accumulate over 3kb, syswrite is more efficient. 153 154 perl -MBenchmark -we 'timethis -10, sub { print STDERR $_ for 1..5 }' 2>/dev/null 155 perl -MBenchmark -we 'timethis -10, sub { print STDERR 1..5 }' 2>/dev/null 156 perl -MBenchmark -we 'timethis -10, sub { my $str = ""; $str .= $_ for 1..5; print STDERR $str }' 2>/dev/null 157 158=back 159 160 161 162=head2 Miscellaneous 163 164=over 165 166=item Avoid hashes 167 168Perl becomes quite slow with many small hashes. If you don't need them, use 169something else. Object orientation works just as well on an array, except 170that the members can't be accessed by name. But you can use numeric constants 171to name the members. For the sake of comparability we use plain numeric keys 172here: 173 174 my $i = 0; our %a = map +($i++, $_), "a".."j"; timethis -10, sub { $b = $a{int rand 10} } 175 our @a = "a".."j"; timethis -10, sub { $b = $a[rand 10] } 176 177 my $i = 0; my %a = map +($i++, $_), "a".."j"; timethis -10, sub { $b = $a{int rand 10} } 178 my @a = "a".."j"; timethis -10, sub { $b = $a[rand 10] } 179 180=item Use int keys for ref sets 181 182When you need a unique reference representation, e.g. for set ops with hashes, 183using the integer form of refs is three times as fast as using the pretty 184printed default string representation. Caveat: the HP/UX 64bitall variant of 185Perl, at least up to 5.8.8 has a buggy C<int> function, where this doesn't 186work reliably. There a hex form is still a fair bit faster than default 187strings. 188 189 my @list = map { bless { $_ => 1 }, "someclass" } 0..9; my( %a, %b ); 190 timethis -10, sub { $a{$_} = 1 for @list }; 191 timethis -10, sub { $b{int()} = 1 for @list }; 192 timethis -10, sub { $b{sprintf '%x', $_} = 1 for @list } 193 194=item Beware of strings 195 196Perl is awful for always copying strings around, even if you're never going to 197modify them. This wastes CPU and memory. Try to avoid that wherever 198reasonably possible. If the string is a function parameter and the function 199has a modest length, don't copy the string into a C<my> variable, access it 200with C<$_[0]> and document the function well. Elsewhere, the aliasing feature 201of C<for(each)> can help. Or just use references to strings, which are fast 202to copy. If you somehow ensure that same strings get stored only once, you 203can do numerical comparison for equality. 204 205=item Avoid bit operations 206 207If you have disjoint bit patterns you can add them instead of or`ing them. 208Shifting can be performed my multiplication or integer division. Retaining 209only the lowest bits can be achieved with modulo. 210 211Separate boolean hash members are faster than stuffing everything into an 212integer with bit operations or into a string with C<vec>. 213 214=item Use order of boolean operations 215 216If you only care whether an expression is true or false, check the cheap 217things, like boolean variables, first, and call functions last. 218 219=item Use undef instead of 0 220 221It takes up a few percent less memory, at least as hash or list values. You 222can still query it as a boolean. 223 224 my %x; $x{$_} = 0 for 0..999_999; system "ps -ovsz $$" 225 my %x; undef $x{$_} for 0..999_999; system "ps -ovsz $$" 226 227 my @x = (0) x 999_999; system "ps -ovsz $$" 228 my @x = (undef) x 999_999; system "ps -ovsz $$" 229 230=item Choose for or map 231 232These are definitely not equivalent. Depending on your use (i.e. the list and 233the complexity of your code), one or the other may be faster. 234 235 my @l = 0..99; 236 for( 0..99_999 ) { map $a = " $_ ", @l } 237 for( 0..99_999 ) { map $a = " $_ ", 0..99 } 238 for( 0..99_999 ) { $a = " $_ " for @l } 239 for( 0..99_999 ) { $a = " $_ " for 0..99 } 240 241=item Don't alias $_ 242 243While it is convenient, it is rather expensive, even copying reasonable 244strings is faster. The last example is twice as fast as the first "for". 245 246 my $x = "abcdefg"; my $b = 0; 247 for( "$x" ) { $b = 1 - $b if /g/ } # Copy needed only if modifying. 248 for( $x ) { $b = 1 - $b if /g/ } 249 local *_ = \$x; $b = 1 - $b if /g/; 250 local $_ = $x; $b = 1 - $b if /g/; # Copy cheaper than alias. 251 my $y = $x; $b = 1 - $b if $y =~ /g/; 252 253=back 254 255 256 257=head1 AUTHOR 258 259Daniel Pfeiffer <occitan@esperanto.org> 260