1.Dd $Mdocdate: September 22 2011 $ 2.Dt WORLDSTONE 1 3.Os 4.Sh NAME 5.Nm worldstone 6.Nd shell script to consistently execute benchmarks 7.Sh SYNOPSIS 8.Nm worldstone 9.Op Fl s 10.Op Fl n Ar iterations 11.Op Fl c Ar command 12.Op Fl p Ar command 13.Op Fl t Ar tag 14.Sh DESCRIPTION 15The 16.Nm 17utility is a shell script and takes care of some of the 18grunt work around benchmarking, in order to make it easier 19to get consistent and comparable benchmark results. Its basic 20operation is: execute the precommand, then execute and time 21the command, and do this a set number of iterations, and record 22the times in a logfile. 23 24Its features are: 25.Bl -tag -width Ds 26.It - 27It executes the precommand and command once without timing it 28in order to mitigate cold cache effects. 29.It - 30It allows a precommand to run before the command, so that the initial 31state can be set up by the precommand without it being part of the timing 32(e.g. make clean). 33.It - 34It redirects the stdout and stderr to /dev/null so that lots of output 35going over a network connection doesn't influence timing. 36.It - 37It does a sync before running the timed command, and makes sure a final 38sync is part of the timed command, to make the i/o more consistent. 39.It - 40It logs the times of each iteration in an easy-to-parse logfile. 41.It - 42It tries to guess a sensible log file name based on the current git 43branch in /usr/src. 44.El 45 46The options are as follows: 47.Bl -tag -width Ds 48.It Fl s 49If set, perform statistical profiling by invoking 50.Xr profile 1 51and executing sprofalyze, suitable for feeding to sprofdiff. This 52requires building world with SPROFILE. 53.It Fl n Ar iterations 54Set the number of iterations to perform, after the initial run. 55The default is 5. 56.It Fl c Ar command 57Set the command to run to be timed. This is passed to sh -c, so shell constructs 58like loops etc. are okay to do. Default: make all. 59.It Fl p Ar command 60Set the pre-command to run. This command gets run before the timed command in order 61to make the timed command get a consistent state before it starts. 62Default: make clean. 63.It Fl t Ar tag 64Use the given tag name to modify the logfile that the utility uses 65to write its results in. The default is just 'time' plus the git branch you 66are currently on in /usr/src. In order for this to be useful you have to make sure the 67git branch you are on reflects the system you wish to benchmark of course. 68The script checks /usr/src/.git even if you are outside the /usr/src hierarchy 69(such as in pkgsrc). 70.El 71 72The script executes the commands the set number of iterations, redirecting stdout 73and stderr to /dev/null, and records the timed results in the logfile tagged with 74the given tag. 75.Nm 76executes 77.Xr time 1 78with the -C option, resulting in printing the 64-bit cpu cycle counter 79for both HZ-independent high resolution and an easy way not to have to convert minutes 80and seconds to seconds when parsing the results. 81.Pp 82You can feed the two separate logfiles directly to 83.Xr ministat 1 84to have it tell you the statistical properties of the two datasets, and judge whether 85there is a statistically significant difference. 86.Sh ENVIRONMENT 87The default commands, i.e. make all and make clean, can be modified by supplying a MAKE 88environment variable, so e.g. MAKE=bmake worldstone still does something sensible 89by default in /usr/pkgsrc directories. 90.Sh SEE ALSO 91.Xr ministat 1 92