1# unreleased 2 3## Features 4 5## Changes 6 7## Bugfixes 8 9## Other 10 11## Packaging 12 13 14 15# v1.12.0 16 17## Features 18 19- `--command-name` can now take parameter names from `--parameter-*` options, see #351 and #391 (@silathdiir) 20- Exit codes (or signals) are now printed in cases of command failures, see #342 (@KaindlJulian) 21- Exit codes are now part of the JSON output, see #371 (@JordiChauzi) 22- Colorized output should now be enabled on Windows by default, see #427 23 24## Changes 25 26- When `--export-*` commands are used, result files are created before benchmark execution 27 to fail early in case of, e.g., wrong permissions. See #306 (@s1ck). 28- When `--export-*` options are used, result files are written after each individual 29 benchmark command instead of writing after all benchmarks have finished. See #306 (@s1ck). 30- Reduce number of shell startup time measurements from 200 to 50, generally speeding up benchmarks. See #378 31- User and system time are now in consistent time units, see #408 and #409 (@film42) 32 33 34 35# v1.11.0 36 37## Features 38 39- The `-L`/`--parameter-list` option can now be specified multiple times to 40 evaluate all possible combinations of the listed parameters: 41 42 ``` bash 43 hyperfine -L number 1,2 -L letter a,b,c \ 44 "echo {number}{letter}" \ 45 "printf '%s\n' {number}{letter}" 46 # runs 12 benchmarks: 2 commands (echo and printf) times 6 combinations of 47 # the "letter" and "number" parameters 48 ``` 49 50 See: #253, #318 (@wchargin) 51 52- Add CLI option to identify a command with a custom name, see #326 (@scampi) 53 54## Changes 55 56- When parameters are used with `--parameter-list` or `--parameter-scan`, the JSON export format 57 now contains a dictionary `parameters` instead of a single key `parameter`. See #253, #318. 58- The `plot_parametrized.py` script now infers the parameter name, and its `--parameter-name` 59 argument has been deprecated. See #253, #318. 60 61## Bugfixes 62 63- Fix a bug in the outlier detection which would only detect "slow outliers" but not the fast 64 ones (runs that are much faster than the rest of the benchmarking runs), see #329 65- Better error messages for very fast commands that would lead to inf/nan results in the relative 66 speed comparison, see #319 67- Show error message if `--warmup` or `--*runs` arguments can not be parsed, see #337 68- Keep output colorized when the output is not interactive and `--style=full` or `--style=color` is used. 69 70 71# v1.10.0 72 73## Features 74 75- Hyperfine now comes with shell completion files for Bash, Zsh, Fish 76 and PowerShell, see #290 (@four0000four). 77- Hyperfine now comes with a basic man page, see #257 (@cadeef) 78- During execution of benchmarks, hyperfine will now set a `HYPERFINE_RANDOMIZED_ENVIRONMENT_OFFSET` environment variable in order to randomize the memory layout. See #235 and #241 for references and details. 79- A few enhancements for the histogram plotting scripts and the 80 advanced statistics script 81- Updates for the `plot_whisker.py` script, see #275 (@ghaiklor) 82 83## Bugfixes 84 85- Fix Spin Icon on Windows, see #229 86- A few typos have been fixed, see #292 (@McMartin) 87 88## Packaging 89 90- `hyperfine` is now available on MacPorts for macOS, see #281 (@herbygillot) 91- `hyperfine` is now available on OpenBSD, see #289 (@minusf) 92 93Package authors: note that Hyperfine now comes with a set of shell completion files and a man page (see above) 94 95# v1.9.0 96 97## Features 98 99- The new `--parameter-list <VAR> <VALUES>` option can be used to run 100 a parametrized benchmark on a user-specified list of values. 101 This is similar to `--parameter-scan <VAR> <MIN> <MAX>`, but doesn't 102 necessarily required numeric arguments. 103 104 ``` bash 105 hyperfine --parameter-list compiler "gcc,clang" \ 106 "{compiler} -O2 main.cpp" 107 ``` 108 109 See: #227, #234 (@JuanPotato) 110 111- Added `none` as a possible choice for the `--style` option to 112 run `hyperfine` without any output, see #193 (@knidarkness) 113 114- Added a few new scripts for plotting various types of benchmark 115 results (https://github.com/sharkdp/hyperfine/tree/master/scripts) 116 117## Changes 118 119- The `--prepare` command is now also run during the warmup 120 phase, see #182 (@sseemayer) 121 122- Better estimation of the remaining benchmark time due to an update 123 of the `indicatif` crate. 124 125## Other 126 127- `hyperfine` is now available on NixOS, see #240 (@tuxinaut) 128 129# v1.8.0 130 131## Features 132 133- The `--prepare <CMD>` option can now be specified multiple times to 134 run specific preparation commands for each of the benchmarked programs: 135 136 ``` bash 137 hyperfine --prepare "make clean; git checkout master" "make" \ 138 --prepare "make clean; git checkout feature" "make" 139 ``` 140 141 See: #216, #218 (@iamsauravsharma) 142 143- Added a new [`welch_ttest.py`](https://github.com/sharkdp/hyperfine/blob/master/scripts/welch_ttest.py) script to test whether or not the two benchmark 144 results are the same, see #222 (@uetchy) 145 146- The Markdown export has been improved. The relative speed is now exported 147 with a higher precision (see #208) and includes the standard deviation 148 (see #225). 149 150## Other 151 152- Improved documentation for [`scripts`](https://github.com/sharkdp/hyperfine/tree/master/scripts) folder (@matthieusb) 153 154# v1.7.0 155 156## Features 157 158- Added a new `-D`,`--parameter-step-size` option that can be used to control 159 the step size for `--parameter-scan` benchmarks. In addition, decimal numbers 160 are now allowed for parameter scans. For example, the following command runs 161 `sleep 0.3`, `sleep 0.5` and `sleep 0.7`: 162 ``` bash 163 hyperfine --parameter-scan delay 0.3 0.7 -D 0.2 'sleep {delay}' 164 ``` 165 For more details, see #184 (@piyushrungta25) 166 167## Other 168 169- hyperfine is now in the official Alpine repositories, see #177 (@maxice8, @5paceToast) 170- hyperfine is now in the official Fedora repositories, see #196 (@ignatenkobrain) 171- hyperfine is now in the official Arch Linux repositories 172- hyperfine can be installed on FreeBSD, see #204 (@0mp) 173- Enabled LTO for slightly smaller binary sizes, see #179 (@Calinou) 174- Various small improvements all over the code base, see #194 (@phimuemue) 175 176# v1.6.0 177 178## Features 179 180- Added a `-c, --cleanup <CMD>` option to execute `CMD` after the completion of all benchmarking runs for a given command. This is useful if the commands to be benchmarked produce artifacts that need to be cleaned up. See #91 (@RalfJung and @colinwahl) 181- Add parameter values (for `--parameter-scan` benchmarks) to exported CSV and JSON files. See #131 (@bbannier) 182- Added AsciiDoc export option, see #137 (@5paceToast) 183- The relative speed is now part of the Markdown export, see #127 (@mathiasrw and @sharkdp). 184- The *median* run time is now exported via CSV and JSON, see #171 (@hosewiejacke and @sharkdp). 185 186## Other 187 188- Hyperfine has been updated to Rust 2018 (@AnderEnder). The minimum supported Rust version is now 1.31. 189 190# v1.5.0 191 192## Features 193 194- Show the number of runs in `hyperfine`s output (@tcmal) 195- Added two Python scripts to post-process exported benchmark results (see [`scripts/`](https://github.com/sharkdp/hyperfine/tree/master/scripts) folder) 196 197## Other 198 199- Refined `--help` text for the `--export-*` flags (@psteinb) 200- Added Snapcraft file (@popey) 201- Small improvements in the progress bar "experience". 202 203# v1.4.0 204 205## Features 206 207- Added `-S`/`--shell` option to override the default shell, see #61 (@mqudsi and @jasonpeacock) 208- Added `-u`/`--time-unit` option to change the unit of time (`second` or `millisecond`), see #80 (@jasonpeacock) 209- Markdown export auto-selects time unit, see #71 (@jasonpeacock) 210 211# v1.3.0 212 213## Feature 214 215- Compute and print standard deviation of the speed ratio, see #83 (@Shnatsel) 216- More compact output format, see #70 (@jasonpeacock) 217- Added `--style=color`, see #70 (@jasonpeacock) 218- Added options to specify the max/exact numbers of runs, see #77 (@orium) 219 220## Bugfixes 221 222- Change Windows `cmd` interpreter to `cmd.exe` to prevent accidentally calling other programs, see #74 (@tathanhdinh) 223 224## Other 225 226- Binary releases for Windows are now available, see #87 227 228# v1.2.0 229 230- Support parameters in preparation commands, see #68 (@siiptuo) 231- Updated dependencies, see #69. The minimum required Rust version is now 1.24. 232 233# v1.1.0 234 235* Added `--show-output` option (@chrisduerr and @sevagh) 236* Refactoring work (@stevepentland) 237 238# v1.0.0 239 240## Features 241 242* Support for various export-formats like CSV, JSON and Markdown - see #38, #44, #49, #42 (@stevepentland) 243* Summary output that compares the different benchmarks, see #6 (@stevepentland) 244* Parameterized benchmarks via `-P`, `--parameter-scan <VAR> <MIN> <MAX>`, see #19 245 246## Thanks 247 248I'd like to say a big THANK YOU to @stevepentland for implementing new features, 249for reviewing pull requests and for giving very valuable feedback. 250 251# v0.5.0 252 253* Proper Windows support (@stevepentland) 254* Added `--style auto/basic/nocolor/full` option (@stevepentland) 255* Correctly estimate the full execution time, see #27 (@rleungx) 256* Added Void Linux install instructions (@wpbirney) 257 258# v0.4.0 259 260- New `--style` option to disable output coloring and interactive CLI features, see #24 (@stevepentland) 261- Statistical outlier detection, see #23 #18 262 263# v0.3.0 264 265## Features 266 267- In addition to 'real' (wall clock) time, Hyperfine can now also measure 'user' and 'system' time (see #5). 268- Added `--prepare` option that can be used to clear up disk caches before timing runs, for example (see #8). 269 270## Other 271 272- [Arch Linux package](https://aur.archlinux.org/packages/hyperfine) for Hyperfine (@jD91mZM2). 273- Ubuntu/Debian packages are now are available. 274 275# v0.2.0 276 277Initial public release 278