1# NSQ Changelog 2 3## Releases 4 5### 1.2.1 - 2021-08-15 6 7**Upgrading** 8 9 * #1227 - bump dependencies, officially drop `dep` support, drop Go `1.9` support 10 11Features: 12 13 * #1347 - `nsqadmin`: switch to go:embed for static assets 14 * #1355 / #1364 - arm64 builds (thanks @danbf) 15 * #1346 - `nsqd`: ability to skip ephemeral topics/channels in statsd output 16 * #1336 / #1341 / #1343 - `nsqd`: ability to configure auth endpoint path (thanks @tufanbarisyildirim) 17 * #1307 - remove `Context` to use stdlib `context` 18 * #1295 / #1296 - switch to GitHub Actions CI 19 * #1292 - `nsqd`: minimize allocations on message send (thanks @imxyb) 20 * #1289 - optimize `uniq` (thanks @donutloop) 21 * #1230 / #1232 - `nsqd`: ability to omit memory stats from `/stats` (thanks @creker) 22 * #1226 - `nsqd`: only update internal `RDY` count for client when it changes (thanks @andyxning) 23 * #1221 / #1363 - test against more recent versions of Go 24 * #1209 - `nsqd`: bump `go-diskqueue` (interface change) (thanks @bitpeng) 25 * #1206 - prefer idiomatic `sort.Ints` over `sort.Sort` (thanks @lelenanam) 26 * #1197 / #1362 - Dockerfile: update Alpine base image, use /data by default 27 * #1178 - `nsqd`: configurable queue scan worker pool (thanks @andyxning) 28 * #1159 - `nsqd`: don't buffer messages when `--mem-queue-size=0` (thanks @bitpeng) 29 * #1073 / #1297 - `nsqd`: support separate broadcast ports for TCP and HTTP (thanks @shyam-king) 30 31Bugs: 32 33 * #1347 - `nsqadmin`: fix graphite key for ephemeral topics/channels 34 * #765 / #1195 / #1203 / #1205 - fix build on illumos (thanks @i-sevostyanov) 35 * #1333 - fix race detector tests on non-bash shells 36 * #1330 - fix `log_level` support in configuration file (thanks @edoger) 37 * #1319 / #1331 / #1361 - `nsqd`: handle SIGTERM 38 * #1287 - `nsqadmin`: fix `--proxy-graphite` support (thanks @fanlix) 39 * #1270 / #1271 - `nsqlookupd`: fix incorrect error message for HTTP listener (thanks @TangDH03) 40 * #1264 - fix benchmark script 41 * #1251 / #1314 / #1327 - `nsqd`: fix live lock for high churn ephemeral topic/channel reconnections (thanks @slayercat) 42 * #1237 - Dockerfile: add `nsswitch.conf` to ensure go resolver uses `/etc/hosts` first 43 * #1217 / #1220 - `nsqd`: improve error message when `--data-path` does not exist (thanks @mdh67899) 44 * #1198 / #1190 / #1262 - synchronize close of all connections on Exit (thanks @benjsto) 45 * #1188 / #1189 - `nsqadmin`: fix channel delete, fix source-maps in Firefox (thanks @avtrifanov) 46 * #1186 - `nsqadmin`: fix nodes list with ipv6 addresses (thanks @andyxning) 47 48### 1.2.0 - 2019-08-26 49 50**Upgrading** 51 52 * #1055 - `nsqd`: removed support for old metadata scheme used in v0.3.8 and earlier 53 * you cannot upgrade directly from v0.3.8 to v1.2.0, you must go through v1.0.0-compat or v1.1.0 54 * #1115 - manage dependencies with go modules 55 * `dep` support still present for now, but deprecated 56 57Features: 58 59 * #1136 - `nsqd`: add `--max-channel-consumers` (default unlimited) (thanks @andyxning) 60 * #1133 - `nsqd`: add `--min-output-buffer-timeout` (default 25ms) to limit how low a timeout a consumer can request 61 * and raise default `--max-output-buffer-timeout` to 30 seconds (lower timeout, more cpu usage) 62 * #1127 - `nsqd`: add topic total message bytes to stats (thanks @andyxning) 63 * #1125 - `nsqd`: add flag to adjust default `--output-buffer-timeout` (thanks @andyxning) 64 * #1163 - `nsqd`: add random load balancing for authd requests (thanks @shenhui0509) 65 * #1119 - `nsqd`: include client TLS cert CommonName in authd requests 66 * #1147 - `nsq_to_file`: include topic/channel in most log messages 67 * #1117 - `nsq_to_file`: add `--log-level` and `--log-prefix` flags 68 * #1117/#1120/#1123 - `nsq_to_file`: big refactor, more robust file switching and syncing and error handling 69 * #1118 - `nsqd`: add param to `/stats` endpoint to allow skipping per-client stats (much faster if many clients) 70 * #1118 - `nsqadmin`, `nsq_stat`: use `include_clients` param for `/stats` for a big speedup for big clusters 71 * #1110 - `nsq_to_file`: support for separate working directory with `--work-dir` (thanks @mccutchen) 72 * #856 - `nsqadmin`: add `--base-path` flag (thanks @blinklv) 73 * #1072 - `nsq_to_http`: add `--header` flag (thanks @alwindoss) 74 * #881 - `nsqd`: add producer client tcp connections to stats (thanks @sparklxb) 75 * #1071/#1074 - `nsq_to_file`: new flag `--sync-interval` (default same as previous behavior, 30 seconds) (thanks @alpaker) 76 77Bugs: 78 79 * #1153 - `nsqd`: close connections that don't send "magic" header (thanks @JoseFeng) 80 * #1140 - `nsqd`: exit on all fatal Accept() errors - restart enables better recovery for some conditions (thanks @mdh67899) 81 * #1140 - `nsqd`, `nsqlookupd`, `nsqadmin`: refactor LogLevel, general refactor to better exit on all fatal errors 82 * #1140 - `nsqadmin`: switch to using `judwhite/go-svc` like `nsqd` and `nsqadmin` do 83 * #1134 - `nsqadmin`: fix clients count and channel total message rate (new bugs introduced in this cycle) 84 * #1132 - `nsqd`, `nsqlookupd`, `nsqadmin`: fix http error response unreliable json serialization 85 * #1116 - `nsqlookupd`: fix orphaned ephemeral topics in registration DB 86 * #1109 - `nsqd`: fix topic message mis-counting if there are backend write errors (thanks @SwanSpouse) 87 * #1099 - `nsqlookupd`: optimize `/nodes` endpoint, much better for hundreds of nsqd (thanks @andyxning) 88 * #1085 - switch `GOFLAGS` to `BLDFLAGS` in `Makefile` now that `GOFLAGS` is automatically used by go 89 * #1080 - `nsqadmin`: eslint reported fixes/cleanups 90 91### 1.1.0 - 2018-08-19 92 93**Upgrading from 1.0.0-compat**: Just a few backwards incompatible changes: 94 95 * #1056 - Removed the `nsq_pubsub` utility 96 * #873 - `nsqd` flags `--msg-timeout` and `--statsd-interval` only take duration strings 97 * plain integer no longer supported 98 * #921 - `nsqd`: http `/mpub` endpoint `binary` param interprets "0" or "false" to mean text mode 99 * previously any value meant to use binary mode instead of text mode - (thanks @andyxning) 100 101The previous release, version "1.0.0-compat", was curiously-named to indicate an almost 102(but not quite) complete transition to a 1.0 api-stable release line. Confusingly, this 103follow-up release which completes the transition comes more than a year later. Because there 104have been a fair number of changes and improvements in the past year, an additional minor 105version bump seems appropriate. 106 107Features: 108 109 * #874 - `nsqd`: add memory stats to http `/stats` response (thanks @sparklxb) 110 * #892 - `nsqd`, `nsqlookupd`, `nsqadmin`: add `--log-level` option (deprecating `--verbose`) (thanks @antihax) 111 * #898 - `nsqd`, `nsqlookupd`, `nsqadmin`: logging refactor to use log levels everywhere 112 * #914 - `nsqadmin`: `X-Forwarded-User` based "admin" permission (thanks @chen-anders) 113 * #929 - `nsqd`: add topic/channel filter to `/stats`, use in `nsqadmin` and `nsq_stat` for efficiency (thanks @andyxning) 114 * #936 - `nsq_to_file`: refactor/cleanup 115 * #945 - `nsq_to_nsq`: support multiple `--topic` flags (thanks @jlr52) 116 * #957 - `nsq_tail`: support multiple `--topic` flags (thanks @soar) 117 * #946 - `nsqd`, `nsqadmin`: update internal http client with new go `http.Transport` features (keepalives, timeouts, dualstack) 118 * affects metadata/stats requests between `nsqadmin`, `nsqd`, `nsqlookupd` 119 * #954 - manage dependencies with `dep` (replacing `gpm`) (thanks @judwhite) 120 * #957 - multi-stage docker image build (thanks @soar) 121 * #996 - `nsqd`: better memory usage when messages have different sizes (thanks @andyxning) 122 * #1019 - `nsqd`: optimize random channel selection in queueScanLoop (thanks @vearne) 123 * #1025 - `nsqd`: buffer and spread statsd udp sends (avoid big burst of udp, less chance of loss) 124 * #1038 - `nsqlookupd`: optimize for many producers (thousands) (thanks @andyxning) 125 * #1050/#1053 - `nsqd`: new topic can be unlocked faster after creation 126 * #1062 - `nsqadmin`: update JS deps 127 128Bugs: 129 130 * #753 - `nsqadmin`: fix missing channels in topic list 131 * #867 - `to_nsq`: fix divide-by-zero issue when `--rate` not specified (thanks @adamweiner) 132 * #868 - `nsqd`: clamp requeue timeout to range instead of dropping connection (thanks @tsholmes) 133 * #891 - `nsqd`: fix race when client subscribes to ephemeral topic or channel while it is being cleaned up (reported by @slayercat) 134 * #927 - `nsqd`: fix deflate level handling 135 * #934 - `nsqd`: fix channel shutdown flush race 136 * #935 - `nsq_to_file`: fix connection leaks when using `--topic-pattern` (thanks @jxskiss) 137 * #951 - mention docker images and binaries for additional platforms in README (thanks @DAXaholic) 138 * #950 - `nsqlookupd`: close connection when magic read fails (thanks @yc90s) 139 * #971 - `nsqd`: fix some races getting ChannelStats (thanks @daroot) 140 * #988 - `nsqd`: fix e2e timings config example, add range validation (thanks @protoss-player) 141 * #991 - `nsq_tail`: logging to stderr (only nsq messages to stdout) 142 * #1000 - `nsq_to_http`: fix http connect/request timeout flags (thanks @kamyanskiy) 143 * #993/#1008 - `nsqd`: fix possible lookupd-identify-error busy-loop (reported by @andyxning) 144 * #1005 - `nsqadmin`: fix typo "Delfate" in connection attrs list (thanks @arussellsaw) 145 * #1032 - `nsqd`: fix loading metadata with messages queued on un-paused topic with multiple channels (thanks @michaelyou) 146 * #1004 - `nsqlookupd`: exit with error when failed to listen on ports (thanks @stephens2424) 147 * #1068 - `nsqadmin`: fix html escaping for large_graph url inside javascript 148 * misc test suite improvements and updates (go versions, tls certs, ...) 149 150### 1.0.0-compat - 2017-03-21 151 152**Upgrading from 0.3.8**: Numerous backwards incompatible changes: 153 154 * Deprecated `nsqd` features removed: 155 * Pre-V1 HTTP endpoints / response format: 156 * `/{m,}put` (use `/{m,}pub`) 157 * `/{create,delete,empty,pause,unpause}_{topic,channel}` (use `/{topic,channel}/<operation>`) 158 * `--max-message-size` flag (use `--max-msg-size`) 159 * `V2` protocol `IDENTIFY` command `short_id` and `long_id` properties (use `client_id`, `hostname`, and `user_agent`) 160 * `/stats` HTTP response `name` property (use `client_id`) 161 * Deprecated `nsqlookupd` features removed: 162 * Pre-V1 HTTP endpoints / response format: 163 * `/{create,delete}_{topic,channel}` (use `/{topic,channel}/<operation>`) 164 * `/tombstone_topic_producer` (use `/topic/tombstone`) 165 * Deprecated `nsqadmin` features removed: 166 * `--template-dir` flag (not required, templates are compiled into binary) 167 * `--use-statsd-prefixes` flag (use `--statsd-counter-format` and `--statsd-gauge-format`) 168 * `nsq_stat` `--status-every` flag (use `--interval`) 169 * `--reader-opt` on all binaries that had this flag (use `--consumer-opt`) 170 * `nsq_to_file` `--gzip-compression` flag (use `--gzip-level`) 171 * `nsq_to_http` `--http-timeout` and `--http-timeout-ms` flags (use `--http-connect-timeout` and `--http-request-timeout`) 172 * `nsq_to_http` `--round-robin` flag (use `--mode=round-robin`) 173 * `nsq_to_http` `--max-backoff-duration` flag (use `--consumer-opt=max_backoff_duration,X`) 174 * `nsq_to_http` `--throttle-fraction` flag (use `--sample=X`) 175 * `nsq_to_nsq` `--max-backoff-duration` flag (use `--consumer-opt=max_backoff_duration,X`) 176 * `nsqd` `--worker-id` deprecated in favor of `--node-id` (to be fully removed in subsequent release) 177 178This is a compatibility release that drops a wide range of previously deprecated features (#367) 179while introducing some new deprecations (#844) that we intend to fully remove in a subsequent 1.0 180release. 181 182Of note, all of the pre-1.0 HTTP endpoints (and response formats) are gone. Any clients or tools 183that use these endpoints/response formats won't work with this release. These changes have been 184available since 0.2.29 (released in July of 2014). Clients wishing to forwards-compatibly upgrade 185can either use the new endpoints or send the following header: 186 187 Accept: application/vnd.nsq version=1.0 188 189Also, many command line flags have been removed — in almost all cases an alternative is available 190with a (hopefully) more obvious name. These changes have the same affect on config file option 191names. 192 193On Linux, this release will automatically migrate `nsq.<worker-id>.dat` named metadata files to 194`nsq.dat` in a way that allows users to seamlessly _downgrade_ from this release back to 0.3.8, if 195necessary. A subsequent release will clean up these convenience symlinks and observe only 196`nsq.dat`. See the discussion in #741 and the changes #844 for more details. 197 198Performance wise, #741 landed which significantly reduces global contention on internal message ID 199generation, providing a ~1.75x speed improvement on multi-topic benchmarks. 200 201Finally, a number of minor issues were resolved spanning contributions from 9 community members! 202Thanks! 203 204Features: 205 206 * #766 - use `alpine` base image for official Docker container (thanks @kenjones-cisco) 207 * #775 - `nsqadmin:` `/config` API (thanks @kenjones-cisco) 208 * #776 - `nsqadmin`, `nsq_stat`, `nsq_to_file`, `nsq_to_http`: HTTP client connect/request timeouts (thanks @kenjones-cisco) 209 * #777/#778/#783/#785 - improve test coverage (thanks @kenjones-cisco) 210 * #788 - `to_nsq`: add `--rate` message throttling option 211 * #367 - purge deprecated features (see above) 212 * #741 - `nsqd`: per-topic message IDs (multi-topic pub benchmarks up to ~1.75x faster) 213 * #850 - `nsqd`, `nsqlookupd`, `nsqadmin`: add `--log-prefix` option (thanks @ploxiln) 214 * #844 - `nsqd`: deprecate `--worker-id` for `--node-id` and drop ID from `nsqd.dat` file (thanks @ploxiln) 215 216Bugs: 217 218 * #787 - `nsqlookupd`: properly close TCP connection in `IOLoop` (thanks @JackDrogon) 219 * #792 - `nsqdmin`: fix root CA verification (thanks @joshuarubin) 220 * #794 - `nsq_to_file`: require `--topic` or `--topic-pattern` (thanks @judwhite) 221 * #816/#823 - `nsqadmin`: fix handling of IPv6 broadcast addresses (thanks @magnetised) 222 * #805/#832 - `nsqd`: fix requeue and deferred message accounting (thanks @sdbaiguanghe) 223 * #532/#830 - `nsqd`: switch to golang/snappy to fix snappy deadlock 224 * #826/#831/#837/#839 - `nsqd`: fix default `--broadcast-address` and error when `nsqlookupd` reqs fail (thanks @ploxiln @stephensearles) 225 * #822/#835 - `nsqd`: prevent panic in binary `/mpub` (thanks @yangyifeng01) 226 * #841 - `nsqadmin`: allow ctrl/meta+click to open a new tab 227 * #843 - `nsqd`: check for exit before requeing 228 229### 0.3.8 - 2016-05-26 230 231**Upgrading from 0.3.7**: Binaries contain no backwards incompatible changes. 232 233This release fixes a critical regression in `0.3.7` that could result in message loss when 234attempting to cleanly shutdown `nsqd` by sending it a `SIGTERM`. The expected behavior was for it 235to flush messages in internal buffers to disk before exiting. See #757 and #759 for more details. 236 237A few performance improvements landed including #743, which improves channel throughput by ~17%, 238and #740, which reduces garbage when reading messages from disk. 239 240We're now stripping debug info, reducing binary size, in the official binary downloads and Windows 241binaries are now bundled with the appropriate `.exe` extension (#726 and #751). 242 243Features: 244 245 * #743 - `nsqd`: remove channel `messagePump` 246 * #751 - strip debug info from binaries (thanks @ploxiln) 247 * #740 - `nsqd`: reduce garbage when reading from diskqueue (thanks @dieterbe) 248 249Bugs: 250 251 * #757/#759 - `nsqd`: properly handle `SIGTERM` (thanks @judwhite) 252 * #738 - updates for latest `go-options` 253 * #730 - `nsqd`: diskqueue sync count on both read/write 254 * #734 - `nsqadmin`: make `rate` column work without `--proxy-graphite` (thanks @ploxiln) 255 * #726 - add `.exe` extension to Windows binaries (thanks @ploxiln) 256 * #722 - `nsqadmin`: fix connected duration > `1hr` 257 258### 0.3.7 - 2016-02-23 259 260**Upgrading from 0.3.6**: Binaries contain no backwards incompatible changes. 261 262This release has been built with Go 1.6. 263 264Highlights include the various work done to reduce `nsqd` lock contention, significantly improving 265the impact of high load on the `/stats` endpoint, addressing issues with timeouts and failures 266in `nsqadmin` (#700, #701, #703, #709). 267 268Thanks to @judwhite, `nsqd` and `nsqlookupd` now natively support being run as a Windows service 269(#718). We're also now publishing official Windows releases. 270 271`nsqd` will now `flock` its data directory on linux, preventing two `nsqd` from running 272simultaneously pointed at the same path (#583). 273 274On the bugfix side, the most noteworthy change is that `nsqd` will now correctly reset health state 275on a successful backend write (#671). 276 277Features: 278 279 * #700/#701/#703/#709 - `nsqd`: reduce lock contention (thanks @zachbadgett @absolute8511) 280 * #718 - `nsqd`/`nsqlookupd`: support running as a windows service (thanks @judwhite) 281 * #706 - `nsqd`: support enabling/disabling block profile via HTTP (thanks @absolute8511) 282 * #710 - `nsqd`: support `POST` `/debug/pprof/symbol` (thanks @absolute8511) 283 * #662 - `nsqadmin`: add flags for formatting statsd keys (thanks @kesutton) 284 * #583 - `nsqd`: `flock` `--data-path` on linux 285 * #663 - `nsqd`: optimize GUID generation (thanks @ploxiln) 286 287Bugs: 288 289 * #672 - `nsqd`: fix max size accounting in `diskqueue` (thanks @judwhite) 290 * #671 - `nsqd`: reset health on successful backend write (thanks @judwhite) 291 * #615 - `nsqd`: prevent OOM when reading from `nsqlookupd` peer 292 * #664/#666 - dist.sh/Makefile cleanup (thanks @ploxiln) 293 294### 0.3.6 - 2015-09-24 295 296**Upgrading from 0.3.5**: Binaries contain no backwards incompatible changes. 297 298We've adopted the [Contributor Covenant 1.2 Code of Conduct](CODE_OF_CONDUCT.md) (#593). Help us 299keep NSQ open and inclusive by reading and following this document. 300 301We closed a few longstanding issues related to `nsqadmin`, namely (#323, et al.) converting it to 302an API and single-page app (so that it is _much_ easier to develop), displaying fine-grained errors 303(#421, #657), and enabling support for `--tls-required` configurations (#396). 304 305For `nsqd`, we added support for deferred publishing aka `DPUB` (#293), which allows a producer to 306specify a duration of time to delay initial delivery of the message. We also addressed performance 307issues relating to large numbers of topics/channels (#577) by removing some per-channel goroutines 308in favor of a centralized, periodic, garbage collection approach. 309 310In order to provide more flexibility when deploying NSQ in dynamically orchestrated topologies, 311`nsqd` now supports the ability to configure `nsqlookupd` peers at runtime via HTTP (#601), 312eliminating the need to restart the daemon. 313 314As part of the large `nsqadmin` refactoring, we took the opportunity to cleanup the internals for 315_all_ of the daemon's HTTP code paths (#601, #610, #612, #641) as well as improving the test suite 316so that it doesn't leave around temporary files (#553). 317 318Features: 319 320 * #593 - add code of conduct 321 * #323/#631/#632/#642/#421/#649/#650/#651/#652/#654 - `nsqadmin`: convert to API / single-page app 322 * #653 - `nsqadmin`: expand notification context 323 * #293 - `nsqd`: add deferred pub (`DPUB`) 324 * #577 - `nsqd`: drop per-channel queue workers in favor of centralized queue GC 325 * #584 - `nsqlookupd`: improve registration DB performance (thanks @xiaost) 326 * #601 - `nsqd`: HTTP endpoints to dynamically configure `nsqlookupd` peers 327 * #608 - `nsqd`: support for filtering `/stats` to topic/channel (thanks @chrusty) 328 * #601/#610/#612/#641 - improved HTTP internal routing / log HTTP requests 329 * #628 - `nsqd`: clarify help text for `--e2e-processing-latency-percentile` 330 * #640 - switch `--{consumer,producer}-opt` to `nsq.ConfigFlag` 331 332Bugs: 333 334 * #656 - `nsqadmin`: update `statsd` prefix to `stats.counters` 335 * #421/#657 - `nsqadmin`: display upstream/partial errors 336 * #396 - `nsqdamin`/`nsqd`: support for `--tls-required` 337 * #558 - don't overwrite docker root FS 338 * #582 - `nsqd`: ignore benign EOF errors 339 * #587 - `nsqd`: GUID error handling / catch errors if GUID goes backwards (thanks @mpe) 340 * #586 - `nsqd`: fix valid range for `--worker-id` 341 * #550/#602/#617/#618/#619/#620/#622 - `nsqd`: fix benchmarks (thanks @Dieterbe) 342 * #553 - cleanup test dirs 343 * #600 - `nsqd`: enforce diskqueue min/max message size (thanks @twmb) 344 345### 0.3.5 - 2015-04-26 346 347**Upgrading from 0.3.3**: Binaries contain no backwards incompatible changes. 348 349This is another quick bug fix release to address the broken `nsqadmin` binary in the distribution 350(see #578). 351 352### 0.3.4 - 2015-04-26 353 354**WARNING**: please upgrade to `v0.3.5` to address the broken `nsqadmin` binary. 355 356**Upgrading from 0.3.3**: Binaries contain no backwards incompatible changes. 357 358This is a quick bug fix release to fix the outdated `go-nsq` dependency in `v0.3.3` 359for the bundled utilities (see 6e8504e). 360 361### 0.3.3 - 2015-04-26 362 363**WARNING**: please upgrade to `v0.3.5` to address the outdated `go-nsq` dependency for the 364bundled utilities and the broken `nsqadmin` binary. 365 366**Upgrading from 0.3.2**: Binaries contain no backwards incompatible changes. 367 368This release is primarily a bug fix release after cleaning up and reorganizing the codebase. 369`nsqadmin` is now importable, which paves the way for completing #323. The bundled utilities 370received a few feature additions and bug fixes (mostly from bug fixes on the `go-nsq` side). 371 372Features: 373 374 * #569 - `nsqadmin`: re-org into importable package 375 * #562 - `nsq_to_{nsq,http}`: add `epsilon-greedy` mode (thanks @twmb) 376 * #547 - `nsqd`: adds `start_time` to `/stats` (thanks @ShawnSpooner) 377 * #544 - `nsq_to_http`: accept any `200` response as success (thanks @mikedewar) 378 * #548 - `nsq_to_http`: read entire request body (thanks @imgix) 379 * #552/#554/#555/#556/#561 - code cleanup and `/internal` package re-org (thanks @cespare) 380 381Bugs: 382 383 * #573 - `nsqd`: don't persist metadata upon startup (thanks @xiaost) 384 * #560 - `nsqd`: do not print `EOF` error when client closes cleanly (thanks @twmb) 385 * #557 - `nsqd`: fix `--tls-required=tcp-https` with `--tls-client-auth-policy` (thanks @twmb) 386 * #545 - enable shell expansion in official Docker image (thanks @paddyforan) 387 388NOTE: the bundled utilities are built against [`go-nsq` `v1.0.4`][go-nsq_104] and include all of 389those features/fixes. 390 391[go-nsq_104]: https://github.com/nsqio/go-nsq/releases/tag/v1.0.4 392 393### 0.3.2 - 2015-02-08 394 395**Upgrading from 0.3.1**: Binaries contain no backwards incompatible changes however as of this 396release we've updated our official Docker images. 397 398We now provide a single Docker image [`nsqio/nsq`](https://registry.hub.docker.com/r/nsqio/nsq/) 399that includes *all* of the NSQ binaries. We did this for several reasons, primarily because the 400tagged versions in the previous incarnation were broken (and did not actually pin to a version!). 401The new image is an order of magnitude smaller, weighing in around 70mb. 402 403In addition, the impetus for this quick release is to address a slew of reconnect related bug fixes 404in the utility apps (`nsq_to_nsq`, `nsq_to_file`, etc.), for details see the [`go-nsq` `v1.0.3` 405release notes](https://github.com/nsqio/go-nsq/releases/tag/v1.0.3). 406 407Features: 408 409 * #534/#539/#540 - single Dockerfile approach (thanks @paddyforan) 410 411Bugs: 412 413 * #529 - nsqadmin: fix more `#ephemeral` topic deletion issues 414 * #530 - nsqd: fix the provided sample config file (thanks @jnewmano) 415 * #538 - nsqd: fix orphaned ephemeral channels (thanks @adamsathailo) 416 417### 0.3.1 - 2015-01-21 418 419**Upgrading from 0.3.0**: No backwards incompatible changes. 420 421This release contains minor bug fixes and feature additions. 422 423There are a number of functionality improvements to the `nsq_stat` and `nsq_to_file` helper 424applications (and general support for `#ephemeral` topics, broken in `0.2.30`). 425 426Additionally, the TLS options continue to improve with support for setting `--tls-min-version` and 427a work-around for a bug relating to `TLS_FALLBACK_SCSV` ([to be fixed in Go 4281.5](https://go-review.googlesource.com/#/c/1776/)). 429 430Features: 431 432 * #527 - nsq_stat: deprecate `--status-every` in favor of `--interval` 433 * #524 - nsq_stat: add `--count` option (thanks @nordicdyno) 434 * #518 - nsqd: set defaults for `--tls-min-version` and set TLS max version to 1.2 435 * #475/#513/#518 - nsqd: `--tls-required` can be disabled for HTTP / add `--tls-min-version` 436 (thanks @twmb) 437 * #496 - nsq_to_file: add `<PID>` to filename and rotation by size/interval (thanks @xiaost) 438 * #507 - nsq_stat: add rates (thanks @xiaost) 439 * #505 - nsqd: speed up failure path of `BytesToBase10` (thanks @iand) 440 441Bugs: 442 443 * #522 - nsqadmin: fix `#ephemeral` topic deletion issues 444 * #509 - nsqd: fix `diskqueue` atomic rename on Windows (thanks @allgeek) 445 * #479 - nsqd: return `output_buffer_*` resolved settings in `IDENTIFY` response (thanks @tj) 446 447### 0.3.0 - 2014-11-18 448 449**Upgrading from 0.2.31**: No backwards incompatible changes. 450 451This release includes a slew of bug fixes and few key feature additions. 452 453The biggest functional change is that `nsqd` no longer decrements its `RDY` count for clients. This 454means that client libraries no longer have to periodically re-send `RDY`. For some context, `nsqd` 455already provided back-pressure due to the fact that a client must respond to messages before 456receiving new ones. The decremented `RDY` count only made the implementation of the server and 457client more complex without additional benefit. Now the `RDY` command can be treated as an "on/off" 458switch. For details see #404 and the associated changes in nsqio/go-nsq#83 and nsqio/pynsq#98. 459 460The second biggest change (and oft-requested feature!) is `#ephemeral` topics. Their behavior 461mirrors that of channels. This feature is incredibly useful for situations where you're using 462topics to "route" messages to consumers (like RPC) or when a backlog of messages is undesirable. 463 464There are now scripts in the `bench` directory that automate the process of running a distributed 465benchmark. This is a work-in-progress, but it already provides a closer-to-production setup and 466therefore more accurate results. There's much work to do here! 467 468A whole bunch of bugs were fixed - notably all were 3rd-party contributions! Thanks! 469 470 * #305 - `#ephemeral` topics 471 * #404/#459 - don't decr `RDY` / send `RDY` before `FIN`/`REQ` 472 * #472 - improve `nsqd` `diskqueue` sync strategies 473 * #488 - ability to filter topics by regex in `nsq_to_file` (thanks @lxfontes) 474 * #438 - distributed pub-sub benchmark scripts 475 * #448 - better `nsqd` `IOLoop` logging (thanks @rexposadas) 476 * #458 - switch to [gpm](https://github.com/pote/gpm) for builds 477 478Bugs: 479 480 * #493 - ensure all `nsqd` `Notify()` goroutines have exited prior to shutdown (thanks @allgeek) 481 * #492 - ensure `diskqueue` syncs at end of benchmarks (thanks @Dieterbe) 482 * #490 - de-flake `TestPauseMetadata` (thanks @allgeek) 483 * #486 - require ports to be specified for daemons (thanks @jnewmano) 484 * #482 - better bash in `dist.sh` (thanks @losinggeneration) 485 * #480 - fix panic when `nsqadmin` checks stats for missing topic (thanks @jnewmano) 486 * #469 - fix panic when misbehaving client sends corrupt command (thanks @prio) 487 * #461 - fix panic when `nsqd` decodes corrupt message data (thanks @twmb) 488 * #454/#455 - fix 32-bit atomic ops in `nsq_to_nsq`/`nsq_to_http` (thanks @leshik) 489 * #451 - fix `go get` compatibility (thanks @adams-sarah) 490 491### 0.2.31 - 2014-08-26 492 493**Upgrading from 0.2.30**: No backwards incompatible changes. 494 495This release includes a few key changes. First, we improved feedback and back-pressure when `nsqd` 496writes to disk. Previously this was asynchronous and would result in clients not knowing that their 497`PUB` had failed. Interestingly, this refactoring improved performance of `PUB` by 41%, by removing 498the topic's goroutine responsible for message routing in favor of `N:N` Go channel communication. 499For details see #437. 500 501@paddyforan contributed official Dockerfiles that are now built automatically via Docker Hub. 502Please begin to use (and improve these) as the various older images we had been maintaining will be 503deprecated. 504 505The utility apps deprecated the `--reader-opt` flag in favor of `--consumer-opt` and `nsq_to_nsq` 506and `to_nsq` received a `--producer-opt` flag, for configuring details of the connection publishing 507to `nsqd`. Additionally, it is now possible to configure client side TLS certificates via 508`tls_cert` and `tls_key` opts. 509 510As usual, we fixed a few minor bugs, see below for details. 511 512New Features / Enhancements: 513 514 * #422/#437 - `nsqd`: diskqueue error feedback/backpressure (thanks @boyand) 515 * #412 - official Dockerfiles for `nsqd`, `nsqlookupd`, `nsqadmin` (thanks @paddyforan) 516 * #442 - utilities: add `--consumer-opt` alias for `--reader-opt` and 517 add `--producer-opt` to `nsq_to_nsq` (also support configuration 518 of `tls_cert` and `tls_key`) 519 * #448 - `nsqd`: improve IOLoop error messages (thanks @rexposadas) 520 521Bugs: 522 523 * #440 - `nsqd`: fixed statsd GC stats reporting (thanks @jphines) 524 * #434/#435 - refactored/stabilized tests and logging 525 * #429 - `nsqd`: improve handling/documentation of `--worker-id` (thanks @bschwartz) 526 * #428 - `nsqd`: `IDENTIFY` should respond with materialized `msg_timeout` (thanks @visionmedia) 527 528### 0.2.30 - 2014-07-28 529 530**Upgrading from 0.2.29**: No backwards incompatible changes. 531 532**IMPORTANT**: this is a quick bug-fix release to address a panic in `nsq_to_nsq` and 533`nsq_to_http`, see #425. 534 535New Features / Enhancements: 536 537 * #417 - `nsqadmin`/`nsqd`: expose TLS connection state 538 * #425 - `nsq_to_nsq`/`nsq_to_file`: display per-destination-address timings 539 540Bugs: 541 542 * #425 - `nsq_to_nsq`/`nsq_to_file`: fix shared mutable state panic 543 544### 0.2.29 - 2014-07-25 545 546**Upgrading from 0.2.28**: No backwards incompatible changes. 547 548This release includes a slew of new features and bug fixes, with contributions from 8 549members of the community, thanks! 550 551The most important new feature is authentication (the `AUTH` command for `nsqd`), added in #356. 552When `nsqd` is configured with an `--auth-http-address` it will require clients to send the `AUTH` 553command. The `AUTH` command body is opaque to `nsqd`, it simply passes it along to the configured 554auth daemon which responds with well formed JSON, indicating which topics/channels and properties 555on those entities are accessible to that client (rejecting the client if it accesses anything 556prohibited). For more details, see [the spec](https://nsq.io/clients/tcp_protocol_spec.html) or [the 557`nsqd` guide](https://nsq.io/components/nsqd.html#auth). 558 559Additionally, we've improved performance in a few areas. First, we refactored in-flight handling in 560`nsqd` to reduce garbage creation and improve baseline performance 6%. End-to-end processing 561latency calculations are also significantly faster, thanks to improvements in the 562[`perks`](https://github.com/bmizerany/perks/pulls/7) package. 563 564HTTP response formats have been improved (removing the redundant response wrapper) and cleaning up 565some of the endpoint namespaces. This change is backwards compatible. Clients wishing to move 566towards the new response format can either use the new endpoint names or send the following header: 567 568 Accept: application/vnd.nsq version=1.0 569 570Other changes including officially bumping the character limit for topic and channel names to 64 571(thanks @svmehta), making the `REQ` timeout limit configurable in `nsqd` (thanks @AlphaB), and 572compiling static asset dependencies into `nsqadmin` to simplify deployment (thanks @crossjam). 573 574Finally, `to_nsq` was added to the suite of bundled apps. It takes a stdin stream and publishes to 575`nsqd`, an extremely flexible solution (thanks @matryer)! 576 577As for bugs, they're mostly minor, see the pull requests referenced in the section below for 578details. 579 580New Features / Enhancements: 581 582 * #304 - apps: added `to_nsq` for piping stdin to NSQ (thanks @matryer) 583 * #406 - `nsqadmin`: embed external static asset dependencies (thanks @crossjam) 584 * #389 - apps: report app name and version via `user_agent` 585 * #378/#390 - `nsqd`: improve in-flight message handling (6% faster, GC reduction) 586 * #356/#370/#386 - `nsqd`: introduce `AUTH` 587 * #358 - increase topic/channel name max length to 64 (thanks @svmehta) 588 * #357 - remove internal `go-nsq` dependencies (GC reduction) 589 * #330/#366 - version HTTP endpoints, simplify response format 590 * #352 - `nsqd`: make `REQ` timeout limit configurable (thanks @AlphaB) 591 * #340 - `nsqd`: bump perks dependency (E2E performance improvement, see 25086e4) 592 593Bugs: 594 595 * #384 - `nsqd`: fix statsd GC time reporting 596 * #407 - `nsqd`: fix double `TOUCH` and use of client's configured msg timeout 597 * #392 - `nsqadmin`: fix HTTPS warning (thanks @juliangruber) 598 * #383 - `nsqlookupd`: fix race on last update timestamp 599 * #385 - `nsqd`: properly handle empty `FIN` 600 * #365 - `nsqd`: fix `IDENTIFY` `msg_timeout` response (thanks @visionmedia) 601 * #345 - `nsq_to_file`: set proper permissions on new directories (thanks @bschwartz) 602 * #338 - `nsqd`: fix windows diskqueue filenames (thanks @politician) 603 604### 0.2.28 - 2014-04-28 605 606**Upgrading from 0.2.27**: No backwards incompatible changes. We've deprecated the `short_id` 607and `long_id` options in the `IDENTIFY` command in favor of `client_id` and `hostname`, which 608more accurately reflect the data typically used. 609 610This release includes a few important new features, in particular enhanced `nsqd` 611TLS support thanks to a big contribution by @chrisroberts. 612 613You can now *require* that clients negotiate TLS with `--tls-required` and you can configure a 614client certificate policy via `--tls-client-auth-policy` (`require` or `require-verify`): 615 616 * `require` - the client must offer a certificate, otherwise rejected 617 * `require-verify` - the client must offer a valid certificate according to the default CA or 618 the chain specified by `--tls-root-ca-file`, otherwise rejected 619 620This can be used as a form of client authentication. 621 622Additionally, `nsqd` is now structured such that it is importable in other Go applications 623via `github.com/nsqio/nsq/nsqd`, thanks to @kzvezdarov. 624 625Finally, thanks to @paddyforan, `nsq_to_file` can now archive *multiple* topics or 626optionally archive *all* discovered topics (by specifying no `--topic` params 627and using `--lookupd-http-address`). 628 629New Features / Enhancements: 630 631 * #334 - `nsq_to_file` can archive many topics (thanks @paddyforan) 632 * #327 - add `nsqd` TLS client certificate verification policy, ability 633 to require TLS, and HTTPS support (thanks @chrisroberts) 634 * #325 - make `nsqd` importable (`github.com/nsqio/nsq/nsqd`) (thanks @kzvezdarov) 635 * #321 - improve `IDENTIFY` options (replace `short_id` and `long_id` with 636 `client_id` and `hostname`) 637 * #319 - allow path separator in `nsq_to_file` filenames (thanks @jsocol) 638 * #324 - display memory depth and total depth in `nsq_stat` 639 640Bug Fixes: 641 642 * nsqio/go-nsq#19 and nsqio/go-nsq#29 - fix deadlocks on `nsq.Reader` connection close/exit, this 643 impacts the utilities packaged with the NSQ binary 644 distribution such as `nsq_to_file`, `nsq_to_http`, 645 `nsq_to_nsq` and `nsq_tail`. 646 * #329 - use heartbeat interval for write deadline 647 * #321/#326 - improve benchmarking tests 648 * #315/#318 - fix test data races / flakiness 649 650### 0.2.27 - 2014-02-17 651 652**Upgrading from 0.2.26**: No backwards incompatible changes. We deprecated `--max-message-size` 653in favor of `--max-msg-size` for consistency with the rest of the flag names. 654 655IMPORTANT: this is another quick bug-fix release to address an issue in `nsqadmin` where templates 656were incompatible with older versions of Go (pre-1.2). 657 658 * #306 - fix `nsqadmin` template compatibility (and formatting) 659 * #310 - fix `nsqadmin` behavior when E2E stats are disabled 660 * #309 - fix `nsqadmin` `INVALID_ERROR` on node page tombstone link 661 * #311/#312 - fix `nsqd` client metadata race condition and test flakiness 662 * #314 - fix `nsqd` test races (run w/ `-race` and `GOMAXPROCS=4`) deprecate `--max-message-size` 663 664### 0.2.26 - 2014-02-06 665 666**Upgrading from 0.2.25**: No backwards incompatible changes. 667 668IMPORTANT: this is a quick bug-fix release to address a regression identified in `0.2.25` where 669`statsd` prefixes were broken when using the default (or any) prefix that contained a `%s` for 670automatic host replacement. 671 672 * #303 - fix `nsqd` `--statsd-prefix` when using `%s` host replacement 673 674### 0.2.25 - 2014-02-05 675 676**Upgrading from 0.2.24**: No backwards incompatible changes. 677 678This release adds several commonly requested features. 679 680First, thanks to [@elubow](https://twitter.com/elubow) you can now configure your clients to sample 681the stream they're subscribed to. To read more about the details of the implementation see #286 and 682the original discussion in #223. Eric also contributed an improvement to `nsq_tail` to add 683the ability to tail the last `N` messages and exit. 684 685We added config file support ([TOML](https://github.com/mojombo/toml/blob/master/README.md)) for 686`nsqd`, `nsqlookupd`, and `nsqadmin` - providing even more deployment flexibility. Example configs 687are in the `contrib` directory. Command line arguments override the equivalent option in the config 688file. 689 690We added the ability to pause a *topic* (it is already possible to pause individual *channels*). 691This functionality stops all message flow from topic to channel for *all channels* of a topic, 692queueing at the topic level. This enables all kinds of interesting possibilities like atomic 693channel renames and trivial infrastructure wide operations. 694 695Finally, we now compile the static assets used by `nsqadmin` into the binary, simplifying 696deployment. This means that `--template-dir` is now deprecated and will be removed in a future 697release and you can remove the templates you previously deployed and maintained. 698 699New Features / Enhancements: 700 701 * #286 - add client `IDENTIFY` option to sample a % of messages 702 * #279 - add TOML config file support to `nsqd`, `nsqlookupd`, and `nsqadmin` 703 * #263 - add ability to pause a topic 704 * #291 - compile templates into `nsqadmin` binary 705 * #285/#288 - `nsq_tail` support for `-n #` to get recent # messages 706 * #287/#294 - display client `IDENTIFY` attributes in `nsqadmin` (sample rate, TLS, compression) 707 * #189/#296 - add client user agent to `nsqadmin`` 708 * #297 - add `nsq_to_nsq` JSON message filtering options 709 710### 0.2.24 - 2013-12-07 711 712**Upgrading from 0.2.23**: No backwards incompatible changes. However, as you'll see below, quite a 713few command line flags to the utility apps (`nsq_to_http`, `nsq_to_file`, `nsq_to_http`) were 714deprecated and will be removed in the next release. Please use this release to transition over to 715the new ones. 716 717NOTE: we are now publishing additional binaries built against go1.2 718 719The most prominent addition is the tracking of end-to-end message processing percentiles. This 720measures the amount of time it's taking from `PUB` to `FIN` per topic/channel. The percentiles are 721configurable and, because there is *some* overhead in collecting this data, it can be turned off 722entirely. Please see [the section in the docs](https://nsq.io/components/nsqd.html) for 723implementation details. 724 725Additionally, the utility apps received comprehensive support for all configurable reader options 726(including compression, which was previously missing). This necessitated a bit of command line flag 727cleanup, as follows: 728 729#### nsq_to_file 730 731 * deprecated `--gzip-compression` in favor of `--gzip-level` 732 * deprecated `--verbose` in favor of `--reader-opt=verbose` 733 734#### nsq_to_http 735 736 * deprecated `--throttle-fraction` in favor of `--sample` 737 * deprecated `--http-timeout-ms` in favor of `--http-timeout` (which is a 738 *duration* flag) 739 * deprecated `--verbose` in favor of `--reader-opt=verbose` 740 * deprecated `--max-backoff-duration` in favor of 741 `--reader-opt=max_backoff_duration=X` 742 743#### nsq_to_nsq 744 745 * deprecated `--verbose` in favor of `--reader-opt=verbose` 746 * deprecated `--max-backoff-duration` in favor of 747 `--reader-opt=max_backoff_duration=X` 748 749New Features / Enhancements: 750 751 * #280 - add end-to-end message processing latency metrics 752 * #267 - comprehensive reader command line flags for utilities 753 754### 0.2.23 - 2013-10-21 755 756**Upgrading from 0.2.22**: No backwards incompatible changes. 757 758We now use [godep](https://github.com/kr/godep) in order to achieve reproducible builds with pinned 759dependencies. If you're on go1.1+ you can now just use `godep get github.com/nsqio/nsq/...`. 760 761This release includes `nsqd` protocol compression feature negotiation. 762[Snappy](https://code.google.com/p/snappy/) and [Deflate](http://en.wikipedia.org/wiki/DEFLATE) are 763supported, clients can choose their preferred format. 764 765`--statsd-prefix` can now be used to modify the prefix for the `statsd` keys generated by `nsqd`. 766This is useful if you want to add datacenter prefixes or remove the default host prefix. 767 768Finally, this release includes a "bug" fix that reduces CPU usage for `nsqd` with many clients by 769choosing a more reasonable default for a timer used in client output buffering. For more details 770see #236. 771 772New Features / Enhancements: 773 774 * #266 - use godep for reproducible builds 775 * #229 - compression (Snappy/Deflate) feature negotiation 776 * #241 - binary support for HTTP /mput 777 * #269 - add --statsd-prefix flag 778 779Bug Fixes: 780 781 * #278 - fix nsqd race for client subscription cleanup (thanks @simplereach) 782 * #277 - fix nsqadmin counter page 783 * #275 - stop accessing simplejson internals 784 * #274 - nsqd channel pause state lost during unclean restart (thanks @hailocab) 785 * #236 - reduce "idle" CPU usage by 90% with large # of clients 786 787### 0.2.22 - 2013-08-26 788 789**Upgrading from 0.2.21**: message timestamps are now officially nanoseconds. The protocol docs 790always stated this however `nsqd` was actually sending seconds. This may cause some compatibility 791issues for client libraries/clients that were taking advantage of this field. 792 793This release also introduces support for TLS feature negotiation in `nsqd`. Clients can optionally 794enable TLS by using the appropriate handshake via the `IDENTIFY` command. See #227. 795 796Significant improvements were made to the HTTP publish endpoints and in flight message handling to 797reduce GC pressure and eliminate memory abuse vectors. See #242, #239, and #245. 798 799This release also includes a new utility `nsq_to_nsq` for performant, low-latency, copying of an NSQ 800topic over the TCP protocol. 801 802Finally, a whole suite of debug HTTP endpoints were added (and consolidated) under the 803`/debug/pprof` namespace. See #238, #248, and #252. As a result `nsqd` now supports *direct* 804profiling via Go's `pprof` tool, ie: 805 806 $ go tool pprof --web http://ip.address:4151/debug/pprof/heap 807 808New Features / Enhancements: 809 810 * #227 - TLS feature negotiation 811 * #238/#248/#252 - support for more HTTP debug endpoints 812 * #256 - `nsqadmin` single node view (with GC/mem graphs) 813 * #255 - `nsq_to_nsq` utility for copying a topic over TCP 814 * #230 - `nsq_to_http` takes `--content-type` flag (thanks @michaelhood) 815 * #228 - `nsqadmin` displays tombstoned topics in the `/nodes` list 816 * #242/#239/#245 - reduced GC pressure for inflight and `/mput` 817 818Bug Fixes: 819 820 * #260 - `tombstone_topic_producer` action in `nsqadmin` missing node info 821 * #244 - fix 64bit atomic alignment issues on 32bit platforms 822 * #251 - respect configured limits for HTTP publishing 823 * #247 - publish methods should not allow 0 length messages 824 * #231/#259 - persist `nsqd` metadata on topic/channel changes 825 * #237 - fix potential memory leaks with retained channel references 826 * #232 - message timestamps are now nano 827 * #228 - `nsqlookupd`/`nsqadmin` would display inactive nodes in `/nodes` list 828 * #216 - fix edge cases in `nsq_to_file` that caused empty files 829 830### 0.2.21 - 2013-06-07 831 832**Upgrading from 0.2.20**: there are no backward incompatible changes in this release. 833 834This release introduces a significant new client feature as well as a slew of consistency and 835recovery improvements to diskqueue. 836 837First, we expanded the feature negotiation options for clients. There are many cases where you want 838different output buffering semantics from `nsqd` to your client. You can now control both 839output buffer size and the output buffer timeout via new fields in the `IDENTIFY` command. You can 840even disable output buffering if low latency is a priority. 841 842You can now specify a duration between fsyncs via `--sync-timeout`. This is a far better way to 843manage when the process fsyncs messages to disk (vs the existing `--sync-every` which is based on # 844of messages). `--sync-every` is now considered a deprecated option and will be removed in a future 845release. 846 847Finally, `0.2.20` introduced a significant regression in #176 where a topic would not write messages 848to its channels. It is recommended that all users running `0.2.20` upgrade to this release. For 849additional information see #217. 850 851New Features / Enhancements: 852 853 * #214 - add --sync-timeout for time based fsync, improve when diskqueue syncs 854 * #196 - client configurable output buffering 855 * #190 - nsq_tail generates a random #ephemeral channel 856 857Bug Fixes: 858 859 * #218/#220 - expose --statsd-interval for nsqadmin to handle non 60s statsd intervals 860 * #217 - fix new topic channel creation regression from #176 (thanks @elubow) 861 * #212 - dont use port in nsqadmin cookies 862 * #214 - dont open diskqueue writeFile with O_APPEND 863 * #203/#211 - diskqueue depth accounting consistency 864 * #207 - failure to write a heartbeat is fatal / reduce error log noise 865 * #206 - use broadcast address for statsd prefix 866 * #205 - cleanup example utils exit 867 868### 0.2.20 - 2013-05-13 869 870**Upgrading from 0.2.19**: there are no backward incompatible changes in this release. 871 872This release adds a couple of convenient features (such as adding the ability to empty a *topic*) 873and continues our work to reduce garbage produced at runtime to relieve GC pressure in the Go 874runtime. 875 876`nsqd` now has two new flags to control the max value clients can use to set their heartbeat 877interval as well as adjust a clients maximum RDY count. This is all set/communicated via `IDENTIFY`. 878 879`nsqadmin` now displays `nsqd` -> `nsqlookupd` connections in the "nodes" view. This is useful for 880visualizing how the topology is connected as well as situations where `--broadcast-address` is being 881used incorrectly. 882 883`nsq_to_http` now has a "host pool" mode where upstream state will be adjusted based on 884successful/failed requests and for failures, upstreams will be exponentially backed off. This is an 885incredibly useful routing mode. 886 887As for bugs, we fixed an issue where "fatal" client errors were not actually being treated as fatal. 888Under certain conditions deleting a topic would not clean up all of its files on disk. There was a 889reported issue where the `--data-path` was not writable by the process and this was only discovered 890after message flow began. We added a writability check at startup to improve feedback. Finally. 891`deferred_count` was being sent as a counter value to statsd, it should be a gauge. 892 893New Features / Enhancements: 894 895 * #197 - nsqadmin nodes list improvements (show nsqd -> lookupd conns) 896 * #192 - add golang runtime version to daemon version output 897 * #183 - ability to empty a topic 898 * #176 - optimizations to reduce garbage, copying, locking 899 * #184 - add table headers to nsqadmin channel view (thanks @elubow) 900 * #174/#186 - nsq_to_http hostpool mode and backoff control 901 * #173/#187 - nsq_stat utility for command line introspection 902 * #175 - add nsqd --max-rdy-count configuration option 903 * #178 - add nsqd --max-heartbeat-interval configuration option 904 905Bug Fixes: 906 907 * #198 - fix fatal errors not actually being fatal 908 * #195 - fix delete topic does not delete all diskqueue files 909 * #193 - fix data race in channel requeue 910 * #185 - ensure that --data-path is writable on startup 911 * #182 - fix topic deletion ordering to prevent race conditions with lookupd/diskqueue 912 * #179 - deferred_count as gauge for statsd 913 * #173/#188/#191 - fix nsqadmin counter template error; fix nsqadmin displaying negative rates 914 915### 0.2.19 - 2013-04-11 916 917**Upgrading from 0.2.18**: there are no backward incompatible changes in this release. 918 919This release is a small release that introduces one major client side feature and resolves one 920critical bug. 921 922`nsqd` clients can now configure their own heartbeat interval. This is important because as of 923`0.2.18` *all* clients (including producers) received heartbeats by default. In certain cases 924receiving a heartbeat complicated "simple" clients that just wanted to produce messages and not 925handle asynchronous responses. This gives flexibility for the client to decide how it would like 926behave. 927 928A critical bug was discovered where emptying a channel would leave client in-flight state 929inconsistent (it would not zero) which limited deliverability of messages to those clients. 930 931New Features / Enhancements: 932 933 * #167 - 'go get' compatibility 934 * #158 - allow nsqd clients to configure (or disable) heartbeats 935 936Bug Fixes: 937 938 * #171 - fix race conditions identified testing against go 1.1 (scheduler improvements) 939 * #160 - empty channel left in-flight count inconsistent (thanks @dmarkham) 940 941### 0.2.18 - 2013-02-28 942 943**Upgrading from 0.2.17**: all V2 clients of nsqd now receive heartbeats (previously only clients 944that subscribed would receive heartbeats, excluding TCP *producers*). 945 946**Upgrading from 0.2.16**: follow the notes in the 0.2.17 changelog for upgrading from 0.2.16. 947 948Beyond the important note above regarding heartbeats this release includes `nsq_tail`, an extremely 949useful utility application that can be used to introspect a topic on the command line. If statsd is 950enabled (and graphite in `nsqadmin`) we added the ability to retrieve rates for display in 951`nsqadmin`. 952 953We resolved a few critical issues with data consistency in `nsqlookupd` when channels and topics are 954deleted. First, deleting a topic would cause that producer to disappear from `nsqlookupd` for all 955topics. Second, deleting a channel would cause that producer to disappear from the topic list in 956`nsqlookupd`. 957 958New Features / Enhancements: 959 960 * #131 - all V2 nsqd clients get heartbeats 961 * #154 - nsq_tail example reader 962 * #143 - display message rates in nsqadmin 963 964Bug Fixes: 965 966 * #148 - store tombstone data per registration in nsqlookupd 967 * #153 - fix large graph formulas in nsqadmin 968 * #150/#151 - fix topics disappearing from nsqlookupd when channels are deleted 969 970### 0.2.17 - 2013-02-07 971 972**Upgrading from 0.2.16**: IDENTIFY and SUB now return success responses (they previously only 973responded to errors). The official Go and Python libraries are forwards/backwards compatible with 974this change however 3rd party client libraries may not be. 975 976**Upgrading from 0.2.15**: in #132 deprecations in SUB were removed as well as support for the old, 977line oriented, `nsqd` metadata file format. For these reasons you should upgrade to `0.2.16` first. 978 979New Features / Enhancements: 980 981 * #119 - add TOUCH command to nsqd 982 * #142 - add --broadcast-address flag to nsqd/nsqadmin (thanks @dustismo) 983 * #135 - atomic MPUB 984 * #133 - improved protocol fatal error handling and responses; IDENTIFY/SUB success responses 985 * #118 - switch nsqadmin actions to POST and require confirmation 986 * #117/#147 - nsqadmin action POST notifications 987 * #122 - configurable msg size limits 988 * #132 - deprecate identify in SUB and old nsqd metadata file format 989 990Bug Fixes: 991 992 * #144 - empty channel should clear inflight/deferred messages 993 * #140 - fix MPUB protocol documentation 994 * #139 - fix nsqadmin handling of legacy statsd prefixes for graphs 995 * #138/#145 - fix nsqadmin action redirect handling 996 * #134 - nsqd to nsqlookupd registration fixes 997 * #129 - nsq_to_file gzip file versioning 998 * #106 - nsqlookupd topic producer tombstones 999 * #100 - sane handling of diskqueue read errors 1000 * #123/#125 - fix notify related exit deadlock 1001 1002### 0.2.16 - 2013-01-07 1003 1004**Upgrading from 0.2.15**: there are no backward incompatible changes in this release. 1005 1006However, this release introduces the `IDENTIFY` command (which supersedes sending 1007metadata along with `SUB`) for clients of `nsqd`. The old functionality will be 1008removed in a future release. 1009 1010 * #114 persist paused channels through restart 1011 * #121 fix typo preventing compile of bench_reader (thanks @datastream) 1012 * #120 fix nsqd crash when empty command is sent (thanks @michaelhood) 1013 * #115 nsq_to_file --filename-format --datetime-format parameter and fix 1014 * #101 fix topic/channel delete operations ordering 1015 * #98 nsqadmin fixes when not using lookupd 1016 * #90/#108 performance optimizations / IDENTIFY protocol support in nsqd. For 1017 a single consumer of small messages (< 4k) increases throughput ~400% and 1018 reduces # of allocations ~30%. 1019 * #105 strftime compatible datetime format 1020 * #103 nsq_to_http handler logging 1021 * #102 compatibility with Go tip 1022 * #99 nsq_to_file --gzip flag 1023 * #95 proxy graphite requests through nsqadmin 1024 * #93 fix nqd API response for no topics 1025 * #92 graph rendering options 1026 * #86 nsq_to_http Content-Length headers 1027 * #89 gopkg doc updates 1028 * #88 move pynsq to it's own repo 1029 * #81/#87 reader improvements / introduced MPUB. Fix bug for mem-queue-size < 10 1030 * #76 statsd/graphite support 1031 * #75 administrative ability to create topics and channels 1032 1033### 0.2.15 - 2012-10-25 1034 1035 * #84 fix lookupd hanging on to ephemeral channels w/ no producers 1036 * #82 add /counter page to nsqadmin 1037 * #80 message size benchmark 1038 * #78 send Content-Length for nsq_to_http requests 1039 * #57/#83 documentation updates 1040 1041### 0.2.14 - 2012-10-19 1042 1043 * #77 ability to pause a channel (includes bugfix for message pump/diskqueue) 1044 * #74 propagate all topic changes to lookupd 1045 * #65 create binary releases 1046 1047### 0.2.13 - 2012-10-15 1048 1049 * #70 deadlined nsq_to_http outbound requests 1050 * #69/#72 improved nsq_to_file sync strategy 1051 * #58 diskqueue metadata bug and refactoring 1052 1053### 0.2.12 - 2012-10-10 1054 1055 * #63 consolidated protocol V1 into V2 and fixed PUB bug 1056 * #61 added a makefile for simpler building 1057 * #55 allow topic/channel names with `.` 1058 * combined versions for all binaries 1059 1060### 0.2.7 - 0.2.11 1061 1062 * Initial public release. 1063 1064## go-nsq Client Library 1065 1066 * #264 moved **go-nsq** to its own [repository](https://github.com/nsqio/go-nsq) 1067 1068## pynsq Python Client Library 1069 1070 * #88 moved **pynsq** to its own [repository](https://github.com/nsqio/pynsq) 1071