1# librdkafka release process 2 3This guide outlines the steps needed to release a new version of librdkafka 4and publish packages to channels (NuGet, Homebrew, etc,..). 5 6Releases are done in two phases: 7 * release-candidate(s) - RC1 will be the first release candidate, and any 8 changes to the repository will require a new RC. 9 * final release - the final release is based directly on the last RC tag 10 followed by a single version-bump commit (see below). 11 12Release tag and version format: 13 * release-candidate: vA.B.C-RCn 14 * final release: vA.B.C 15 16 17 18## Write release notes 19 20Go to https://github.com/edenhill/librdkafka/releases and create a new 21release (save as draft), outlining the following sections based on the 22changes since the last release: 23 * What type of release (maintenance or feature release) 24 * A short intro to the release, describing the type of release: maintenance 25 or feature release, as well as fix or feature high-lights. 26 * A section of New features, if any. 27 * A section of Enhancements, if any. 28 * A section of Fixes, if any. 29 30Hint: Use ´git log --oneline vLastReleaseTag..´ to get a list of commits since 31 the last release, filter and sort this list into the above categories, 32 making sure the end result is meaningful to the end-user. 33 Make sure to credit community contributors for their work. 34 35Save this page as Draft until the final tag is created. 36 37The github release asset/artifact checksums will be added later when the 38final tag is pushed. 39 40 41## Update protocol requests and error codes 42 43Check out the latest version of Apache Kafka (not trunk, needs to be a released 44version since protocol may change on trunk). 45 46### Protocol request types 47 48Generate protocol request type codes with: 49 50 $ src/generate_proto.sh ~/src/your-kafka-dir 51 52Cut'n'paste the new defines and strings to `rdkafka_protocol.h` and 53`rdkafka_proto.h`. 54 55### Error codes 56 57Error codes must currently be parsed manually, open 58`clients/src/main/java/org/apache/kafka/common/protocol/Errors.java` 59in the Kafka source directory and update the `rd_kafka_resp_err_t` and 60`RdKafka::ErrorCode` enums in `rdkafka.h` and `rdkafkacpp.h` 61respectively. 62Add the error strings to `rdkafka.c`. 63The Kafka error strings are sometimes a bit too verbose for our taste, 64so feel free to rewrite them (usually removing a couple of 'the's). 65 66**NOTE**: Only add **new** error codes, do not alter existing ones since that 67 will be a breaking API change. 68 69 70## Run regression tests 71 72**Build tests:** 73 74 $ cd tests 75 $ make -j build 76 77**Run the full regression test suite:** (requires Linux and the trivup python package) 78 79 $ make full 80 81 82If all tests pass, carry on, otherwise identify and fix bug and start over. 83 84 85## Pre-release code tasks 86 87**Switch to the release branch which is of the format `A.B.C.x` or `A.B.x`.** 88 89 $ git checkout -b 0.11.1.x 90 91 92**Update in-code versions.** 93 94The last octet in the version hex number is the pre-build/release-candidate 95number, where 0xAABBCCff is the final release for version 0xAABBCC. 96Release candidates start at 200, thus 0xAABBCCc9 is RC1, 0xAABBCCca is RC2, etc. 97 98Change the `RD_KAFKA_VERSION` defines in both `src/rdkafka.h` and 99`src-cpp/rdkafkacpp.h` to the version to build, such as 0x000b01c9 100for v0.11.1-RC1, or 0x000b01ff for the final v0.11.1 release. 101Update the librdkafka version in `vcpkg.json`. 102 103 # Update defines 104 $ $EDITOR src/rdkafka.h src-cpp/rdkafkacpp.h vcpkg.json 105 106 # Reconfigure and build 107 $ ./configure 108 $ make 109 110 # Check git diff for correctness 111 $ git diff 112 113 # Commit 114 $ git commit -m "Version v0.11.1-RC1" src/rdkafka.h src-cpp/rdkafkacpp.h 115 116 117**Create tag.** 118 119 $ git tag v0.11.1-RC1 # for an RC 120 # or for the final release: 121 $ git tag v0.11.1 # for the final release 122 123 124**Push branch and commit to github** 125 126 # Dry-run first to make sure things look correct 127 $ git push --dry-run origin 0.11.1.x 128 129 # Live 130 $ git push origin 0.11.1.x 131**Push tags and commit to github** 132 133 # Dry-run first to make sure things look correct. 134 $ git push --dry-run --tags origin v0.11.1-RC1 135 136 # Live 137 $ git push --tags origin v0.11.1-RC1 138 139 140## Creating packages 141 142As soon as a tag is pushed the CI systems (Travis and AppVeyor) will 143start their builds and eventually upload the packaging artifacts to S3. 144Wait until this process is finished by monitoring the two CIs: 145 146 * https://travis-ci.org/edenhill/librdkafka 147 * https://ci.appveyor.com/project/edenhill/librdkafka 148 149 150## Publish release on github 151 152Open up the release page on github that was created above. 153 154Run the following command to get checksums of the github release assets: 155 156 $ packaging/tools/gh-release-checksums.py <the-tag> 157 158It will take some time for the script to download the files, when done 159paste the output to the end of the release page. 160 161Make sure the release page looks okay, is still correct (check for new commits), 162and has the correct tag, then click Publish release. 163 164 165### Create NuGet package 166 167On a Linux host with docker installed, this will also require S3 credentials 168to be set up. 169 170 $ cd packaging/nuget 171 $ pip3 install -r requirements.txt # if necessary 172 $ ./release.py v0.11.1-RC1 173 174Test the generated librdkafka.redist.0.11.1-RC1.nupkg and 175then upload it to NuGet manually: 176 177 * https://www.nuget.org/packages/manage/upload 178 179 180### Create static bundle (for Go) 181 182 $ cd packaging/nuget 183 $ ./release.py --class StaticPackage v0.11.1-RC1 184 185Follow the Go client release instructions for updating its bundled librdkafka 186version based on the tar ball created here. 187 188 189### Homebrew recipe update 190 191The brew-update-pr.sh script automatically pushes a PR to homebrew-core 192with a patch to update the librdkafka version of the formula. 193This should only be done for final releases and not release candidates. 194 195On a MacOSX host with homebrew installed: 196 197 $ cd package/homebrew 198 # Dry-run first to see that things are okay. 199 $ ./brew-update-pr.sh v0.11.1 200 # If everything looks good, do the live push: 201 $ ./brew-update-pr.sh --upload v0.11.1 202 203 204### Deb and RPM packaging 205 206Debian and RPM packages are generated by Confluent packaging in a separate 207process and the resulting packages are made available on Confluent's 208APT and YUM repositories. 209 210That process is outside the scope of this document. 211 212See the Confluent docs for instructions how to access these packages: 213https://docs.confluent.io/current/installation.html 214