1# librdkafka release process
2
3This guide outlines the steps needed to release a new version of librdkafka
4and publish packages to channels (NuGet, Homebrew, etc,..).
5
6Releases are done in two phases:
7 * release-candidate(s) - RC1 will be the first release candidate, and any
8   changes to the repository will require a new RC.
9 * final release - the final release is based directly on the last RC tag
10   followed by a single version-bump commit (see below).
11
12Release tag and version format:
13 * release-candidate: vA.B.C-RCn
14 * final release: vA.B.C
15
16
17
18## Write release notes
19
20Go to https://github.com/edenhill/librdkafka/releases and create a new
21release (save as draft), outlining the following sections based on the
22changes since the last release:
23 * What type of release (maintenance or feature release)
24 * A short intro to the release, describing the type of release: maintenance
25   or feature release, as well as fix or feature high-lights.
26 * A section of New features, if any.
27 * A section of Enhancements, if any.
28 * A section of Fixes, if any.
29
30Hint: Use ´git log --oneline vLastReleaseTag..´ to get a list of commits since
31      the last release, filter and sort this list into the above categories,
32      making sure the end result is meaningful to the end-user.
33      Make sure to credit community contributors for their work.
34
35Save this page as Draft until the final tag is created.
36
37The github release asset/artifact checksums will be added later when the
38final tag is pushed.
39
40
41## Update protocol requests and error codes
42
43Check out the latest version of Apache Kafka (not trunk, needs to be a released
44version since protocol may change on trunk).
45
46### Protocol request types
47
48Generate protocol request type codes with:
49
50    $ src/generate_proto.sh ~/src/your-kafka-dir
51
52Cut'n'paste the new defines and strings to `rdkafka_protocol.h` and
53`rdkafka_proto.h`.
54
55### Error codes
56
57Error codes must currently be parsed manually, open
58`clients/src/main/java/org/apache/kafka/common/protocol/Errors.java`
59in the Kafka source directory and update the `rd_kafka_resp_err_t` and
60`RdKafka::ErrorCode` enums in `rdkafka.h` and `rdkafkacpp.h`
61respectively.
62Add the error strings to `rdkafka.c`.
63The Kafka error strings are sometimes a bit too verbose for our taste,
64so feel free to rewrite them (usually removing a couple of 'the's).
65
66**NOTE**: Only add **new** error codes, do not alter existing ones since that
67          will be a breaking API change.
68
69
70## Run regression tests
71
72**Build tests:**
73
74    $ cd tests
75    $ make -j build
76
77**Run the full regression test suite:** (requires Linux and the trivup python package)
78
79    $ make full
80
81
82If all tests pass, carry on, otherwise identify and fix bug and start over.
83
84
85## Pre-release code tasks
86
87**Switch to the release branch which is of the format `A.B.C.x` or `A.B.x`.**
88
89    $ git checkout -b 0.11.1.x
90
91
92**Update in-code versions.**
93
94The last octet in the version hex number is the pre-build/release-candidate
95number, where 0xAABBCCff is the final release for version 0xAABBCC.
96Release candidates start at 200, thus 0xAABBCCc9 is RC1, 0xAABBCCca is RC2, etc.
97
98Change the `RD_KAFKA_VERSION` defines in both `src/rdkafka.h` and
99`src-cpp/rdkafkacpp.h` to the version to build, such as 0x000b01c9
100for v0.11.1-RC1, or 0x000b01ff for the final v0.11.1 release.
101Update the librdkafka version in `vcpkg.json`.
102
103   # Update defines
104   $ $EDITOR src/rdkafka.h src-cpp/rdkafkacpp.h vcpkg.json
105
106   # Reconfigure and build
107   $ ./configure
108   $ make
109
110   # Check git diff for correctness
111   $ git diff
112
113   # Commit
114   $ git commit -m "Version v0.11.1-RC1" src/rdkafka.h src-cpp/rdkafkacpp.h
115
116
117**Create tag.**
118
119    $ git tag v0.11.1-RC1 # for an RC
120    # or for the final release:
121    $ git tag v0.11.1     # for the final release
122
123
124**Push branch and commit to github**
125
126    # Dry-run first to make sure things look correct
127    $ git push --dry-run origin 0.11.1.x
128
129    # Live
130    $ git push origin 0.11.1.x
131**Push tags and commit to github**
132
133    # Dry-run first to make sure things look correct.
134    $ git push --dry-run --tags origin v0.11.1-RC1
135
136    # Live
137    $ git push --tags origin v0.11.1-RC1
138
139
140## Creating packages
141
142As soon as a tag is pushed the CI systems (Travis and AppVeyor) will
143start their builds and eventually upload the packaging artifacts to S3.
144Wait until this process is finished by monitoring the two CIs:
145
146 * https://travis-ci.org/edenhill/librdkafka
147 * https://ci.appveyor.com/project/edenhill/librdkafka
148
149
150## Publish release on github
151
152Open up the release page on github that was created above.
153
154Run the following command to get checksums of the github release assets:
155
156    $ packaging/tools/gh-release-checksums.py <the-tag>
157
158It will take some time for the script to download the files, when done
159paste the output to the end of the release page.
160
161Make sure the release page looks okay, is still correct (check for new commits),
162and has the correct tag, then click Publish release.
163
164
165### Create NuGet package
166
167On a Linux host with docker installed, this will also require S3 credentials
168to be set up.
169
170    $ cd packaging/nuget
171    $ pip3 install -r requirements.txt  # if necessary
172    $ ./release.py v0.11.1-RC1
173
174Test the generated librdkafka.redist.0.11.1-RC1.nupkg and
175then upload it to NuGet manually:
176
177 * https://www.nuget.org/packages/manage/upload
178
179
180### Create static bundle (for Go)
181
182    $ cd packaging/nuget
183    $ ./release.py --class StaticPackage v0.11.1-RC1
184
185Follow the Go client release instructions for updating its bundled librdkafka
186version based on the tar ball created here.
187
188
189### Homebrew recipe update
190
191The brew-update-pr.sh script automatically pushes a PR to homebrew-core
192with a patch to update the librdkafka version of the formula.
193This should only be done for final releases and not release candidates.
194
195On a MacOSX host with homebrew installed:
196
197    $ cd package/homebrew
198    # Dry-run first to see that things are okay.
199    $ ./brew-update-pr.sh v0.11.1
200    # If everything looks good, do the live push:
201    $ ./brew-update-pr.sh --upload v0.11.1
202
203
204### Deb and RPM packaging
205
206Debian and RPM packages are generated by Confluent packaging in a separate
207process and the resulting packages are made available on Confluent's
208APT and YUM repositories.
209
210That process is outside the scope of this document.
211
212See the Confluent docs for instructions how to access these packages:
213https://docs.confluent.io/current/installation.html
214