1..
2	Copyright (c) 2017-2019 Varnish Software AS
3	SPDX-License-Identifier: BSD-2-Clause
4	See LICENSE file for full text of license
5
6.. _whatsnew_upgrading_5.2:
7
8%%%%%%%%%%%%%%%%%%%%%%%%
9Upgrading to Varnish 5.2
10%%%%%%%%%%%%%%%%%%%%%%%%
11
12Varnish statistics and logging
13==============================
14
15There are extensive changes under the hood with respect to statistics
16counters, but these should all be transparent at the user-level.
17
18varnishd parameters
19===================
20
21The :ref:`ref_param_vsm_space` and ``cli_buffer``
22parameters are now deprecated and ignored.  They will be removed
23in a future major release.
24
25The updated shared memory implementation manages space automatically, so
26it no longer needs :ref:`ref_param_vsm_space`. Memory for the CLI
27command buffer is now dynamically allocated.
28
29We have updated the documentation for :ref:`ref_param_send_timeout`,
30:ref:`ref_param_idle_send_timeout`, :ref:`ref_param_timeout_idle` and
31:ref:`ref_param_ban_cutoff`.
32
33Added the debug bit ``vmod_so_keep``, see :ref:`ref_param_debug` and
34the notes about changes for developers below.
35
36Changes to VCL
37==============
38
39We have added a few new variables and clarified some matters. VCL
40written for Varnish 5.1 should run without changes on 5.2.
41
42Consistent symbol names
43~~~~~~~~~~~~~~~~~~~~~~~
44
45VCL symbols originate from various parts of Varnish: there are built-in
46variables, subroutines, functions, and the free-form headers. Symbols
47may live in a namespace denoted by the ``'.'`` (dot) character as in
48``req.http.Cache-Control``. When you create a VCL label, a new symbol
49becomes available, named after the label. Storage backends always have
50a name, even if you don't specify one, and they can also be accessed in
51VCL: for example ``storage.Transient``.
52
53Because headers and VCL names could contain dashes, while subroutines or
54VMOD objects couldn't, this created an inconsistency. All symbols follow
55the same rules now and must follow the same (case-insensitive) pattern:
56``[a-z][a-z0-9_-]*``.
57
58You can now write code like::
59
60  sub my-sub {
61      new my-obj = my_vmod.my_constuctor(storage.my-store);
62  }
63
64  sub vcl_init {
65      call my-sub;
66  }
67
68As you may notice in the example above, it is not possible yet to have
69dashes in a vmod symbol.
70
71Long storage backend names used to be truncated due to a limitation in
72the VSC subsystem, this is no longer the case.
73
74VCL variables
75~~~~~~~~~~~~~
76
77``req.hash`` and ``bereq.hash``
78-------------------------------
79
80Added ``req.hash`` and ``bereq.hash``, which contain the hash value
81computed by Varnish for cache lookup in the current transaction, to
82be used in client or backend context, respectively. Their data type
83is BLOB, and they contain the raw binary hash.
84
85You can use :ref:`vmod_blob(3)` to work with the hashes::
86
87  import blob;
88
89  sub vcl_backend_fetch {
90      # Send the transaction hash to the backend as a hex string
91      set bereq.http.Hash = blob.encode(HEX, blob=bereq.hash);
92  }
93
94  sub vcl_deliver {
95      # Send the hash in a response header as a base64 string
96      set resp.http.Hash = blob.encode(BASE64, blob=req.hash);
97  }
98
99``server.identity``
100-------------------
101
102If the ``-i`` option is not set in the invocation of ``varnishd``,
103then ``server.identity`` is set to the host name (as returned by
104``gethostname(3)``). Previously, ``server.identity`` defaulted to the
105value of the ``-n`` option (or the default instance name if ``-n`` was
106not set). See :ref:`varnishd(1)`.
107
108``bereq.is_bgfetch``
109--------------------
110
111Added ``bereq.is_bgfetch``, which is readable in backend contexts, and
112is true if the fetch takes place in the background. That is, it is
113true if Varnish found a response in the cache whose TTL was expired,
114but was still in grace time. Varnish returns the stale cached response
115to the client, and initiates the background fetch to refresh the cache
116object.
117
118``req.backend_hint``
119--------------------
120
121We have clarified what happens to ``req.backend_hint`` on a client
122restart -- it gets reset to the default backend. So you might want to
123make sure that the backend hint gets set the way you want in that
124situation.
125
126vmod_std
127~~~~~~~~
128
129Added :ref:`std.file_exists()`.
130
131New VMODs in the standard distribution
132~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
133
134See :ref:`vmod_blob(3)`, :ref:`vmod_purge(3)` and
135:ref:`vmod_vtc(3)`. Read about them in :ref:`whatsnew_new_vmods`.
136
137Bans
138~~~~
139
140We have clarified the interpretation of a ban when a comparison in the
141ban expression is attempted against an unset field, see
142:ref:`vcl(7)_ban` in :ref:`vcl(7)`.
143
144Other changes
145=============
146
147* ``varnishd(1)``:
148
149  * The total size of the shared memory space for logs and counters
150    no longer needs to be configured explicitly and therefore the
151    second subargument to ``-l`` is now ignored.
152
153  * The default value of ``server.identity`` when the ``-i`` option is
154    not set has been changed as noted above.
155
156  * Also, ``-i`` no longer determines the ``ident`` field used by
157    ``syslog(3)``; now Varnish is always identified by the string
158    ``varnishd`` in the syslog.
159
160  * On a system that supports ``setproctitle(3)``, the Varnish
161    management process will appear in the output of ``ps(1)`` as
162    ``Varnish-Mgt``, and the child process as ``Varnish-Child``. If
163    the ``-i`` option has been set, then these strings in the ps
164    output are followed by ``-i`` and the identity string set by the
165    option.
166
167  * The ``-f`` option for a VCL source file now honors the
168    ``vcl_path`` parameter if a relative file name is used, see
169    :ref:`varnishd(1)` and :ref:`ref_param_vcl_path`.
170
171  * The ``-a`` option can now take a name, for example ``-a
172    admin=127.0.0.1:88`` to identify an address used for
173    administrative requests but not regular client traffic. Otherwise,
174    a default name is selected for the listen address (``a0``, ``a1``
175    and so forth). Endpoint names appear in the log output, as noted
176    below, and may become accessible in VCL in the future.
177
178* ``varnishstat(1)``:
179
180  * In curses mode, the top two lines showing uptimes for the
181    management and child processes show the text ``Not Running`` if
182    one or both of the processes are down.
183
184  * The interpretation of multiple ``-f`` options in the command line
185    has changed slightly, see :ref:`varnishstat(1)`.
186
187  * The ``type`` and ``ident`` fields have been removed from the XML
188    and JSON output formats, see :ref:`varnishstat(1)`.
189
190  * The ``MAIN.s_req`` statistic has been removed, as it was identical
191    to ``MAIN.client_req``.
192
193  * Added the counter ``req_dropped``. Similar to ``sess_dropped``,
194    this is the number of times an HTTP/2 stream was refused because
195    the internal queue is full. See :ref:`varnish-counters(7)` and
196    :ref:`ref_param_thread_queue_limit`.
197
198* ``varnishlog(1)``:
199
200  * The ``Hit``, ``HitMiss`` and ``HitPass`` log records grew an
201    additional field with the remaining TTL of the object at the time
202    of the lookup.  While this should greatly help troubleshooting,
203    it might break tools relying on those records to get the VXID of
204    the object hit during lookup.
205
206    Instead of using ``Hit``, such tools should now use ``Hit[1]``,
207    and the same applies to ``HitMiss`` and ``HitPass``.
208
209    The ``Hit`` record also grew two more fields for the grace and
210    keep periods.  This should again be useful for troubleshooting.
211
212    See :ref:`vsl(7)`.
213
214  * The ``SessOpen`` log record displays the name of the listen address
215    instead of the endpoint in its 3rd field.
216
217    See :ref:`vsl(7)`.
218
219  * The output format of ``VCL_trace`` log records, which appear if
220    you have switched on the ``VCL_trace`` flag in the VSL mask, has
221    changed to include the VCL configuration name. See :ref:`vsl(7)`
222    and :ref:`ref_param_vsl_mask`.
223
224* ``varnishtest(1)`` and ``vtc(7)``:
225
226  * When varnishtest is invoked with ``-L`` or ``-l``, Varnish
227    instances started by a test do not clean up their copies of VMOD
228    shared objects when they stop. See the note about ``vmod_so_keep``
229    below.
230
231  * Added the feature switch ``ignore_unknown_macro`` for test cases,
232    see :ref:`vtc(7)`.
233
234* ``varnishncsa(1)``
235
236  * Field specifiers (such as the 1 in ``Hit[1]``) are now limited to
237    to 255, see :ref:`varnishncsa(1)`.
238
239* The ``-N`` command-line option, which was previously available for
240  ``varnishlog(1)``, ``varnishstat(1)``, ``varnishncsa(1)`` and
241  ``varnishhist(1)``, is not compatible with the changed internal
242  logging API, and has been retired.
243
244* Changes for developers:
245
246  * The VSM and VSC APIs for shared memory and statistics have
247    changed, and may necessitate changes in client applications, see
248    :ref:`whatsnew_vsm_vsc_5.2`.
249
250  * Added the ``$ABI`` directive for VMOD vcc declarations, see
251    :ref:`whatsnew_abi`.
252
253  * There have been some minor changes in the VRT API, which may be
254    used for VMODs and client apps, see :ref:`whatsnew_vrt_5.2`.
255
256  * The VUT API (for Varnish UTilities), which facilitates the
257    development of client apps, is now publicly available, see
258    :ref:`whatsnew_vut_5.2`.
259
260  * The debug bit ``vmod_so_keep`` instructs Varnish not to clean
261    up its copies of VMOD shared objects when it stops. This makes
262    it possible for VMOD authors to load their code into a debugger
263    after a varnishd crash. See :ref:`ref_param_debug`.
264
265*eof*
266