1.. 2 Copyright (c) 2017-2019 Varnish Software AS 3 SPDX-License-Identifier: BSD-2-Clause 4 See LICENSE file for full text of license 5 6.. _whatsnew_upgrading_5.2: 7 8%%%%%%%%%%%%%%%%%%%%%%%% 9Upgrading to Varnish 5.2 10%%%%%%%%%%%%%%%%%%%%%%%% 11 12Varnish statistics and logging 13============================== 14 15There are extensive changes under the hood with respect to statistics 16counters, but these should all be transparent at the user-level. 17 18varnishd parameters 19=================== 20 21The :ref:`ref_param_vsm_space` and ``cli_buffer`` 22parameters are now deprecated and ignored. They will be removed 23in a future major release. 24 25The updated shared memory implementation manages space automatically, so 26it no longer needs :ref:`ref_param_vsm_space`. Memory for the CLI 27command buffer is now dynamically allocated. 28 29We have updated the documentation for :ref:`ref_param_send_timeout`, 30:ref:`ref_param_idle_send_timeout`, :ref:`ref_param_timeout_idle` and 31:ref:`ref_param_ban_cutoff`. 32 33Added the debug bit ``vmod_so_keep``, see :ref:`ref_param_debug` and 34the notes about changes for developers below. 35 36Changes to VCL 37============== 38 39We have added a few new variables and clarified some matters. VCL 40written for Varnish 5.1 should run without changes on 5.2. 41 42Consistent symbol names 43~~~~~~~~~~~~~~~~~~~~~~~ 44 45VCL symbols originate from various parts of Varnish: there are built-in 46variables, subroutines, functions, and the free-form headers. Symbols 47may live in a namespace denoted by the ``'.'`` (dot) character as in 48``req.http.Cache-Control``. When you create a VCL label, a new symbol 49becomes available, named after the label. Storage backends always have 50a name, even if you don't specify one, and they can also be accessed in 51VCL: for example ``storage.Transient``. 52 53Because headers and VCL names could contain dashes, while subroutines or 54VMOD objects couldn't, this created an inconsistency. All symbols follow 55the same rules now and must follow the same (case-insensitive) pattern: 56``[a-z][a-z0-9_-]*``. 57 58You can now write code like:: 59 60 sub my-sub { 61 new my-obj = my_vmod.my_constuctor(storage.my-store); 62 } 63 64 sub vcl_init { 65 call my-sub; 66 } 67 68As you may notice in the example above, it is not possible yet to have 69dashes in a vmod symbol. 70 71Long storage backend names used to be truncated due to a limitation in 72the VSC subsystem, this is no longer the case. 73 74VCL variables 75~~~~~~~~~~~~~ 76 77``req.hash`` and ``bereq.hash`` 78------------------------------- 79 80Added ``req.hash`` and ``bereq.hash``, which contain the hash value 81computed by Varnish for cache lookup in the current transaction, to 82be used in client or backend context, respectively. Their data type 83is BLOB, and they contain the raw binary hash. 84 85You can use :ref:`vmod_blob(3)` to work with the hashes:: 86 87 import blob; 88 89 sub vcl_backend_fetch { 90 # Send the transaction hash to the backend as a hex string 91 set bereq.http.Hash = blob.encode(HEX, blob=bereq.hash); 92 } 93 94 sub vcl_deliver { 95 # Send the hash in a response header as a base64 string 96 set resp.http.Hash = blob.encode(BASE64, blob=req.hash); 97 } 98 99``server.identity`` 100------------------- 101 102If the ``-i`` option is not set in the invocation of ``varnishd``, 103then ``server.identity`` is set to the host name (as returned by 104``gethostname(3)``). Previously, ``server.identity`` defaulted to the 105value of the ``-n`` option (or the default instance name if ``-n`` was 106not set). See :ref:`varnishd(1)`. 107 108``bereq.is_bgfetch`` 109-------------------- 110 111Added ``bereq.is_bgfetch``, which is readable in backend contexts, and 112is true if the fetch takes place in the background. That is, it is 113true if Varnish found a response in the cache whose TTL was expired, 114but was still in grace time. Varnish returns the stale cached response 115to the client, and initiates the background fetch to refresh the cache 116object. 117 118``req.backend_hint`` 119-------------------- 120 121We have clarified what happens to ``req.backend_hint`` on a client 122restart -- it gets reset to the default backend. So you might want to 123make sure that the backend hint gets set the way you want in that 124situation. 125 126vmod_std 127~~~~~~~~ 128 129Added :ref:`std.file_exists()`. 130 131New VMODs in the standard distribution 132~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 133 134See :ref:`vmod_blob(3)`, :ref:`vmod_purge(3)` and 135:ref:`vmod_vtc(3)`. Read about them in :ref:`whatsnew_new_vmods`. 136 137Bans 138~~~~ 139 140We have clarified the interpretation of a ban when a comparison in the 141ban expression is attempted against an unset field, see 142:ref:`vcl(7)_ban` in :ref:`vcl(7)`. 143 144Other changes 145============= 146 147* ``varnishd(1)``: 148 149 * The total size of the shared memory space for logs and counters 150 no longer needs to be configured explicitly and therefore the 151 second subargument to ``-l`` is now ignored. 152 153 * The default value of ``server.identity`` when the ``-i`` option is 154 not set has been changed as noted above. 155 156 * Also, ``-i`` no longer determines the ``ident`` field used by 157 ``syslog(3)``; now Varnish is always identified by the string 158 ``varnishd`` in the syslog. 159 160 * On a system that supports ``setproctitle(3)``, the Varnish 161 management process will appear in the output of ``ps(1)`` as 162 ``Varnish-Mgt``, and the child process as ``Varnish-Child``. If 163 the ``-i`` option has been set, then these strings in the ps 164 output are followed by ``-i`` and the identity string set by the 165 option. 166 167 * The ``-f`` option for a VCL source file now honors the 168 ``vcl_path`` parameter if a relative file name is used, see 169 :ref:`varnishd(1)` and :ref:`ref_param_vcl_path`. 170 171 * The ``-a`` option can now take a name, for example ``-a 172 admin=127.0.0.1:88`` to identify an address used for 173 administrative requests but not regular client traffic. Otherwise, 174 a default name is selected for the listen address (``a0``, ``a1`` 175 and so forth). Endpoint names appear in the log output, as noted 176 below, and may become accessible in VCL in the future. 177 178* ``varnishstat(1)``: 179 180 * In curses mode, the top two lines showing uptimes for the 181 management and child processes show the text ``Not Running`` if 182 one or both of the processes are down. 183 184 * The interpretation of multiple ``-f`` options in the command line 185 has changed slightly, see :ref:`varnishstat(1)`. 186 187 * The ``type`` and ``ident`` fields have been removed from the XML 188 and JSON output formats, see :ref:`varnishstat(1)`. 189 190 * The ``MAIN.s_req`` statistic has been removed, as it was identical 191 to ``MAIN.client_req``. 192 193 * Added the counter ``req_dropped``. Similar to ``sess_dropped``, 194 this is the number of times an HTTP/2 stream was refused because 195 the internal queue is full. See :ref:`varnish-counters(7)` and 196 :ref:`ref_param_thread_queue_limit`. 197 198* ``varnishlog(1)``: 199 200 * The ``Hit``, ``HitMiss`` and ``HitPass`` log records grew an 201 additional field with the remaining TTL of the object at the time 202 of the lookup. While this should greatly help troubleshooting, 203 it might break tools relying on those records to get the VXID of 204 the object hit during lookup. 205 206 Instead of using ``Hit``, such tools should now use ``Hit[1]``, 207 and the same applies to ``HitMiss`` and ``HitPass``. 208 209 The ``Hit`` record also grew two more fields for the grace and 210 keep periods. This should again be useful for troubleshooting. 211 212 See :ref:`vsl(7)`. 213 214 * The ``SessOpen`` log record displays the name of the listen address 215 instead of the endpoint in its 3rd field. 216 217 See :ref:`vsl(7)`. 218 219 * The output format of ``VCL_trace`` log records, which appear if 220 you have switched on the ``VCL_trace`` flag in the VSL mask, has 221 changed to include the VCL configuration name. See :ref:`vsl(7)` 222 and :ref:`ref_param_vsl_mask`. 223 224* ``varnishtest(1)`` and ``vtc(7)``: 225 226 * When varnishtest is invoked with ``-L`` or ``-l``, Varnish 227 instances started by a test do not clean up their copies of VMOD 228 shared objects when they stop. See the note about ``vmod_so_keep`` 229 below. 230 231 * Added the feature switch ``ignore_unknown_macro`` for test cases, 232 see :ref:`vtc(7)`. 233 234* ``varnishncsa(1)`` 235 236 * Field specifiers (such as the 1 in ``Hit[1]``) are now limited to 237 to 255, see :ref:`varnishncsa(1)`. 238 239* The ``-N`` command-line option, which was previously available for 240 ``varnishlog(1)``, ``varnishstat(1)``, ``varnishncsa(1)`` and 241 ``varnishhist(1)``, is not compatible with the changed internal 242 logging API, and has been retired. 243 244* Changes for developers: 245 246 * The VSM and VSC APIs for shared memory and statistics have 247 changed, and may necessitate changes in client applications, see 248 :ref:`whatsnew_vsm_vsc_5.2`. 249 250 * Added the ``$ABI`` directive for VMOD vcc declarations, see 251 :ref:`whatsnew_abi`. 252 253 * There have been some minor changes in the VRT API, which may be 254 used for VMODs and client apps, see :ref:`whatsnew_vrt_5.2`. 255 256 * The VUT API (for Varnish UTilities), which facilitates the 257 development of client apps, is now publicly available, see 258 :ref:`whatsnew_vut_5.2`. 259 260 * The debug bit ``vmod_so_keep`` instructs Varnish not to clean 261 up its copies of VMOD shared objects when it stops. This makes 262 it possible for VMOD authors to load their code into a debugger 263 after a varnishd crash. See :ref:`ref_param_debug`. 264 265*eof* 266