1.. _publisher: 2 3========= 4Publisher 5========= 6 7The Publisher subsystem is a pluggable interface for outbound messages, 8structured following the OpenConfig / IETF YANG models. 9The messages can be published over a variety of services -- see 10:ref:`publisher-modules`. 11From the command line, the Publisher module can be selected using the 12``--publisher`` option, e.g.: 13 14.. code-block:: bash 15 16 $ napalm-logs --publisher kafka 17 18From the configuration file, the Publisher can be specified using the 19``publisher`` option, eventually with several options. The options depend on the 20nature of the Publisher. 21 22Example: publisher configuration using the default configuration 23 24.. code-block:: yaml 25 26 publisher: zmq 27 28Example: publisher configuration using custom options 29 30.. code-block:: yaml 31 32 publisher: 33 kafka: 34 topic: napalm-logs-out 35 36.. note:: 37 38 The IP Address / port for the Publisher be specified using the 39 :ref:`configuration-options-publish-address` and 40 :ref:`configuration-options-publish-port` 41 configuration options. 42 43Multiple publishers 44------------------- 45 46.. versionadded:: 0.4.0 47 48It is possible to export the structured napalm-logs structured documents into 49multiple systems, over multiple channels, each with its separate configuration 50options. This feature is available only from the configuration file, e.g.: 51 52.. code-block:: yaml 53 54 publisher: 55 - zmq: 56 address: 1.2.3.4 57 port: 5678 58 - kafka: 59 topic: napalm-logs-out 60 - http: 61 address: https://example.com/webhook 62 63.. _publisher-modules: 64 65Available publishers and their options 66--------------------------------------- 67 68.. toctree:: 69 :maxdepth: 1 70 71 alerta 72 cli 73 http 74 kafka 75 log 76 zmq 77 78Globally available options 79-------------------------- 80 81Additionally, the user can configure the following options, available to all 82publishers: 83 84.. _publisher-opts-disable-security: 85 86``disable_security``: ``False`` 87------------------------------- 88 89The message encryption can be disabled per publisher as well. Similar to the 90main :ref:`configuration-options-disable-security` configuration option, is it 91recommended **not to disable security**, though this can be needed in certain 92particular cases. 93 94Configuration example: 95 96.. code-block:: yaml 97 98 publisher: 99 - cli: 100 disable_security: true 101 - zmq: {} 102 103.. _publisher-opts-error-whitelist: 104 105``error_whitelist``: ``[]`` 106--------------------------- 107 108.. versionadded:: 0.4.0 109 110Publish only the error messages included in this list. The whitelist/blacklist 111logic is implemented in such a way that if anything is added in this list, 112*only* these message types will be published and nothing else. 113 114Default: ``None`` (empty list) 115 116Configuration example: 117 118.. code-block:: yaml 119 120 publisher: 121 - kafka: 122 error_whitelist: 123 - UNKNOWN 124 - RAW 125 - zmq: 126 error_whitelist: 127 - BGP_MD5_INCORRECT 128 - BGP_NEIGHBOR_STATE_CHANGED 129 130.. _publisher-opts-error-blacklist: 131 132``error_blacklist``: ``['RAW', 'UNKNOWN']`` 133------------------------------------------- 134 135.. versionadded:: 0.4.0 136 137Filter out the error types publisher. The error messages included in this list 138will not be published. 139 140Default: ``RAW, UNKNOWN`` (both ``RAW`` and ``UNKNOWN`` message types will not 141be published by default). 142 143Configuration example: 144 145.. code-block:: yaml 146 147 publisher: 148 - kafka: 149 error_blacklist: 150 - UNKNOWN 151 - RAW 152 - USER_ENTER_CONFIG_MODE 153 - zmq: 154 error_blacklist: 155 - UNKNOWN 156 157.. _publisher-opts-no-encrypt: 158 159``no_encrypt``: ``False`` 160------------------------- 161 162.. versionadded:: 0.4.2 163 164Do not encrypt messages for this over this publisher 165 166Configuration example: 167 168.. code-block:: yaml 169 170 publisher: 171 - kafka: 172 no_encrypt: True 173 174.. _publisher-opts-only-raw: 175 176``only_raw``: ``False`` 177----------------------- 178 179.. versionadded:: 0.4.0 180 181When this option is enabled, the publisher will publish *only* the syslog 182messages that could not be parsed. 183 184Example: 185 186.. code-block:: yaml 187 188 publisher: 189 - zmq: 190 address: 1.2.3.4 191 port: 1234 192 - zmq: 193 address: 5.6.7.8 194 port: 5678 195 only_raw: true 196 197.. note:: 198 199 This option is a shortcut to the :ref:`publisher-opts-error-whitelist` 200 configuration option introduced in 0.4.0 (codename Crowbar), by adding the 201 ``RAW`` message to the whitelist message types, i.e., 202 203 .. code-block:: yaml 204 205 publisher: 206 - zmq: 207 address: 1.2.3.4 208 port: 1234 209 - zmq: 210 address: 5.6.7.8 211 port: 5678 212 error_whitelist: 213 - RAW 214 215.. _publisher-opts-only-unknown: 216 217``only_unknown``: ``False`` 218--------------------------- 219 220.. versionadded:: 0.4.0 221 222When this option is configured, napalm-logs will publish *only* the structured 223documents that are marked as ``UNKNWON`` (i.e., napalm-logs was unable to parse 224the message and determine the operating system). 225 226Example: 227 228.. code-block:: yaml 229 230 publisher: 231 kafka: 232 only_unknown: true 233 234.. note:: 235 236 This option is a shortcut to the :ref:`publisher-opts-error-whitelist` 237 option introduced in 0.4.0 (codename Crowbar), by adding the ``UNKNOW`` 238 message type to the whitelist, i.e., 239 240 .. code-block:: yaml 241 242 publisher: 243 kafka: 244 error_whitelist: 245 - UNKNOWN 246 247 248.. _publisher-opts-send-raw: 249 250``send_raw``: ``False`` 251----------------------- 252 253If this option is set, all processed syslog messages, even ones that have not 254matched a configured error, will be published over the specified transport. 255This can be used to forward to log server for storage. 256 257Example: 258 259.. code-block:: yaml 260 261 publisher: 262 zmq: 263 send_raw: true 264 265.. note:: 266 267 This option is just a shortcut to the 268 :ref:`publisher-opts-error-blacklist` configuration option introduced in 269 0.4.0 (codename Crowbar), by removing the ``RAW`` error type from the 270 blacklisted message types, i.e., 271 272 .. code-block:: yaml 273 274 publisher: 275 zmq: 276 error_blacklist: 277 - UNKNOWN 278 279.. _publisher-opts-send-unknown: 280 281``send_unknown``: ``False`` 282--------------------------- 283 284If this option is set, all processed syslog messages, even ones that have not 285matched a certain operating system, will be published over the specified 286transport. This can be used to forward to log server for storage. 287 288Example: 289 290.. code-block:: yaml 291 292 publisher: 293 kafka: 294 send_unknown: true 295 296.. note:: 297 298 This option is just a shortcut to the 299 :ref:`publisher-opts-error-blacklist` option introduced in 0.4.0 (codename 300 Crowbar), by removing the ``UNKNOWN`` message from the blacklist, i.e., 301 302 .. code-block:: yaml 303 304 publisher: 305 kafka: 306 error_blacklist: 307 - RAW 308 309.. _publisher-opts-serializer: 310 311``serializer``: ``msgpack`` 312--------------------------- 313 314.. versionadded:: 0.4.0 315 316The serializer to be used when publishing the structure napalm-logs document. 317 318Default: :ref:`serializer-msgpack`. 319 320You can specify a separate serialize per publisher, e.g.: 321 322.. code-block:: yaml 323 324 publisher: 325 - kafka: 326 serializer: json 327 - cli: 328 serializer: pprint 329 330.. _publisher-opts-strip-message-details: 331 332``strip_message_details``: ``False`` 333------------------------------------ 334 335.. versionadded:: 0.7.0 336 337Strip the ``message_details`` key before publishing the object. 338 339Configuration example: 340 341.. code-block:: yaml 342 343 publisher: 344 - kafka: 345 strip_message_details: true 346