1.. Licensed to the Apache Software Foundation (ASF) under one 2 or more contributor license agreements. See the NOTICE file 3 distributed with this work for additional information 4 regarding copyright ownership. The ASF licenses this file 5 to you under the Apache License, Version 2.0 (the 6 "License"); you may not use this file except in compliance 7 with the License. You may obtain a copy of the License at 8 9 http://www.apache.org/licenses/LICENSE-2.0 10 11 Unless required by applicable law or agreed to in writing, 12 software distributed under the License is distributed on an 13 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 14 KIND, either express or implied. See the License for the 15 specific language governing permissions and limitations 16 under the License. 17 18.. include:: ../../../common.defs 19 20.. _developer-plugins-io: 21 22IO 23** 24 25This chapter contains the following sections: 26 27.. toctree:: 28 :maxdepth: 2 29 30 net-vconnections.en 31 transformations.en 32 vios.en 33 io-buffers.en 34 cache-api.en 35 36.. _sdk-vconnections: 37 38Vconnections 39============ 40 41A User's Perspective 42-------------------- 43 44To use a vconnection, a user must first get a handle to one. This 45is usually accomplished by having it handed to the user; the user 46may also simply issue a call that creates a vconnection (such as 47c:func:`TSNetConnect`). In the case of transform plugins, the plugin 48creates a transformation vconnection viav :c:func:`TSTransformCreate` 49and then accesses the output vconnection using 50:c:func:`TSTransformOutputVConnGet`. 51 52After getting a handle to a vconnection, the user can then issue a read 53or write call. It's important to note that not all vconnections support 54both reading and writing - as of yet, there has not been a need to query 55a vconnection about whether it can perform a read or write operation. 56That ability should be obvious from context. 57 58To issue a read or write operation, a user calls :c:func:`TSVConnRead` or 59:c:func:`TSVConnWrite`. These two operations both return ``VIO (TSVIO)``. The 60VIO describes the operation being performed and how much progress has 61been made. Transform plugins initiate output to the downstream 62vconnection by calling :c:func:`TSVConnWrite`. 63 64A vconnection read or write operation is different from a normal UNIX 65:manpage:`read(2)` or :manpage:`write(2)` operation. Specifically, the vconnection 66operation can specify more data to be read or written than exists in the 67buffer handed to the operation. For example, it's typical to issue a 68read for ``INT64_MAX`` (9 quintillion) bytes from a network vconnection 69in order to read all the data from the network connection until the end 70of stream is reached. This contrasts with the usual UNIX fashion of 71issuing repeated calls to :manpage:`read(2)` until one of the calls finally 72returns ``0`` to indicate the end of stream was reached (indeed, the 73underlying implementation of vconnections on UNIX still does issue those 74calls to :manpage:`read(2)`, but the interface does not expose that detail). 75 76At most, a given vconnection can have one read operation and one write 77operation being performed on it. This is restricted both by design and 78common sense: if two write operations were performed on a single 79vconnection, then the user would not be able to specify which should 80occur first and the output would occur in an intermingled fashion. Note 81that both a read operation and a write operation can happen on a single 82vconnection at the same time; the restriction is for more than one 83operation of the same type. 84 85One obvious issue is that the buffer passed to :c:func:`TSVConnRead` and 86:c:func:`TSVConnWrite` won't be large enough - there is no reasonable way to 87make a buffer that can hold ``INT64_MAX`` (9 quintillion) bytes! The 88secret is that vconnections engage in a protocol whereby they signal to 89the user (via the continuation passed to :c:func:`TSVConnRead` and 90:c:func:`TSVConnWrite`) that they have emptied the buffers passed to them and 91are ready for more data. When this occurs, it is up to the user to add 92more data to the buffers (or wait for more data to be added) and then 93wake up the vconnection by calling :c:func:`TSVIOReenable` on the VIO 94describing the operation. :c:func:`TSVIOReenable` specifies that the buffer 95for the operation has been modified and that the vconnection should 96reexamine it to see if it can make further progress. 97 98The null transform plugin provides an example of how this is done. Below 99is a prototype for :c:func:`TSVConnWrite`: 100 101.. code-block:: c 102 103 TSVIO TSVConnWrite (TSVConn connp, TSCont contp, TSIOBufferReader readerp, int nbytes) 104 105The ``connp`` is the vconnection the user is writing to and ``contp`` is 106the "user" - i.e., the continuation that ``connp`` calls back when it 107has emptied its buffer and is ready for more data. 108 109The call made in the null transform plugin is: 110 111.. code-block:: c 112 113 TSVConnWrite (output_conn, contp, data->output_reader, TSVIONBytesGet (input_vio)); 114 115In the example above, ``contp`` is the transformation vconnection that 116is writing to the output vconnection. The number of bytes to be written 117is obtained from ``input_vio`` by :c:func:`TSVIONBytesGet`. 118 119When a vconnection calls back its user to indicate that it wants more 120data (or when some other condition has occurred), it issues a call to 121:c:func:`TSContCall`. It passes the ``TSVIO`` describing the operation as the 122data parameter, and one of the values below as the event parameter. 123 124``TS_EVENT_ERROR`` 125 Indicates an error has occurred on the vconnection. This will 126 happen for network IO if the underlying :manpage:`read(2)` or 127 :manpage:`write(2)` call returns an error. 128 129``TS_EVENT_VCONN_READ_READY`` 130 The vconnection has placed data in the buffer passed to an 131 :c:func:`TSVConnRead` operation and it would like to do more IO, but the 132 buffer is now full. When the user consumes the data from the buffer, 133 this should re-enable the VIO so it indicates to the vconnection 134 that the buffer has been modified. 135 136``TS_EVENT_VCONN_WRITE_READY`` 137 The vconnection has removed data from the buffer passed to an 138 :c:func:`TSVConnWrite` operation and it would like to do more IO, but the 139 buffer does not have enough data in it. When placing more data in 140 the buffer, the user should re-enable the VIO so it indicates to the 141 vconnection that the buffer has been modified. 142 143``TS_EVENT_VCONN_READ_COMPLETE`` 144 The vconnection has read all the bytes specified by an 145 :c:func:`TSVConnRead` operation. The vconnection can now be used to 146 initiate a new IO operation. 147 148``TS_EVENT_VCONN_WRITE_COMPLETE`` 149 The vconnection has written all the bytes specified by an 150 :c:func:`TSVConnWrite` operation and can now be used to initiate a new IO 151 operation. 152 153``TS_EVENT_VCONN_EOS`` 154 An attempt was made to read past the end of the stream of bytes 155 during the handling of an :c:func:`TSVConnRead` operation. This event 156 occurs when the number of bytes available for reading from a 157 vconnection is less than the number of bytes the user specifies 158 should be read from the vconnection in a call to :c:func:`TSVConnRead`. A 159 common case where this occurs is when the user specifies that 160 ``INT64_MAX`` bytes are to be read from a network connection. 161 162For example: the null transform plugin's transformation receives 163``TS_EVENT_VCONN_WRITE_READY`` and ``TS_EVENT_VCONN_WRITE_COMPLETE`` 164events from the downstream vconnection as a result of the call to 165:c:func:`TSVConnWrite`. 166 167After using a vconnection, the user must call :c:func:`TSVConnClose` or 168:c:func:`TSVConnAbort`. While both calls indicate that the vconnection can 169destroy itself, :c:func:`TSVConnAbort` should be used when the connection is 170being closed abnormally. After a call to :c:func:`TSVConnClose` or 171:c:func:`TSVConnAbort`, the user will not be called back by the vconnection 172again. 173 174Sometimes it's desirable to simply close down the write portion of a 175connection while keeping the read portion open. This can be accomplished 176via the :c:func:`TSVConnShutdown` function, which shuts down either the read 177or write portion of a vconnection. *Shutdown* means that the vconnection 178will no longer call back the user with events for the portion of the 179connection that was shut down. For example: if the user shuts down the 180write portion of a connection, then the ``TS_EVENT_VCONN_WRITE_READY`` 181or ``TS_EVENT_VCONN_WRITE_COMPLETE`` events will not be produced. In the 182null transform plugin, the write operation is shut down with a call to 183:c:func:`TSVConnShutdown`. To learn how vconnections are used in 184transformation plugins, see :ref:`Writing Content Transform 185Plugins <WritingContentTransformPlugin>`. 186 187The vconnection functions are listed below: 188 189- :c:func:`TSVConnAbort` 190- :c:func:`TSVConnClose` 191- :c:func:`TSVConnClosedGet` 192- :c:func:`TSVConnRead` 193- :c:func:`TSVConnReadVIOGet` 194- :c:func:`TSVConnShutdown` 195- :c:func:`TSVConnWrite` 196- :c:func:`TSVConnWriteVIOGet` 197