1.. Licensed to the Apache Software Foundation (ASF) under one
2   or more contributor license agreements.  See the NOTICE file
3   distributed with this work for additional information
4   regarding copyright ownership.  The ASF licenses this file
5   to you under the Apache License, Version 2.0 (the
6   "License"); you may not use this file except in compliance
7   with the License.  You may obtain a copy of the License at
8
9   http://www.apache.org/licenses/LICENSE-2.0
10
11   Unless required by applicable law or agreed to in writing,
12   software distributed under the License is distributed on an
13   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14   KIND, either express or implied.  See the License for the
15   specific language governing permissions and limitations
16   under the License.
17
18.. include:: ../../../common.defs
19
20.. _developer-plugins-io:
21
22IO
23**
24
25This chapter contains the following sections:
26
27.. toctree::
28   :maxdepth: 2
29
30   net-vconnections.en
31   transformations.en
32   vios.en
33   io-buffers.en
34   cache-api.en
35
36.. _sdk-vconnections:
37
38Vconnections
39============
40
41A User's Perspective
42--------------------
43
44To use a vconnection, a user must first get a handle to one. This
45is usually accomplished by having it handed to the user; the user
46may also simply issue a call that creates a vconnection (such as
47c:func:`TSNetConnect`). In the case of transform plugins, the plugin
48creates a transformation vconnection viav :c:func:`TSTransformCreate`
49and then accesses the output vconnection using
50:c:func:`TSTransformOutputVConnGet`.
51
52After getting a handle to a vconnection, the user can then issue a read
53or write call. It's important to note that not all vconnections support
54both reading and writing - as of yet, there has not been a need to query
55a vconnection about whether it can perform a read or write operation.
56That ability should be obvious from context.
57
58To issue a read or write operation, a user calls :c:func:`TSVConnRead` or
59:c:func:`TSVConnWrite`. These two operations both return ``VIO (TSVIO)``. The
60VIO describes the operation being performed and how much progress has
61been made. Transform plugins initiate output to the downstream
62vconnection by calling :c:func:`TSVConnWrite`.
63
64A vconnection read or write operation is different from a normal UNIX
65:manpage:`read(2)` or :manpage:`write(2)` operation. Specifically, the vconnection
66operation can specify more data to be read or written than exists in the
67buffer handed to the operation. For example, it's typical to issue a
68read for ``INT64_MAX`` (9 quintillion) bytes from a network vconnection
69in order to read all the data from the network connection until the end
70of stream is reached. This contrasts with the usual UNIX fashion of
71issuing repeated calls to :manpage:`read(2)` until one of the calls finally
72returns ``0`` to indicate the end of stream was reached (indeed, the
73underlying implementation of vconnections on UNIX still does issue those
74calls to :manpage:`read(2)`, but the interface does not expose that detail).
75
76At most, a given vconnection can have one read operation and one write
77operation being performed on it. This is restricted both by design and
78common sense: if two write operations were performed on a single
79vconnection, then the user would not be able to specify which should
80occur first and the output would occur in an intermingled fashion. Note
81that both a read operation and a write operation can happen on a single
82vconnection at the same time; the restriction is for more than one
83operation of the same type.
84
85One obvious issue is that the buffer passed to :c:func:`TSVConnRead` and
86:c:func:`TSVConnWrite` won't be large enough - there is no reasonable way to
87make a buffer that can hold ``INT64_MAX`` (9 quintillion) bytes! The
88secret is that vconnections engage in a protocol whereby they signal to
89the user (via the continuation passed to :c:func:`TSVConnRead` and
90:c:func:`TSVConnWrite`) that they have emptied the buffers passed to them and
91are ready for more data. When this occurs, it is up to the user to add
92more data to the buffers (or wait for more data to be added) and then
93wake up the vconnection by calling :c:func:`TSVIOReenable` on the VIO
94describing the operation. :c:func:`TSVIOReenable` specifies that the buffer
95for the operation has been modified and that the vconnection should
96reexamine it to see if it can make further progress.
97
98The null transform plugin provides an example of how this is done. Below
99is a prototype for :c:func:`TSVConnWrite`:
100
101.. code-block:: c
102
103     TSVIO TSVConnWrite (TSVConn connp, TSCont contp, TSIOBufferReader readerp, int nbytes)
104
105The ``connp`` is the vconnection the user is writing to and ``contp`` is
106the "user" - i.e., the continuation that ``connp`` calls back when it
107has emptied its buffer and is ready for more data.
108
109The call made in the null transform plugin is:
110
111.. code-block:: c
112
113      TSVConnWrite (output_conn, contp, data->output_reader, TSVIONBytesGet (input_vio));
114
115In the example above, ``contp`` is the transformation vconnection that
116is writing to the output vconnection. The number of bytes to be written
117is obtained from ``input_vio`` by :c:func:`TSVIONBytesGet`.
118
119When a vconnection calls back its user to indicate that it wants more
120data (or when some other condition has occurred), it issues a call to
121:c:func:`TSContCall`. It passes the ``TSVIO`` describing the operation as the
122data parameter, and one of the values below as the event parameter.
123
124``TS_EVENT_ERROR``
125    Indicates an error has occurred on the vconnection. This will
126    happen for network IO if the underlying :manpage:`read(2)` or
127    :manpage:`write(2)` call returns an error.
128
129``TS_EVENT_VCONN_READ_READY``
130    The vconnection has placed data in the buffer passed to an
131    :c:func:`TSVConnRead` operation and it would like to do more IO, but the
132    buffer is now full. When the user consumes the data from the buffer,
133    this should re-enable the VIO so it indicates to the vconnection
134    that the buffer has been modified.
135
136``TS_EVENT_VCONN_WRITE_READY``
137    The vconnection has removed data from the buffer passed to an
138    :c:func:`TSVConnWrite` operation and it would like to do more IO, but the
139    buffer does not have enough data in it. When placing more data in
140    the buffer, the user should re-enable the VIO so it indicates to the
141    vconnection that the buffer has been modified.
142
143``TS_EVENT_VCONN_READ_COMPLETE``
144    The vconnection has read all the bytes specified by an
145    :c:func:`TSVConnRead` operation. The vconnection can now be used to
146    initiate a new IO operation.
147
148``TS_EVENT_VCONN_WRITE_COMPLETE``
149    The vconnection has written all the bytes specified by an
150    :c:func:`TSVConnWrite` operation and can now be used to initiate a new IO
151    operation.
152
153``TS_EVENT_VCONN_EOS``
154    An attempt was made to read past the end of the stream of bytes
155    during the handling of an :c:func:`TSVConnRead` operation. This event
156    occurs when the number of bytes available for reading from a
157    vconnection is less than the number of bytes the user specifies
158    should be read from the vconnection in a call to :c:func:`TSVConnRead`. A
159    common case where this occurs is when the user specifies that
160    ``INT64_MAX`` bytes are to be read from a network connection.
161
162For example: the null transform plugin's transformation receives
163``TS_EVENT_VCONN_WRITE_READY`` and ``TS_EVENT_VCONN_WRITE_COMPLETE``
164events from the downstream vconnection as a result of the call to
165:c:func:`TSVConnWrite`.
166
167After using a vconnection, the user must call :c:func:`TSVConnClose` or
168:c:func:`TSVConnAbort`. While both calls indicate that the vconnection can
169destroy itself, :c:func:`TSVConnAbort` should be used when the connection is
170being closed abnormally. After a call to :c:func:`TSVConnClose` or
171:c:func:`TSVConnAbort`, the user will not be called back by the vconnection
172again.
173
174Sometimes it's desirable to simply close down the write portion of a
175connection while keeping the read portion open. This can be accomplished
176via the :c:func:`TSVConnShutdown` function, which shuts down either the read
177or write portion of a vconnection. *Shutdown* means that the vconnection
178will no longer call back the user with events for the portion of the
179connection that was shut down. For example: if the user shuts down the
180write portion of a connection, then the ``TS_EVENT_VCONN_WRITE_READY``
181or ``TS_EVENT_VCONN_WRITE_COMPLETE`` events will not be produced. In the
182null transform plugin, the write operation is shut down with a call to
183:c:func:`TSVConnShutdown`. To learn how vconnections are used in
184transformation plugins, see :ref:`Writing Content Transform
185Plugins <WritingContentTransformPlugin>`.
186
187The vconnection functions are listed below:
188
189-  :c:func:`TSVConnAbort`
190-  :c:func:`TSVConnClose`
191-  :c:func:`TSVConnClosedGet`
192-  :c:func:`TSVConnRead`
193-  :c:func:`TSVConnReadVIOGet`
194-  :c:func:`TSVConnShutdown`
195-  :c:func:`TSVConnWrite`
196-  :c:func:`TSVConnWriteVIOGet`
197