1/* $Id$ 2 3 Part of SWI-Prolog 4 5 Author: Jan Wielemaker 6 E-mail: wielemak@science.uva.nl 7 WWW: http://www.swi-prolog.org 8 Copyright (C): 1985-2007, University of Amsterdam 9 10 This program is free software; you can redistribute it and/or 11 modify it under the terms of the GNU General Public License 12 as published by the Free Software Foundation; either version 2 13 of the License, or (at your option) any later version. 14 15 This program is distributed in the hope that it will be useful, 16 but WITHOUT ANY WARRANTY; without even the implied warranty of 17 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 18 GNU General Public License for more details. 19 20 You should have received a copy of the GNU General Public 21 License along with this library; if not, write to the Free Software 22 Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 23 24 As a special exception, if you link this library with other files, 25 compiled with a Free Software compiler, to produce an executable, this 26 library does not by itself cause the resulting executable to be covered 27 by the GNU General Public License. This exception does not however 28 invalidate any other reasons why the executable file might be covered by 29 the GNU General Public License. 30*/ 31 32:- module(http_stream, 33 [ http_chunked_open/3, % +Stream, -DataStream, +Options 34 stream_range_open/3, % +Stream, -DataStream, +Options 35 % CGI Stream interaction 36 cgi_open/4, % +Stream, -DataStream, :Hook, +Options 37 cgi_property/2, % +Stream, -Property 38 cgi_set/2, % +Stream, -Property 39 cgi_discard/1, % +Stream 40 is_cgi_stream/1 % +Stream 41 ]). 42 43:- expects_dialect(swi). 44:- assert(system:swi_io). 45 46:- use_module(library(shlib)). 47 48:- use_foreign_library(foreign(http_stream)). 49 50/** <module> HTTP Streams 51 52This module realises encoding and decoding filters, implemented as 53Prolog streams that read/write to an underlying stream. This allows for 54sequences of streams acting as an in-process pipeline. 55 56The predicate http_chunked_open/3 realises encoding and decoding of the 57HTTP _Chunked_ encoding. This encoding is an obligatory part of the HTTP 581.1 specification. Messages are split into chunks, each preceeded by the 59length of the chunk. Chunked encoding allows sending messages over a 60serial link (typically a TCP/IP stream) for which the reader knows when 61the message is ended. Unlike standard HTTP though, the sender does not 62need to know the message length in advance. The protocol allows for 63sending short chunks. This is supported totally transparent using a 64flush on the output stream. 65 66The predicate stream_range_open/3 handles the Content-length on an input 67stream for handlers that are designed to process an entire file. The 68filtering stream claims end-of-file after reading a specified number of 69bytes, dispite the fact that the underlying stream may be longer. 70 71@see The HTTP 1.1 protocol http://www.w3.org/Protocols/rfc2616/rfc2616.html 72@author Jan Wielemaker 73*/ 74 75%% http_chunked_open(+RawStream, -DataStream, +Options) is det. 76% 77% Create a stream to realise HTTP chunked encoding or decoding. 78% The technique is similar to library(zlib), using a Prolog stream 79% as a filter on another stream. Options: 80% 81% * close_parent(+Bool) 82% If =true= (default =false=), the parent stream is closed 83% if DataStream is closed. 84% 85% * max_chunk_size(+PosInt) 86% Define the maximum size of a chunk. Default is the 87% default buffer size of fully buffered streams (4096). 88% Larger values may improve throughput. It is also 89% allowed to use =|set_stream(DataStream, buffer(line))|= 90% on the data stream to get line-buffered output. See 91% set_stream/2 for details. Switching buffering to =false= 92% is supported. 93% 94% Here is example code to write a chunked data to a stream 95% 96% == 97% http_chunked_open(Out, S, []), 98% format(S, 'Hello world~n', []), 99% close(S). 100% == 101% 102% If a stream is known to contain chunked data, we can extract 103% this data using 104% 105% == 106% http_chunked_open(In, S, []), 107% read_stream_to_codes(S, Codes), 108% close(S). 109% == 110% 111% The current implementation does not generate chunked extensions 112% or an HTTP trailer. If such extensions appear on the input they 113% are silently ignored. This is compatible with the HTTP 1.1 114% specifications. Although a filtering stream is an excellent 115% mechanism for encoding and decoding the core chunked protocol, 116% it does not well support out-of-band data. 117% 118% After http_chunked_open/3, the encoding of DataStream is the 119% same as the encoding of RawStream, while the encoding of 120% RawStream is =octet=, the only value allowed for HTTP chunked 121% streams. Closing the DataStream restores the old encoding on 122% RawStream. 123% 124% @error io_error(read, Stream) where the message context provides 125% an indication of the problem. This error is raised if 126% the input is not valid HTTP chunked data. 127 128%% stream_range_open(+RawStream, -DataStream, +Options) is det. 129% 130% DataStream is a stream whose size is defined by the option 131% size(ContentLength). Closing DataStream does not close 132% RawStream. 133 134:- meta_predicate cgi_open(+, -, :, +). 135 136 137%% cgi_open(+OutStream, -CGIStream, :Hook, +Options) is det. 138% 139% Process CGI output. OutStream is normally the socket returning 140% data to the HTTP client. CGIStream is the stream the (Prolog) 141% code writes to. The CGIStream provides the following functions: 142% 143% * At the end of the header, it calls Hook using 144% call(Hook, header, Stream), where Stream is a stream holding 145% the buffered header. 146% 147% * If the stream is closed, it calls Hook using 148% call(Hook, data, Stream), where Stream holds the buffered 149% data. 150% 151% The stream calls Hook, adding the event and CGIStream to the 152% closure. Defined events are: 153% 154% * header 155% Called if the header is complete. Typically it uses 156% cgi_property/2 to extract the collected header and combines 157% these with the request and policies to decide on encoding, 158% transfer-encoding, connection parameters and the complete 159% header (as a Prolog term). Typically it uses cgi_set/2 to 160% associate these with the stream. 161% 162% * send_header 163% Called if the HTTP header must be sent. This is immediately 164% after setting the transfer encoding to =chunked= or when the 165% CGI stream is closed. Typically it requests the current 166% header, optionally the content-length and sends the header 167% to the original (client) stream. 168% 169% * close 170% Called from close/1 on the CGI stream after everything is 171% complete. 172% 173% The predicates cgi_property/2 and cgi_set/2 can be used to 174% control the stream and store status info. Terms are stored as 175% Prolog records and can thus be transferred between threads. 176 177%% cgi_property(+CGIStream, ?Property) is det. 178% 179% Inquire the status of the CGI stream. Defined properties are: 180% 181% * request(-Term) 182% The original request 183% * header(-Term) 184% Term is the header term as registered using cgi_set/2 185% * client(-Stream) 186% Stream is the original output stream used to create 187% this stream. 188% * thread(-ThreadID) 189% ThreadID is the identifier of the `owning thread' 190% * transfer_encoding(-Tranfer) 191% One of =chunked= or =none=. 192% * connection(-Connection) 193% One of =Keep-Alife= or =close= 194% * content_length(-ContentLength) 195% Total byte-size of the content. Available in the close 196% handler if the transfer_encoding is =none=. 197% * header_codes(-Codes) 198% Codes represents the header collected. Available in the 199% header handler. 200% * state(-State) 201% One of =header=, =data= or =discarded= 202 203%% cgi_set(+CGIStream, ?Property) is det. 204% 205% Change one of the properies. Supported properties are: 206% 207% * request(+Term) 208% Associate a request to the stream. 209% * header(+Term) 210% Register a reply header. This header is normally retrieved 211% from the =send_header= hook to send the reply header to the 212% client. 213% * connection(-Connection) 214% One of =Keep-Alife= or =close=. 215% * transfer_encoding(-Tranfer) 216% One of =chunked= or =none=. Initially set to =none=. When 217% switching to =chunked= from the =header= hook, it calls the 218% =send_header= hook and if there is data queed this is send 219% as first chunk. Each subsequent write to the CGI stream 220% emits a chunk. 221 222%% cgi_discard(+CGIStream) is det. 223% 224% Discard content produced sofar. It sets the state property to 225% =discarded=, causing close to omit the writing the data. This 226% must be to use an alternate output (e.g. an error page) if the 227% page generator fails. 228 229%% is_cgi_stream(+Stream) is semidet. 230% 231% True if Stream is a CGI stream created using cgi_open/4. 232 233:- multifile 234 http:encoding_filter/3. % +Encoding, +In0, -In 235:- multifile 236 http:current_transfer_encoding/1. % ?Encoding 237 238% http:encoding_filter(+Encoding, +In0, -In) is semidet. 239% 240% Install a filter to deal with =chunked= encoded messages. 241 242http:encoding_filter(chunked, In0, In) :- 243 http_chunked_open(In0, In, 244 [ close_parent(true) 245 ]). 246 247% http:current_transfer_encoding(?Encoding) is semidet. 248% 249% True if Encoding is supported 250 251http:current_transfer_encoding(chunked). 252 253:- retract(system:swi_io). 254