1.\" 2.\" Automated Testing Framework (atf) 3.\" 4.\" Copyright (c) 2007 The NetBSD Foundation, Inc. 5.\" All rights reserved. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions and the following disclaimer. 12.\" 2. Redistributions in binary form must reproduce the above copyright 13.\" notice, this list of conditions and the following disclaimer in the 14.\" documentation and/or other materials provided with the distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND 17.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, 18.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 19.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 20.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY 21.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE 23.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 24.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 25.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR 26.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 27.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28.\" 29.Dd December 20, 2011 30.Dt ATF-FORMATS 5 31.Os 32.Sh NAME 33.Nm atf-formats 34.Nd machine-parseable data formats used by ATF 35.Sh DESCRIPTION 36This manual page describes the multiple data formats used in ATF. 37These formats affect configuration files, control files and any data that 38is externalized or internalized by the tools. 39.Pp 40Data files are always organized as follows: 41.Bd -literal -offset indent 42Header1: Value1 \\ 43 ... | head 44HeaderN: ValueN / 45 mandatory blank line 46Free-form text contents \\ 47 ... | body 48 ... / 49.Ed 50.Pp 51A file must always contain a 52.Sq Content-Type 53header and must always separate that header from the body with a blank 54line, even if the body is empty. 55.Pp 56The 57.Sq Content-Type 58is always of the form: 59.Bd -literal -offset indent 60Content-Type: application/X-atf-<subtype>; version="<version>" 61.Ed 62.Pp 63where 64.Sq subtype 65indicates the specific file format and 66.Sq version 67its format version. 68This header must be the first one of the file. 69.Pp 70The main purpose of the 71.Sq Content-Type 72header, aside from determining the format used in the file, is to allow 73future changes to a given format. 74Whenever an incompatible change is made, the version is bumped by one. 75By keeping the header in the first line, future versions may even remove 76the need for such a header -- e.g. if some format was replaced by XML 77files, which have their own mandatory header. 78.Pp 79The rest of this document details the different format types. 80.Ss Format: application/X-atf-atffile, version: 1 81Atffiles are logically divided into three sections: 82.Bl -bullet 83.It 84Test programs: the list of test programs that define the test suite 85described by the Atffile. 86.It 87Meta-data properties: these define some constant values applicable to 88all the test programs defined in the file. 89In some sense they define the properties that describe the test suite. 90.It 91Configuration variables: defaults for configuration variables that 92can be overridden through configuration files or the command line. 93.El 94.Pp 95The grammar for Atffiles is the following: 96.Bd -literal -offset indent 97DATA ::= ( ( CONF | PROP | TP )? COMMENT? NEWLINE )* EOF 98CONF ::= 'conf:' WORD EQUAL STRING 99PROP ::= 'prop:' WORD EQUAL STRING 100TP ::= TPFILE | TPGLOB 101TPFILE ::= 'tp: ' STRING 102TPGLOB ::= 'tp-glob: ' STRING 103STRING ::= WORD | '"' WORD* '"' 104.Ed 105.Pp 106The meaning of the constructions above is: 107.Bl -tag -width TPGLOBXX 108.It CONF 109Definition of a configuration variable. 110.It PROP 111Definition of a meta-data property. 112.It TPFILE 113Addition of a test program into the test suite. 114The string is taken literally as the program's name, and this program 115must exist. 116.It TPGLOB 117Addition of multiple test programs into the test suite. 118The string is taken as a glob pattern, which may have or not have any 119matches in the current directory. 120.El 121.Pp 122An example: 123.Bd -literal -offset indent 124prop: test-suite = utilities 125 126conf: unprivileged-user = nobody 127 128tp: t_cp 129tp: t_mv 130tp: t_df 131tp-glob: t_dir_* 132.Ed 133.Ss Format: application/X-atf-config, version: 1 134Configuration files are very simple: they only contain a list of variable 135name/variable value pairs. 136Their grammar is: 137.Bd -literal -offset indent 138DATA ::= ( VAR? COMMENT? NEWLINE )* EOF 139VAR ::= WORD EQUAL STRING 140COMMENT ::= HASH WORD* 141STRING ::= WORD | '"' WORD* '"' 142.Ed 143.Pp 144An example: 145.Bd -literal -offset indent 146# This is the system-wide configuration file for ATF. 147# The above and this line are comments placed on their own line. 148 149var1 = this is a variable value 150var2 = this is another one # Optional comment at the end. 151.Ed 152.Ss Format: application/X-atf-tps, version: 3 153The 154.Sq application/X-atf-tps 155format multiplexes the standard output, standard error and results output 156streams from multiple test programs into a single data file. 157This format is used by 158.Xr atf-run 1 159to report the execution of several test programs and is later parsed by 160.Xr atf-report 1 161to inform the user of this process. 162It has the following grammar: 163.Bd -literal -offset indent 164DATA ::= INFO* TPS-COUNT TP-STANZA* INFO* EOF 165INFO ::= 'info' COLON STRING COMMA STRING NEWLINE 166TPS-COUNT ::= 'tps-count' COLON POSITIVE-NUMBER NEWLINE 167TP-STANZA ::= TP-START TC-STANZA* TC-END 168TP-START ::= 'tp-start' COLON TIMESTAMP COMMA STRING COMMA 169 POSITIVE-NUMBER NEWLINE 170TP-END ::= 'tc-end' COLON TIMESTAMP COMMA STRING (COMMA STRING)? 171TC-STANZA ::= TC-START (TC-SO | TC-SE)* TC-END 172TC-START ::= 'tc-start' COLON TIMESTAMP COMMA STRING NEWLINE 173TC-SO ::= 'tc-so' COLON STRING NEWLINE 174TC-SE ::= 'tc-se' COLON STRING NEWLINE 175TC-END ::= 'tc-end' COLON TIMESTAMP COMMA STRING COMMA TCR NEWLINE 176TCR ::= 'passed' | ('failed' | 'skipped') COMMA STRING 177TIMESTAMP ::= [0-9]+.[0-9]+ 178.Ed 179.Pp 180The meaning of the constructions above is: 181.Bl -tag -width TPSXCOUNTXX 182.It TPS-COUNT 183Indicates the number of test programs that will be executed. 184There will be this exact amount of 185.Sq TP-STANZA 186constructions following it. 187.It TP-START 188Indicates the beginning of a test program. 189This includes the program's name and the amount of test cases that 190will follow. 191.It TP-END 192Indicates the completion of a test program. 193This is followed by the program's name and, if the program ended 194prematurely, an error message indicating the reason of its failure. 195A successful execution of a test program (regardless of the status of its 196test cases) must not be accompanied by any reason. 197.It TC-START 198Indicates the beginning of a test case. 199This is accompanied by the test case's name. 200.It TC-SO 201Contains a text line sent to the standard output stream during the 202execution of the test case. 203Leading and trailing space is preserved. 204.It TC-SE 205Contains a text line sent to the standard error stream during the 206execution of the test case. 207Leading and trailing space is preserved. 208.It TC-END 209Indicates the completion of a test case. 210This is accompanied by the test case's name, its result and the reason 211associated with this result (if applicable). 212.El 213.Pp 214An example: 215.Bd -literal -offset indent 216tps-count: 2 217tp-start: calculator, 1324318951.838923, 2 218tc-start: add, 1324318951.839101 219tc-end: add, 1324318951.839500, passed 220tc-start: subtract, 1324318951.840001 221tc-so: 3-2 expected to return 1 but got 0 222tc-end: subtract, 1324318952.000123, failed, Calculated an unexpected value 223tp-end: calculator, 1324318952.002301 224tp-start: files, 1, 1324318952.502348 225tc-start: copy, 1324318952.508291 226tc-se: could not find the cp(1) utility 227tc-end: copy, 1324318953.203145, skipped 228tp-end: files, 1324318953.203800 229.Ed 230.Sh SEE ALSO 231.Xr atf 7 232