1.\" $NetBSD: atf-formats.5,v 1.3 2014/12/10 04:38:04 christos Exp $ 2.\" 3.\" 4.\" Automated Testing Framework (atf) 5.\" 6.\" Copyright (c) 2007 The NetBSD Foundation, Inc. 7.\" All rights reserved. 8.\" 9.\" Redistribution and use in source and binary forms, with or without 10.\" modification, are permitted provided that the following conditions 11.\" are met: 12.\" 1. Redistributions of source code must retain the above copyright 13.\" notice, this list of conditions and the following disclaimer. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 18.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND 19.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, 20.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 21.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 22.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY 23.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 24.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE 25.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 26.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 27.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR 28.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 29.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30.\" 31.Dd December 20, 2011 32.Dt ATF-FORMATS 5 33.Os 34.Sh NAME 35.Nm atf-formats 36.Nd machine-parseable data formats used by ATF 37.Sh DESCRIPTION 38This manual page describes the multiple data formats used in ATF. 39These formats affect configuration files, control files and any data that 40is externalized or internalized by the tools. 41.Pp 42Data files are always organized as follows: 43.Bd -literal -offset indent 44Header1: Value1 \\ 45 ... | head 46HeaderN: ValueN / 47 mandatory blank line 48Free-form text contents \\ 49 ... | body 50 ... / 51.Ed 52.Pp 53A file must always contain a 54.Sq Content-Type 55header and must always separate that header from the body with a blank 56line, even if the body is empty. 57.Pp 58The 59.Sq Content-Type 60is always of the form: 61.Bd -literal -offset indent 62Content-Type: application/X-atf-<subtype>; version="<version>" 63.Ed 64.Pp 65where 66.Sq subtype 67indicates the specific file format and 68.Sq version 69its format version. 70This header must be the first one of the file. 71.Pp 72The main purpose of the 73.Sq Content-Type 74header, aside from determining the format used in the file, is to allow 75future changes to a given format. 76Whenever an incompatible change is made, the version is bumped by one. 77By keeping the header in the first line, future versions may even remove 78the need for such a header -- e.g. if some format was replaced by XML 79files, which have their own mandatory header. 80.Pp 81The rest of this document details the different format types. 82.Ss Format: application/X-atf-atffile, version: 1 83Atffiles are logically divided into three sections: 84.Bl -bullet 85.It 86Test programs: the list of test programs that define the test suite 87described by the Atffile. 88.It 89Meta-data properties: these define some constant values applicable to 90all the test programs defined in the file. 91In some sense they define the properties that describe the test suite. 92.It 93Configuration variables: defaults for configuration variables that 94can be overridden through configuration files or the command line. 95.El 96.Pp 97The grammar for Atffiles is the following: 98.Bd -literal -offset indent 99DATA ::= ( ( CONF | PROP | TP )? COMMENT? NEWLINE )* EOF 100CONF ::= 'conf:' WORD EQUAL STRING 101PROP ::= 'prop:' WORD EQUAL STRING 102TP ::= TPFILE | TPGLOB 103TPFILE ::= 'tp: ' STRING 104TPGLOB ::= 'tp-glob: ' STRING 105STRING ::= WORD | '"' WORD* '"' 106.Ed 107.Pp 108The meaning of the constructions above is: 109.Bl -tag -width TPGLOBXX 110.It CONF 111Definition of a configuration variable. 112.It PROP 113Definition of a meta-data property. 114.It TPFILE 115Addition of a test program into the test suite. 116The string is taken literally as the program's name, and this program 117must exist. 118.It TPGLOB 119Addition of multiple test programs into the test suite. 120The string is taken as a glob pattern, which may have or not have any 121matches in the current directory. 122.El 123.Pp 124An example: 125.Bd -literal -offset indent 126prop: test-suite = utilities 127 128conf: unprivileged-user = nobody 129 130tp: t_cp 131tp: t_mv 132tp: t_df 133tp-glob: t_dir_* 134.Ed 135.Ss Format: application/X-atf-config, version: 1 136Configuration files are very simple: they only contain a list of variable 137name/variable value pairs. 138Their grammar is: 139.Bd -literal -offset indent 140DATA ::= ( VAR? COMMENT? NEWLINE )* EOF 141VAR ::= WORD EQUAL STRING 142COMMENT ::= HASH WORD* 143STRING ::= WORD | '"' WORD* '"' 144.Ed 145.Pp 146An example: 147.Bd -literal -offset indent 148# This is the system-wide configuration file for ATF. 149# The above and this line are comments placed on their own line. 150 151var1 = this is a variable value 152var2 = this is another one # Optional comment at the end. 153.Ed 154.Ss Format: application/X-atf-tps, version: 3 155The 156.Sq application/X-atf-tps 157format multiplexes the standard output, standard error and results output 158streams from multiple test programs into a single data file. 159This format is used by 160.Xr atf-run 1 161to report the execution of several test programs and is later parsed by 162.Xr atf-report 1 163to inform the user of this process. 164It has the following grammar: 165.Bd -literal -offset indent 166DATA ::= INFO* TPS-COUNT TP-STANZA* INFO* EOF 167INFO ::= 'info' COLON STRING COMMA STRING NEWLINE 168TPS-COUNT ::= 'tps-count' COLON POSITIVE-NUMBER NEWLINE 169TP-STANZA ::= TP-START TC-STANZA* TC-END 170TP-START ::= 'tp-start' COLON TIMESTAMP COMMA STRING COMMA 171 POSITIVE-NUMBER NEWLINE 172TP-END ::= 'tc-end' COLON TIMESTAMP COMMA STRING (COMMA STRING)? 173TC-STANZA ::= TC-START (TC-SO | TC-SE)* TC-END 174TC-START ::= 'tc-start' COLON TIMESTAMP COMMA STRING NEWLINE 175TC-SO ::= 'tc-so' COLON STRING NEWLINE 176TC-SE ::= 'tc-se' COLON STRING NEWLINE 177TC-END ::= 'tc-end' COLON TIMESTAMP COMMA STRING COMMA TCR NEWLINE 178TCR ::= 'passed' | ('failed' | 'skipped') COMMA STRING 179TIMESTAMP ::= [0-9]+.[0-9]+ 180.Ed 181.Pp 182The meaning of the constructions above is: 183.Bl -tag -width TPSXCOUNTXX 184.It TPS-COUNT 185Indicates the number of test programs that will be executed. 186There will be this exact amount of 187.Sq TP-STANZA 188constructions following it. 189.It TP-START 190Indicates the beginning of a test program. 191This includes the program's name and the amount of test cases that 192will follow. 193.It TP-END 194Indicates the completion of a test program. 195This is followed by the program's name and, if the program ended 196prematurely, an error message indicating the reason of its failure. 197A successful execution of a test program (regardless of the status of its 198test cases) must not be accompanied by any reason. 199.It TC-START 200Indicates the beginning of a test case. 201This is accompanied by the test case's name. 202.It TC-SO 203Contains a text line sent to the standard output stream during the 204execution of the test case. 205Leading and trailing space is preserved. 206.It TC-SE 207Contains a text line sent to the standard error stream during the 208execution of the test case. 209Leading and trailing space is preserved. 210.It TC-END 211Indicates the completion of a test case. 212This is accompanied by the test case's name, its result and the reason 213associated with this result (if applicable). 214.El 215.Pp 216An example: 217.Bd -literal -offset indent 218tps-count: 2 219tp-start: calculator, 1324318951.838923, 2 220tc-start: add, 1324318951.839101 221tc-end: add, 1324318951.839500, passed 222tc-start: subtract, 1324318951.840001 223tc-so: 3-2 expected to return 1 but got 0 224tc-end: subtract, 1324318952.000123, failed, Calculated an unexpected value 225tp-end: calculator, 1324318952.002301 226tp-start: files, 1, 1324318952.502348 227tc-start: copy, 1324318952.508291 228tc-se: could not find the cp(1) utility 229tc-end: copy, 1324318953.203145, skipped 230tp-end: files, 1324318953.203800 231.Ed 232.Sh SEE ALSO 233.Xr atf 7 234