1.\" 2.\" Automated Testing Framework (atf) 3.\" 4.\" Copyright (c) 2007 The NetBSD Foundation, Inc. 5.\" All rights reserved. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions and the following disclaimer. 12.\" 2. Redistributions in binary form must reproduce the above copyright 13.\" notice, this list of conditions and the following disclaimer in the 14.\" documentation and/or other materials provided with the distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND 17.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, 18.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 19.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 20.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY 21.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE 23.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 24.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 25.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR 26.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 27.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28.\" 29.Dd December 16, 2011 30.Dt ATF-REPORT 1 31.Os 32.Sh NAME 33.Nm atf-report 34.Nd transforms the output of atf-run to different formats 35.Sh SYNOPSIS 36.Nm 37.Op Fl o Ar fmt1:path1 Op .. Fl o Ar fmtN:pathN 38.Nm 39.Fl h 40.Sh DESCRIPTION 41.Nm 42reads the output of 43.Nm atf-run 44and transforms it to different formats. 45Some of these are user-friendly and others are machine-parseable, which 46opens a wide range of possibilities to analyze the results of a test 47suite's execution. 48See 49.Sx Output formats 50below for more details on which these formats are. 51.Pp 52In the first synopsis form, 53.Nm 54reads the output of 55.Nm atf-run 56through its standard input and, if no 57.Fl o 58options are given, prints a user-friendly report on its standard 59output using the 60.Sq ticker 61format. 62If 63.Fl o 64options are provided (more than one are allowed), they specify the complete 65list of reports to generate. 66They are all generated simultaneously, and for obvious reasons, two reports 67cannot be written to the same file. 68Note that the default output is suppressed when 69.Fl o 70is provided. 71.Pp 72In the second synopsis form, 73.Nm 74will print information about all supported options and their purpose. 75.Pp 76The following options are available: 77.Bl -tag -width XoXfmtXpathXX 78.It Fl h 79Shows a short summary of all available options and their purpose. 80.It Fl o Ar fmt:path 81Adds a new output format. 82.Ar fmt 83is one of the formats described later on in 84.Sx Output formats . 85.Ar path 86specifies where the report will be written to. 87Depending on the chosen format, this may refer to a single file or to 88a directory. 89For those formats that write to a single file, specifying a 90.Sq - 91as the path will redirect the report to the standard output. 92.El 93.Ss Output formats 94The following output formats are allowed: 95.Bl -tag -width tickerXX 96.It csv 97A machine-parseable Comma-Separated Values (CSV) file. 98This file contains the results for all test cases and test programs. 99Test cases are logged using the following syntax: 100.Bd -literal -offset indent 101tc, duration, test-program, test-case, result[, reason] 102.Ed 103.Pp 104The 105.Sq result 106field for test cases is always one of 107.Sq passed , 108.Sq skipped 109or 110.Sq failed . 111The last two are always followed by a reason. 112.Pp 113Test programs are logged with the following syntax: 114.Bd -literal -offset indent 115tp, duration, test-program, result[, reason] 116.Ed 117.Pp 118In this case, the 119.Sq result 120can be one of: 121.Sq passed , 122which denotes test programs that ran without any failure; 123.Sq failed , 124which refers to test programs in which one or more test cases failed; 125or 126.Sq bogus , 127which mentions those test programs that failed to execute by some reason. 128The reason field is only available in the last case. 129.Pp 130The time required to execute each test case and test program is 131also provided. 132You should not rely on the order of the entries in the resulting output. 133.It ticker 134A user-friendly report that shows the progress of the test suite's 135execution as it operates. 136This type of report should always be redirected to a virtual terminal, 137not a file, as it may use control sequences that will make the output 138unreadable in regular files. 139.It xml 140A report contained in a single XML file. 141Ideal for later processing with 142.Xr xsltproc 1 143to generate nice HTML reports. 144.El 145.Sh EXAMPLES 146The most simple way of running a test suite is to pipe the output of 147.Nm atf-run 148through 149.Nm 150without any additional flags. 151This will use the default output format, which is suitable to most users: 152.Bd -literal -offset indent 153atf-run | atf-report 154.Ed 155.Pp 156In some situations, it may be interesting to get a machine-parseable file 157aside from the standard report. 158This can be done as follows: 159.Bd -literal -offset indent 160atf-run | atf-report -o csv:testsuite.csv -o ticker:- 161.Ed 162.Pp 163Or if the standard report is not desired, thus achieving completely silent 164operation: 165atf-run | atf-report -o csv:testsuite.csv 166.Sh SEE ALSO 167.Xr atf-run 1 , 168.Xr atf 7 169