1.\" $NetBSD: atf-report.1,v 1.3 2014/12/10 04:38:03 christos Exp $ 2.\" 3.\" 4.\" Automated Testing Framework (atf) 5.\" 6.\" Copyright (c) 2007 The NetBSD Foundation, Inc. 7.\" All rights reserved. 8.\" 9.\" Redistribution and use in source and binary forms, with or without 10.\" modification, are permitted provided that the following conditions 11.\" are met: 12.\" 1. Redistributions of source code must retain the above copyright 13.\" notice, this list of conditions and the following disclaimer. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 18.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND 19.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, 20.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 21.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 22.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY 23.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 24.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE 25.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 26.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 27.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR 28.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 29.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30.\" 31.Dd December 16, 2011 32.Dt ATF-REPORT 1 33.Os 34.Sh NAME 35.Nm atf-report 36.Nd transforms the output of atf-run to different formats 37.Sh SYNOPSIS 38.Nm 39.Op Fl o Ar fmt1:path1 Op .. Fl o Ar fmtN:pathN 40.Nm 41.Fl h 42.Sh DESCRIPTION 43.Nm 44reads the output of 45.Nm atf-run 46and transforms it to different formats. 47Some of these are user-friendly and others are machine-parseable, which 48opens a wide range of possibilities to analyze the results of a test 49suite's execution. 50See 51.Sx Output formats 52below for more details on which these formats are. 53.Pp 54In the first synopsis form, 55.Nm 56reads the output of 57.Nm atf-run 58through its standard input and, if no 59.Fl o 60options are given, prints a user-friendly report on its standard 61output using the 62.Sq ticker 63format. 64If 65.Fl o 66options are provided (more than one are allowed), they specify the complete 67list of reports to generate. 68They are all generated simultaneously, and for obvious reasons, two reports 69cannot be written to the same file. 70Note that the default output is suppressed when 71.Fl o 72is provided. 73.Pp 74In the second synopsis form, 75.Nm 76will print information about all supported options and their purpose. 77.Pp 78The following options are available: 79.Bl -tag -width XoXfmtXpathXX 80.It Fl h 81Shows a short summary of all available options and their purpose. 82.It Fl o Ar fmt:path 83Adds a new output format. 84.Ar fmt 85is one of the formats described later on in 86.Sx Output formats . 87.Ar path 88specifies where the report will be written to. 89Depending on the chosen format, this may refer to a single file or to 90a directory. 91For those formats that write to a single file, specifying a 92.Sq - 93as the path will redirect the report to the standard output. 94.El 95.Ss Output formats 96The following output formats are allowed: 97.Bl -tag -width tickerXX 98.It csv 99A machine-parseable Comma-Separated Values (CSV) file. 100This file contains the results for all test cases and test programs. 101Test cases are logged using the following syntax: 102.Bd -literal -offset indent 103tc, duration, test-program, test-case, result[, reason] 104.Ed 105.Pp 106The 107.Sq result 108field for test cases is always one of 109.Sq passed , 110.Sq skipped 111or 112.Sq failed . 113The last two are always followed by a reason. 114.Pp 115Test programs are logged with the following syntax: 116.Bd -literal -offset indent 117tp, duration, test-program, result[, reason] 118.Ed 119.Pp 120In this case, the 121.Sq result 122can be one of: 123.Sq passed , 124which denotes test programs that ran without any failure; 125.Sq failed , 126which refers to test programs in which one or more test cases failed; 127or 128.Sq bogus , 129which mentions those test programs that failed to execute by some reason. 130The reason field is only available in the last case. 131.Pp 132The time required to execute each test case and test program is 133also provided. 134You should not rely on the order of the entries in the resulting output. 135.It ticker 136A user-friendly report that shows the progress of the test suite's 137execution as it operates. 138This type of report should always be redirected to a virtual terminal, 139not a file, as it may use control sequences that will make the output 140unreadable in regular files. 141.It xml 142A report contained in a single XML file. 143Ideal for later processing with 144.Xr xsltproc 1 145to generate nice HTML reports. 146.El 147.Sh EXAMPLES 148The most simple way of running a test suite is to pipe the output of 149.Nm atf-run 150through 151.Nm 152without any additional flags. 153This will use the default output format, which is suitable to most users: 154.Bd -literal -offset indent 155atf-run | atf-report 156.Ed 157.Pp 158In some situations, it may be interesting to get a machine-parseable file 159aside from the standard report. 160This can be done as follows: 161.Bd -literal -offset indent 162atf-run | atf-report -o csv:testsuite.csv -o ticker:- 163.Ed 164.Pp 165Or if the standard report is not desired, thus achieving completely silent 166operation: 167atf-run | atf-report -o csv:testsuite.csv 168.Sh SEE ALSO 169.Xr atf-run 1 , 170.Xr atf 7 171