1.\" 2.\" Automated Testing Framework (atf) 3.\" 4.\" Copyright (c) 2007 The NetBSD Foundation, Inc. 5.\" All rights reserved. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions and the following disclaimer. 12.\" 2. Redistributions in binary form must reproduce the above copyright 13.\" notice, this list of conditions and the following disclaimer in the 14.\" documentation and/or other materials provided with the distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND 17.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, 18.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 19.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 20.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY 21.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE 23.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 24.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 25.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR 26.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 27.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28.\" 29.Dd August 28, 2010 30.Dt ATF 7 31.Os 32.Sh NAME 33.Nm ATF 34.Nd introduction to the Automated Testing Framework 35.Sh DESCRIPTION 36.Em IMPORTANT: If you are here because you want to know how to run the tests in 37.Em Pa __TESTSDIR__ , 38.Em you most likely want to read the 39.Em Xr tests 7 40.Em manual page instead. 41.Pp 42The Automated Testing Framework 43.Pf ( Nm ) 44is a collection of libraries and utilities designed to ease unattended 45application testing in the hands of developers and end users of a specific 46piece of software. 47.Pp 48As regards developers, 49.Nm 50provides the necessary means to easily create 51test suites composed of multiple test programs, which in turn are a 52collection of test cases. 53It also attempts to simplify the debugging of problems when these test 54cases detect an error by providing as much information as possible 55about the failure. 56.Pp 57As regards users, it simplifies the process of running the test suites and, 58in special, encourages end users to run them often: they do not need to 59have source trees around nor any other development tools installed to be 60able to certify that a given piece of software works on their machine as 61advertised. 62.Pp 63If your operating systems distributes 64.Nm , 65it is possible that it provides an introductory 66.Xr tests 7 67manual page. 68You are encouraged to read it now. 69.Ss License 70.Nm 71is distributed under the terms of the TNF License, a 2-clause BSD license. 72For more details please see: 73.Bd -literal -offset indent 74.Pa __DOCDIR__/COPYING 75.Ed 76.Ss Components 77.Nm 78is a highly modular piece of software. 79It provides a couple of libraries to ease the implementation of test 80programs: one for the C and C++ languages and another one for shell 81scripts. 82It also includes multiple small utilities that follow the principle of 83doing a single thing but doing it right. 84This section outlines which these components are. 85.Pp 86Public utilities: 87.Bl -tag -width atfXtestXprogramXXXXX 88.It Xr atf-check 1 89Executes a command and checks that its exit code, its standard output 90and its standard error output match pre-specified expected values. 91.It Xr atf-config 1 92Queries static configuration information. 93.It Xr atf-report 1 94Converts the output of 95.Nm atf-run 96to user-friendly and/or machine-parseable reports. 97.It Xr atf-run 1 98Automates the execution of a series of test programs and collects their 99results in a unified report. 100.It Xr atf-sh 1 101Shell interpreter for shell-based test programs. 102.El 103.Pp 104Programming interfaces: 105.Bl -tag -width atfXtestXprogramXXXXX 106.It Xr atf-c-api 3 107C programming interface for test programs. 108.It Xr atf-c++-api 3 109C++ programming interface for test programs. 110.It Xr atf-sh-api 3 111POSIX shell programming interface for test programs. 112.El 113.Pp 114Other: 115.Bl -tag -width atfXtestXprogramXXXXX 116.It Xr atf-formats 5 117Description of the machine-parseable data formats used by the tools. 118.It Xr atf-test-case 4 119Generic description of test cases, independent of the language they are 120implemented in. 121.It Xr atf-test-program 1 122Common interface provided by the test programs written using the 123.Nm 124libraries. 125.El 126.Ss Recommended reading order 127For end users wishing to run tests: 128.Bl -enum -compact 129.It 130.Xr tests 7 131(only if provided by your operating system). 132.It 133.Xr atf-test-program 1 134.It 135.Xr atf-run 1 136.It 137.Xr atf-report 1 138.It 139.Xr atf-config 1 140.El 141.Pp 142For developers wanting to write their own tests: 143.Bl -enum -compact 144.It 145Everything recommended to users. 146.It 147.Xr atf-test-case 4 148.It 149.Xr atf-c-api 3 150.It 151.Xr atf-c++-api 3 152.It 153.Xr atf-sh-api 3 154.It 155.Xr atf-sh 1 156.It 157.Xr atf-check 1 158.El 159.Pp 160For those interested in 161.Nm 162internals: 163.Bl -enum -compact 164.It 165Everything recommended to users. 166.It 167Everything recommended to developers. 168.It 169.Xr atf-formats 5 170.El 171.Sh SEE ALSO 172.Xr tests 7 173.Sh HISTORY 174.Nm 175started as a Google Summer of Code 2007 project mentored by The NetBSD 176Foundation. 177Its original goal was to provide a testing framework for The NetBSD 178Operating System, but it grew as an independent project because the 179framework itself did not need to be tied to a specific operating system. 180.Pp 181For more details on this subject, please see: 182.Bd -literal -offset indent 183.Pa __DOCDIR__/NEWS 184.Pa __DOCDIR__/ROADMAP 185.Ed 186.Sh AUTHORS 187For more details on the people that made 188.Nm 189possible, please see: 190.Bd -literal -offset indent 191.Pa __DOCDIR__/AUTHORS 192.Ed 193