1eda14cbcSMatt Macy# ZFS Test Suite README 2eda14cbcSMatt Macy 3*681ce946SMartin Matuska### 1) Building and installing the ZFS Test Suite 4eda14cbcSMatt Macy 5eda14cbcSMatt MacyThe ZFS Test Suite runs under the test-runner framework. This framework 6eda14cbcSMatt Macyis built along side the standard ZFS utilities and is included as part of 7eda14cbcSMatt Macyzfs-test package. The zfs-test package can be built from source as follows: 8eda14cbcSMatt Macy 9eda14cbcSMatt Macy $ ./configure 10eda14cbcSMatt Macy $ make pkg-utils 11eda14cbcSMatt Macy 12eda14cbcSMatt MacyThe resulting packages can be installed using the rpm or dpkg command as 13eda14cbcSMatt Macyappropriate for your distributions. Alternately, if you have installed 14eda14cbcSMatt MacyZFS from a distributions repository (not from source) the zfs-test package 15eda14cbcSMatt Macymay be provided for your distribution. 16eda14cbcSMatt Macy 17eda14cbcSMatt Macy - Installed from source 18eda14cbcSMatt Macy $ rpm -ivh ./zfs-test*.rpm, or 19eda14cbcSMatt Macy $ dpkg -i ./zfs-test*.deb, 20eda14cbcSMatt Macy 21eda14cbcSMatt Macy - Installed from package repository 22eda14cbcSMatt Macy $ yum install zfs-test 23eda14cbcSMatt Macy $ apt-get install zfs-test 24eda14cbcSMatt Macy 25*681ce946SMartin Matuska### 2) Running the ZFS Test Suite 26eda14cbcSMatt Macy 27eda14cbcSMatt MacyThe pre-requisites for running the ZFS Test Suite are: 28eda14cbcSMatt Macy 29eda14cbcSMatt Macy * Three scratch disks 30eda14cbcSMatt Macy * Specify the disks you wish to use in the $DISKS variable, as a 31eda14cbcSMatt Macy space delimited list like this: DISKS='vdb vdc vdd'. By default 32eda14cbcSMatt Macy the zfs-tests.sh script will construct three loopback devices to 33eda14cbcSMatt Macy be used for testing: DISKS='loop0 loop1 loop2'. 34eda14cbcSMatt Macy * A non-root user with a full set of basic privileges and the ability 35eda14cbcSMatt Macy to sudo(8) to root without a password to run the test. 36eda14cbcSMatt Macy * Specify any pools you wish to preserve as a space delimited list in 37eda14cbcSMatt Macy the $KEEP variable. All pools detected at the start of testing are 38eda14cbcSMatt Macy added automatically. 39eda14cbcSMatt Macy * The ZFS Test Suite will add users and groups to test machine to 40eda14cbcSMatt Macy verify functionality. Therefore it is strongly advised that a 41eda14cbcSMatt Macy dedicated test machine, which can be a VM, be used for testing. 42eda14cbcSMatt Macy 43eda14cbcSMatt MacyOnce the pre-requisites are satisfied simply run the zfs-tests.sh script: 44eda14cbcSMatt Macy 45eda14cbcSMatt Macy $ /usr/share/zfs/zfs-tests.sh 46eda14cbcSMatt Macy 47eda14cbcSMatt MacyAlternately, the zfs-tests.sh script can be run from the source tree to allow 48eda14cbcSMatt Macydevelopers to rapidly validate their work. In this mode the ZFS utilities and 49eda14cbcSMatt Macymodules from the source tree will be used (rather than those installed on the 50eda14cbcSMatt Macysystem). In order to avoid certain types of failures you will need to ensure 51eda14cbcSMatt Macythe ZFS udev rules are installed. This can be done manually or by ensuring 52eda14cbcSMatt Macysome version of ZFS is installed on the system. 53eda14cbcSMatt Macy 54eda14cbcSMatt Macy $ ./scripts/zfs-tests.sh 55eda14cbcSMatt Macy 56eda14cbcSMatt MacyThe following zfs-tests.sh options are supported: 57eda14cbcSMatt Macy 58eda14cbcSMatt Macy -v Verbose zfs-tests.sh output When specified additional 59eda14cbcSMatt Macy information describing the test environment will be logged 60eda14cbcSMatt Macy prior to invoking test-runner. This includes the runfile 61eda14cbcSMatt Macy being used, the DISKS targeted, pools to keep, etc. 62eda14cbcSMatt Macy 63eda14cbcSMatt Macy -q Quiet test-runner output. When specified it is passed to 64eda14cbcSMatt Macy test-runner(1) which causes output to be written to the 65eda14cbcSMatt Macy console only for tests that do not pass and the results 66eda14cbcSMatt Macy summary. 67eda14cbcSMatt Macy 68eda14cbcSMatt Macy -x Remove all testpools, dm, lo, and files (unsafe). When 69eda14cbcSMatt Macy specified the script will attempt to remove any leftover 70eda14cbcSMatt Macy configuration from a previous test run. This includes 71eda14cbcSMatt Macy destroying any pools named testpool, unused DM devices, 72eda14cbcSMatt Macy and loopback devices backed by file-vdevs. This operation 73eda14cbcSMatt Macy can be DANGEROUS because it is possible that the script 74eda14cbcSMatt Macy will mistakenly remove a resource not related to the testing. 75eda14cbcSMatt Macy 76eda14cbcSMatt Macy -k Disable cleanup after test failure. When specified the 77eda14cbcSMatt Macy zfs-tests.sh script will not perform any additional cleanup 78eda14cbcSMatt Macy when test-runner exists. This is useful when the results of 79eda14cbcSMatt Macy a specific test need to be preserved for further analysis. 80eda14cbcSMatt Macy 81eda14cbcSMatt Macy -f Use sparse files directly instead of loopback devices for 82eda14cbcSMatt Macy the testing. When running in this mode certain tests will 83eda14cbcSMatt Macy be skipped which depend on real block devices. 84eda14cbcSMatt Macy 85eda14cbcSMatt Macy -c Only create and populate constrained path 86eda14cbcSMatt Macy 87eda14cbcSMatt Macy -I NUM Number of iterations 88eda14cbcSMatt Macy 89eda14cbcSMatt Macy -d DIR Create sparse files for vdevs in the DIR directory. By 90eda14cbcSMatt Macy default these files are created under /var/tmp/. 91eda14cbcSMatt Macy 92eda14cbcSMatt Macy -s SIZE Use vdevs of SIZE (default: 4G) 93eda14cbcSMatt Macy 94eda14cbcSMatt Macy -r RUNFILES Run tests in RUNFILES (default: common.run,linux.run) 95eda14cbcSMatt Macy 96eda14cbcSMatt Macy -t PATH Run single test at PATH relative to test suite 97eda14cbcSMatt Macy 98eda14cbcSMatt Macy -T TAGS Comma separated list of tags (default: 'functional') 99eda14cbcSMatt Macy 100eda14cbcSMatt Macy -u USER Run single test as USER (default: root) 101eda14cbcSMatt Macy 102eda14cbcSMatt Macy 103eda14cbcSMatt MacyThe ZFS Test Suite allows the user to specify a subset of the tests via a 104eda14cbcSMatt Macyrunfile or list of tags. 105eda14cbcSMatt Macy 106eda14cbcSMatt MacyThe format of the runfile is explained in test-runner(1), and 107eda14cbcSMatt Macythe files that zfs-tests.sh uses are available for reference under 108eda14cbcSMatt Macy/usr/share/zfs/runfiles. To specify a custom runfile, use the -r option: 109eda14cbcSMatt Macy 110eda14cbcSMatt Macy $ /usr/share/zfs/zfs-tests.sh -r my_tests.run 111eda14cbcSMatt Macy 112eda14cbcSMatt MacyOtherwise user can set needed tags to run only specific tests. 113eda14cbcSMatt Macy 114*681ce946SMartin Matuska### 3) Test results 115eda14cbcSMatt Macy 116eda14cbcSMatt MacyWhile the ZFS Test Suite is running, one informational line is printed at the 117eda14cbcSMatt Macyend of each test, and a results summary is printed at the end of the run. The 118eda14cbcSMatt Macyresults summary includes the location of the complete logs, which is logged in 119eda14cbcSMatt Macythe form `/var/tmp/test_results/[ISO 8601 date]`. A normal test run launched 120eda14cbcSMatt Macywith the `zfs-tests.sh` wrapper script will look something like this: 121eda14cbcSMatt Macy 122eda14cbcSMatt Macy $ /usr/share/zfs/zfs-tests.sh -v -d /tmp/test 123eda14cbcSMatt Macy 124eda14cbcSMatt Macy --- Configuration --- 125eda14cbcSMatt Macy Runfile: /usr/share/zfs/runfiles/linux.run 126eda14cbcSMatt Macy STF_TOOLS: /usr/share/zfs/test-runner 127eda14cbcSMatt Macy STF_SUITE: /usr/share/zfs/zfs-tests 128eda14cbcSMatt Macy STF_PATH: /var/tmp/constrained_path.G0Sf 129eda14cbcSMatt Macy FILEDIR: /tmp/test 130eda14cbcSMatt Macy FILES: /tmp/test/file-vdev0 /tmp/test/file-vdev1 /tmp/test/file-vdev2 131eda14cbcSMatt Macy LOOPBACKS: /dev/loop0 /dev/loop1 /dev/loop2 132eda14cbcSMatt Macy DISKS: loop0 loop1 loop2 133eda14cbcSMatt Macy NUM_DISKS: 3 134eda14cbcSMatt Macy FILESIZE: 4G 135eda14cbcSMatt Macy ITERATIONS: 1 136eda14cbcSMatt Macy TAGS: functional 137eda14cbcSMatt Macy Keep pool(s): rpool 138eda14cbcSMatt Macy 139eda14cbcSMatt Macy 140eda14cbcSMatt Macy /usr/share/zfs/test-runner/bin/test-runner.py -c /usr/share/zfs/runfiles/linux.run \ 141eda14cbcSMatt Macy -T functional -i /usr/share/zfs/zfs-tests -I 1 142eda14cbcSMatt Macy Test: /usr/share/zfs/zfs-tests/tests/functional/arc/setup (run as root) [00:00] [PASS] 143eda14cbcSMatt Macy ...more than 1100 additional tests... 144eda14cbcSMatt Macy Test: /usr/share/zfs/zfs-tests/tests/functional/zvol/zvol_swap/cleanup (run as root) [00:00] [PASS] 145eda14cbcSMatt Macy 146eda14cbcSMatt Macy Results Summary 147eda14cbcSMatt Macy SKIP 52 148eda14cbcSMatt Macy PASS 1129 149eda14cbcSMatt Macy 150eda14cbcSMatt Macy Running Time: 02:35:33 151eda14cbcSMatt Macy Percent passed: 95.6% 152eda14cbcSMatt Macy Log directory: /var/tmp/test_results/20180515T054509 153*681ce946SMartin Matuska 154*681ce946SMartin Matuska### 4) Example of adding and running test-case (zpool_example) 155*681ce946SMartin Matuska 156*681ce946SMartin Matuska This broadly boils down to 5 steps 157*681ce946SMartin Matuska 1. Create/Set password-less sudo for user running test case. 158*681ce946SMartin Matuska 2. Edit configure.ac, Makefile.am appropriately 159*681ce946SMartin Matuska 3. Create/Modify .run files 160*681ce946SMartin Matuska 4. Create actual test-scripts 161*681ce946SMartin Matuska 5. Run Test case 162*681ce946SMartin Matuska 163*681ce946SMartin Matuska Will look at each of them in depth. 164*681ce946SMartin Matuska 165*681ce946SMartin Matuska * Set password-less sudo for 'Test' user as test script cannot be run as root 166*681ce946SMartin Matuska * Edit file **configure.ac** and include line under AC_CONFIG_FILES section 167*681ce946SMartin Matuska ~~~~ 168*681ce946SMartin Matuska tests/zfs-tests/tests/functional/cli_root/zpool_example/Makefile 169*681ce946SMartin Matuska ~~~~ 170*681ce946SMartin Matuska * Edit file **tests/runfiles/Makefile.am** and add line *zpool_example*. 171*681ce946SMartin Matuska ~~~~ 172*681ce946SMartin Matuska pkgdatadir = $(datadir)/@PACKAGE@/runfiles 173*681ce946SMartin Matuska dist_pkgdata_DATA = \ 174*681ce946SMartin Matuska zpool_example.run \ 175*681ce946SMartin Matuska common.run \ 176*681ce946SMartin Matuska freebsd.run \ 177*681ce946SMartin Matuska linux.run \ 178*681ce946SMartin Matuska longevity.run \ 179*681ce946SMartin Matuska perf-regression.run \ 180*681ce946SMartin Matuska sanity.run \ 181*681ce946SMartin Matuska sunos.run 182*681ce946SMartin Matuska ~~~~ 183*681ce946SMartin Matuska * Create file **tests/runfiles/zpool_example.run**. This defines the most 184*681ce946SMartin Matuska common properties when run with test-runner.py or zfs-tests.sh. 185*681ce946SMartin Matuska ~~~~ 186*681ce946SMartin Matuska [DEFAULT] 187*681ce946SMartin Matuska timeout = 600 188*681ce946SMartin Matuska outputdir = /var/tmp/test_results 189*681ce946SMartin Matuska tags = ['functional'] 190*681ce946SMartin Matuska 191*681ce946SMartin Matuska tests = ['zpool_example_001_pos'] 192*681ce946SMartin Matuska ~~~~ 193*681ce946SMartin Matuska If adding test-case to an already existing suite the runfile would 194*681ce946SMartin Matuska already be present and it needs to be only updated. For example, adding 195*681ce946SMartin Matuska **zpool_example_002_pos** to the above runfile only update the **"tests ="** 196*681ce946SMartin Matuska section of the runfile as shown below 197*681ce946SMartin Matuska ~~~~ 198*681ce946SMartin Matuska [DEFAULT] 199*681ce946SMartin Matuska timeout = 600 200*681ce946SMartin Matuska outputdir = /var/tmp/test_results 201*681ce946SMartin Matuska tags = ['functional'] 202*681ce946SMartin Matuska 203*681ce946SMartin Matuska tests = ['zpool_example_001_pos', 'zpool_example_002_pos'] 204*681ce946SMartin Matuska ~~~~ 205*681ce946SMartin Matuska 206*681ce946SMartin Matuska * Edit **tests/zfs-tests/tests/functional/cli_root/Makefile.am** and add line 207*681ce946SMartin Matuska under SUBDIRS. 208*681ce946SMartin Matuska ~~~~ 209*681ce946SMartin Matuska zpool_example \ (Make sure to escape the line end as there will be other folders names following) 210*681ce946SMartin Matuska ~~~~ 211*681ce946SMartin Matuska * Create new file **tests/zfs-tests/tests/functional/cli_root/zpool_example/Makefile.am** 212*681ce946SMartin Matuska the contents of the file could be as below. What it says it that now we have 213*681ce946SMartin Matuska a test case *zpool_example_001_pos.ksh* 214*681ce946SMartin Matuska ~~~~ 215*681ce946SMartin Matuska pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zpool_example 216*681ce946SMartin Matuska dist_pkgdata_SCRIPTS = \ 217*681ce946SMartin Matuska zpool_example_001_pos.ksh 218*681ce946SMartin Matuska ~~~~ 219*681ce946SMartin Matuska * We can now create our test-case zpool_example_001_pos.ksh under 220*681ce946SMartin Matuska **tests/zfs-tests/tests/functional/cli_root/zpool_example/**. 221*681ce946SMartin Matuska ~~~~ 222*681ce946SMartin Matuska # DESCRIPTION: 223*681ce946SMartin Matuska # zpool_example Test 224*681ce946SMartin Matuska # 225*681ce946SMartin Matuska # STRATEGY: 226*681ce946SMartin Matuska # 1. Demo a very basic test case 227*681ce946SMartin Matuska # 228*681ce946SMartin Matuska 229*681ce946SMartin Matuska DISKS_DEV1="/dev/loop0" 230*681ce946SMartin Matuska DISKS_DEV2="/dev/loop1" 231*681ce946SMartin Matuska TESTPOOL=EXAMPLE_POOL 232*681ce946SMartin Matuska 233*681ce946SMartin Matuska function cleanup 234*681ce946SMartin Matuska { 235*681ce946SMartin Matuska # Cleanup 236*681ce946SMartin Matuska destroy_pool $TESTPOOL 237*681ce946SMartin Matuska log_must rm -f $DISKS_DEV1 238*681ce946SMartin Matuska log_must rm -f $DISKS_DEV2 239*681ce946SMartin Matuska } 240*681ce946SMartin Matuska 241*681ce946SMartin Matuska log_assert "zpool_example" 242*681ce946SMartin Matuska # Run function "cleanup" on exit 243*681ce946SMartin Matuska log_onexit cleanup 244*681ce946SMartin Matuska 245*681ce946SMartin Matuska # Prep backend device 246*681ce946SMartin Matuska log_must dd if=/dev/zero of=$DISKS_DEV1 bs=512 count=140000 247*681ce946SMartin Matuska log_must dd if=/dev/zero of=$DISKS_DEV2 bs=512 count=140000 248*681ce946SMartin Matuska 249*681ce946SMartin Matuska # Create pool 250*681ce946SMartin Matuska log_must zpool create $TESTPOOL $type $DISKS_DEV1 $DISKS_DEV2 251*681ce946SMartin Matuska 252*681ce946SMartin Matuska log_pass "zpool_example" 253*681ce946SMartin Matuska ~~~~ 254*681ce946SMartin Matuska * Run Test case, which can be done in two ways. Described in detail above in 255*681ce946SMartin Matuska section 2. 256*681ce946SMartin Matuska * test-runner.py (This takes run file as input. See *zpool_example.run*) 257*681ce946SMartin Matuska * zfs-tests.sh. Can execute the run file or individual tests 258