1============================================================================= 2ACPI/SMBIOS avocado tests using biosbits 3============================================================================= 4 5Biosbits is a software written by Josh Triplett that can be downloaded 6from https://biosbits.org/. The github codebase can be found 7`here <https://github.com/biosbits/bits/tree/master>`__. It is a software that executes 8the bios components such as acpi and smbios tables directly through acpica 9bios interpreter (a freely available C based library written by Intel, 10downloadable from https://acpica.org/ and is included with biosbits) without an 11operating system getting involved in between. 12There are several advantages to directly testing the bios in a real physical 13machine or VM as opposed to indirectly discovering bios issues through the 14operating system. For one thing, the OSes tend to hide bios problems from the 15end user. The other is that we have more control of what we wanted to test 16and how by directly using acpica interpreter on top of the bios on a running 17system. More details on the inspiration for developing biosbits and its real 18life uses can be found in [#a]_ and [#b]_. 19This directory contains tests written in python using avocado framework that 20exercises the QEMU bios components using biosbits and reports test failures. 21For QEMU, we maintain a fork of bios bits in gitlab along with all the 22dependent submodules: 23https://gitlab.com/qemu-project/biosbits-bits 24This fork contains numerous fixes, a newer acpica and changes specific to 25running this avocado QEMU tests using bits. The author of this document 26is the sole maintainer of the QEMU fork of bios bits repo. 27 28Under the directory ``tests/avocado/``, ``acpi-bits.py`` is a QEMU avocado 29test that drives all this. 30 31A brief description of the various test files follows. 32 33Under ``tests/avocado/`` as the root we have: 34 35:: 36 37 ├── acpi-bits 38 │ ├── bits-config 39 │ │ └── bits-cfg.txt 40 │ ├── bits-tests 41 │ │ ├── smbios.py2 42 │ │ ├── testacpi.py2 43 │ │ └── testcpuid.py2 44 │ └── README 45 ├── acpi-bits.py 46 47* ``tests/avocado``: 48 49 ``acpi-bits.py``: 50 This is the main python avocado test script that generates a 51 biosbits iso. It then spawns a QEMU VM with it, collects the log and reports 52 test failures. This is the script one would be interested in if they wanted 53 to add or change some component of the log parsing, add a new command line 54 to alter how QEMU is spawned etc. Test writers typically would not need to 55 modify this script unless they wanted to enhance or change the log parsing 56 for their tests. In order to enable debugging, you can set **V=1** 57 environment variable. This enables verbose mode for the test and also dumps 58 the entire log from bios bits and more information in case failure happens. 59 60 In order to run this test, please perform the following steps from the QEMU 61 build directory: 62 :: 63 64 $ make check-venv (needed only the first time to create the venv) 65 $ ./tests/venv/bin/avocado run -t acpi tests/avocado 66 67 The above will run all acpi avocado tests including this one. 68 In order to run the individual tests, perform the following: 69 :: 70 71 $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py --tap - 72 73 The above will produce output in tap format. You can omit "--tap -" in the 74 end and it will produce output like the following: 75 :: 76 77 $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py 78 Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits 79 JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef 80 JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log 81 (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.09 s) 82 RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 83 JOB TIME : 39.22 s 84 85 You can inspect the log file for more information about the run or in order 86 to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs 87 would be found in the test log. 88 89* ``tests/avocado/acpi-bits/bits-config``: 90 91 This location contains biosbits configuration files that determine how the 92 software runs the tests. 93 94 ``bits-config.txt``: 95 This is the biosbits config file that determines what tests 96 or actions are performed by bits. The description of the config options are 97 provided in the file itself. 98 99* ``tests/avocado/acpi-bits/bits-tests``: 100 101 This directory contains biosbits python based tests that are run from within 102 the biosbits environment in the spawned VM. New additions of test cases can 103 be made in the appropriate test file. For example, new acpi tests can go 104 into testacpi.py2 and one would call testsuite.add_test() to register the new 105 test so that it gets executed as a part of the ACPI tests. 106 It might be occasionally necessary to disable some subtests or add a new 107 test that belongs to a test suite not already present in this directory. To 108 do this, please clone the bits source from 109 https://gitlab.com/qemu-project/biosbits-bits/-/tree/qemu-bits. 110 Note that this is the "qemu-bits" branch and not the "bits" branch of the 111 repository. "qemu-bits" is the branch where we have made all the QEMU 112 specific enhancements and we must use the source from this branch only. 113 Copy the test suite/script that needs modification (addition of new tests 114 or disabling them) from python directory into this directory. For 115 example, in order to change cpuid related tests, copy the following 116 file into this directory and rename it with .py2 extension: 117 https://gitlab.com/qemu-project/biosbits-bits/-/blob/qemu-bits/python/testcpuid.py 118 Then make your additions and changes here. Therefore, the steps are: 119 120 (a) Copy unmodified test script to this directory from bits source. 121 (b) Add a SPDX license header. 122 (c) Perform modifications to the test. 123 124 Commits (a), (b) and (c) should go under separate commits so that the original 125 test script and the changes we have made are separated and clear. 126 127 The test framework will then use your modified test script to run the test. 128 No further changes would be needed. Please check the logs to make sure that 129 appropriate changes have taken effect. 130 131 The tests have an extension .py2 in order to indicate that: 132 133 (a) They are python2.7 based scripts and not python 3 scripts. 134 (b) They are run from within the bios bits VM and is not subjected to QEMU 135 build/test python script maintainance and dependency resolutions. 136 (c) They need not be loaded by avocado framework when running tests. 137 138 139Author: Ani Sinha <ani@anisinha.ca> 140 141References: 142----------- 143.. [#a] https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf 144.. [#b] https://www.youtube.com/watch?v=36QIepyUuhg 145 146