13cab2bb3Spatrick======================================== 23cab2bb3SpatrickCompiler-rt Testing Infrastructure Guide 33cab2bb3Spatrick======================================== 43cab2bb3Spatrick 53cab2bb3Spatrick.. contents:: 63cab2bb3Spatrick :local: 73cab2bb3Spatrick 83cab2bb3SpatrickOverview 93cab2bb3Spatrick======== 103cab2bb3Spatrick 113cab2bb3SpatrickThis document is the reference manual for the compiler-rt modifications to the 123cab2bb3Spatricktesting infrastructure. Documentation for the infrastructure itself can be found at 133cab2bb3Spatrick:ref:`llvm_testing_guide`. 143cab2bb3Spatrick 153cab2bb3SpatrickLLVM testing infrastructure organization 163cab2bb3Spatrick======================================== 173cab2bb3Spatrick 183cab2bb3SpatrickThe compiler-rt testing infrastructure contains regression tests which are run 193cab2bb3Spatrickas part of the usual ``make check-all`` and are expected to always pass -- they 203cab2bb3Spatrickshould be run before every commit. 213cab2bb3Spatrick 223cab2bb3SpatrickQuick start 233cab2bb3Spatrick=========== 243cab2bb3Spatrick 253cab2bb3SpatrickThe regressions tests are in the "compiler-rt" module and are normally checked 263cab2bb3Spatrickout in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all`` 273cab2bb3Spatrickto run the regression tests after building compiler-rt. 283cab2bb3Spatrick 293cab2bb3SpatrickREQUIRES, XFAIL, etc. 303cab2bb3Spatrick--------------------- 313cab2bb3Spatrick 323cab2bb3SpatrickSometimes it is necessary to restrict a test to a specific target or mark it as 333cab2bb3Spatrickan "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or 34*810390e3Srobert``XFAIL:`` and the ``target=<target-triple>`` feature, typically with a regular 35*810390e3Srobertexpression matching an appropriate substring of the triple. Unfortunately, the 363cab2bb3Spatrickbehaviour of this is somewhat quirky in compiler-rt. There are two main 373cab2bb3Spatrickpitfalls to avoid. 383cab2bb3Spatrick 39*810390e3SrobertThe first pitfall is that these regular expressions may inadvertently match 40*810390e3Srobertmore triples than expected. For example, ``XFAIL: target=mips{{.*}}`` matches 41*810390e3Srobert``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and 42*810390e3Srobert``mips64el-linux-gnu``. Including a trailing ``-`` such as in 43*810390e3Srobert``XFAIL: target=mips-{{.*}}`` can help to mitigate this quirk but even that has 44*810390e3Srobertissues as described below. 453cab2bb3Spatrick 463cab2bb3SpatrickThe second pitfall is that the default target triple is often inappropriate for 473cab2bb3Spatrickcompiler-rt tests since compiler-rt tests may be compiled for multiple targets. 483cab2bb3SpatrickFor example, a typical build on an ``x86_64-linux-gnu`` host will often run the 49*810390e3Sroberttests for both x86_64 and i386. In this situation ``XFAIL: target=x86_64{{{.*}}`` 50*810390e3Srobertwill mark both the x86_64 and i386 tests as an expected failure while 51*810390e3Srobert``XFAIL: target=i386{{.*}}`` will have no effect at all. 523cab2bb3Spatrick 533cab2bb3SpatrickTo remedy both pitfalls, compiler-rt tests provide a feature string which can 543cab2bb3Spatrickbe used to specify a single target. This string is of the form 553cab2bb3Spatrick``target-is-${arch}`` where ``${arch}}`` is one of the values from the 563cab2bb3Spatrickfollowing lines of the CMake output:: 573cab2bb3Spatrick 583cab2bb3Spatrick -- Compiler-RT supported architectures: x86_64;i386 593cab2bb3Spatrick -- Builtin supported architectures: i386;x86_64 603cab2bb3Spatrick 613cab2bb3SpatrickSo for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail 623cab2bb3Spatrickon x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386`` 633cab2bb3Spatrickwill mark a test as expected to fail on i386 even if the default target triple 643cab2bb3Spatrickis ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string 653cab2bb3Spatrickrequire exact matches so ``XFAIL: target-is-mips``, 663cab2bb3Spatrick``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and 673cab2bb3Spatrick``XFAIL: target-is-mips64el`` all refer to different MIPS targets. 68