1.. _my-label: lnt
2
3.. _lnt:
4
5=================================
6LLVM LNT / Test-Suite Integration
7=================================
8
9Recommended reading materials
10-----------------------------
11  * https://blog.llvm.org/posts/2016-06-15-using-lnt-to-track-performance/
12  * https://llvm.org/docs/lnt/quickstart.html
13  * https://llvm.org/docs/TestSuiteGuide.html
14
15Prerequisites
16-------------
17  * all of the normal prerequisites for building **development** version of RawSpeed.
18  * python's `virtualenv <https://packages.debian.org/unstable/virtualenv>`_
19  * `llvm-size`, `llvm-lit <https://llvm.org/docs/CommandGuide/lit.html>`_ (from
20    `llvm <https://packages.debian.org/unstable/llvm-9>`_,
21    `llvm-tools <https://packages.debian.org/unstable/llvm-9-tools>`_ packages)
22  * A checkout of raw sample archive of you choice. It is suggested to use
23    `https://raw.pixls.us <https://raw.pixls.us>`_ masterset.
24    Please see :ref:`RSA` page for details.
25  * Reference hashes for the raws in the sampleset.
26
27    Generate them via ``$ ninja rstest-create`` from your **trusted** (!) dev build.
28    Please see :ref:`integration_testing` and
29    :ref:`producing_trusted_reference_hashes` pages for details.
30
31Getting it done
32---------------
33.. code:: bash
34
35  # Get external dependencies.
36  git clone https://github.com/llvm/llvm-lnt.git /build/lnt
37  git clone https://github.com/llvm/llvm-test-suite.git /build/test-suite
38  # 'Add' RawSpeed into test-suite.
39  ln -s /home/lebedevri/rawspeed/lnt /build/test-suite/RawSpeed
40  # Prepare 'chroot'.
41  export SANDBOX=/tmp/mysandbox
42  export BUILDSANDBOX=/tmp/mybuildsandbox
43  export PERFDB=/tmp/myperfdb
44  virtualenv $SANDBOX
45  # Setup 'chroot'.
46  $SANDBOX/bin/python /build/lnt/setup.py develop
47  $SANDBOX/bin/lnt create $PERFDB
48  # Benchmarking time!
49  $SANDBOX/bin/lnt runtest test_suite --test-suite /build/test-suite \
50      --sandbox $BUILDSANDBOX \
51      --cc clang --cxx clang++ \
52      --cmake-define CMAKE_BUILD_TYPE=Release \
53      --cmake-define TEST_SUITE_SUBDIRS="RawSpeed" \
54      [--benchmarking-only] \
55      [--use-perf profile] \
56      --build-threads 8 --threads 1 \
57      --submit $PERFDB
58  # View results.
59  $SANDBOX/bin/lnt runserver $PERFDB
60
61
62.. seealso::
63
64  * https://llvm.org/docs/TestSuiteGuide.html#common-configuration-options
65  * ``$ $SANDBOX/bin/lnt runtest test_suite --help``
66  * ``$ $SANDBOX/bin/lnt --help``
67