1Compatibility test suites that use lib/compat.exp test interoperability
2of two compilers that follow the same ABI, or of multiple sets of
3compiler options that are not intended to affect binary compatibility.
4They do not test conformance to a formal ABI; they check that things are
5done consistently, not that they are done correctly.  They are meant to
6complement ABI test suites, not to replace them.
7
8Each test in these suites splits up functionality into two source files
9that can be compiled with two different versions of GCC and with two
10sets of compiler options.
11
12  Override the default pairs of compiler options (from compat.exp) by
13  defining the environment variable COMPAT_OPTIONS as:
14  COMPAT_OPTIONS="[list [list {tst1} {alt1}]...[list {tstn} {altn}]]"
15  where tsti and alti are lists of options, with "tsti" used by the
16  compiler under test and "alti" used by the alternate compiler. The
17  characters "[", "{", "}", and "]" are really in the string, as in
18  "[list [list {-g} {-O3}] [list {-O3} {-g}]]".
19
20  To use an alternate compiler, define the environment variable for the
21  language you're testing; e.g., for C++ define ALT_CXX_UNDER_TEST to be
22  the full pathname of an installed compiler.  That will be written to
23  the site.exp file used by DejaGNU.  The default is to build each test
24  with the compiler under test using the first of each pair of compiler
25  options from COMPAT_OPTIONS.  When ALT_CXX_UNDER_TEST is "same", each
26  test is built using the compiler under test but with combinations of
27  the options from COMPAT_OPTIONS.  If the alternate compiler needs a
28  different setting for LD_LIBRARY_PATH, specify it in the environment
29  varible ALT_LD_LIBRARY_PATH.  If it needs different options to build
30  a shared object, define those in ALT_SHOBJ_OPTIONS.
31
32  Normally, only a small amount of compatibility tests is run.
33  Setting RUN_ALL_COMPAT_TESTS=1 in the environment before running the
34  testsuite enables running all compatibility tests, but might take
35  significantly longer than it takes without this variable.
36
37To run only the C++ compatibility tests, optionally specifying an
38alternate compiler and/or sets of options, do from $objdir/gcc:
39
40  rm site.exp
41  make -k ALT_CXX_UNDER_TEST=<prefix>/bin/g++ \
42    ALT_LD_LIBRARY_PATH=<prefix>/lib \
43    COMPAT_OPTIONS="list as shown above" check-c++ \
44    RUNTESTFLAGS="compat.exp"
45
46A test that fails when the pieces are compiled with different compilers
47but passes when the pieces are compiled with the same compiler
48demonstrates incompatibility of the generated code or runtime support.
49A test that fails for the alternate compiler but passes for the compiler
50under test probably tests for a fix that is not present in the alternate
51compiler.
52
53There's a lot more that can be done with these tests.
54
55  Increase the robustness of the existing tests:
56
57   -  recognize target triples in dg-options strings in test files
58
59   -  continue to split up tests that rely on language runtime support
60      and add them to this set of tests
61
62   -  make sure they work for testing with a simulator
63
64   -  multilibs support?
65
66  Build test pieces into more combinations that will be likely to find
67  unusual errors:
68
69   -  link using the alternate compiler; this requires support for
70      LD_LIBRARY_PATH for the alternate compiler in the support for each
71      language
72
73   -  build pieces of the test into static libraries; this requires
74      knowledge about how to build a static library for each target
75
76  Continue to add tests to g++.dg/compat, based on existing tests for
77  language runtime support and for test cases in PRs that didn't used to
78  fit into the test suite.
79
80  Set up similar tests in libstdc++-v3/testsuite.
81
82  Set up similar tests for additional languages and libraries.
83
84Janis Johnson, <janis187@us.ibm.com>
85October 2002
86
87
88Copyright (C) 2002-2014 Free Software Foundation, Inc.
89
90Copying and distribution of this file, with or without modification,
91are permitted in any medium without royalty provided the copyright
92notice and this notice are preserved.
93