This is a collection of tests for GCC. For further information about
the C testsuite, see README.gcc.

The driver that runs this testsuite is called DejaGnu and you will
need a current DejaGnu snapshot, which is available from
ftp://gcc.gnu.org/pub/gcc/infrastructure, for example.

These tests are included "as is". If any of them fails, do not report
a bug.  Bug reports for DejaGnu can go to bug-dejagnu@gnu.org.
Discussion and comments about this testsuite should be sent to
gcc@gcc.gnu.org; additions and changes to should go to sent to
gcc-patches@gcc.gnu.org.

The entire testsuite is invoked by `make check` at the top level of
the GCC tree. `make check-g++` runs the C++ testsuite only.

STRUCTURE OF THE G++ TESTSUITE

  g++.dg tests:

  All new tests should be placed in an appropriate subdirectory of g++.dg.

  g++.old-deja tests:

  g++.benjamin	Tests by Benjamin Koz
  g++.bob
  g++.brendan	Tests by Brendan Kehoe
  g++.bugs
  g++.eh  	Tests for exception handling
  g++.ext  	Tests for g++ extensions
  g++.gb  	Tests by Gerald Baumgartner
  g++.jason	Tests by Jason Merill
  g++.jeff	Tests by Jeffrey A Law
  g++.martin	Tests by Martin v. L�wis
  g++.mike	Tests by Mike Stump
  g++.niklas	Tests by Niklas Hallqvist
  g++.ns  	Tests for namespaces
  g++.other
  g++.pt  	Tests for templates
  g++.rfg
  g++.robertl	Tests from gcc-bugs@gcc.gnu.org, gathered by Robert Lipe

Finally, some random last minute notes by Mike Stump <mrs@cygnus.com>, on
how to run tests (in the GCC 2.7 era):

	runtest --tool g++ --srcdir ./testsuite

where

	runtest	Is the name used to invoke DejaGnu.   If DejaGnu is not
		install this will be the relative path name for runtest.

	--tool	This tells DejaGnu which tool you are testing. It is
		mainly used to find the testsuite directories for a
		particular tool when several testsuites are in the
		same directory. (like the gcc and g++ testsuites)

	--srcdir This points to the top level of the directory
		containing the sources of the testsuite. This is
		./testsuite if you are in the directory that has the
		testsuite directory.


Copyright (C) 1998-2013 Free Software Foundation, Inc.

Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved.

Compatibility test suites that use lib/compat.exp test interoperability
of two compilers that follow the same ABI, or of multiple sets of
compiler options that are not intended to affect binary compatibility.
They do not test conformance to a formal ABI; they check that things are
done consistently, not that they are done correctly.  They are meant to
complement ABI test suites, not to replace them.

Each test in these suites splits up functionality into two source files
that can be compiled with two different versions of GCC and with two
sets of compiler options.

  Override the default pairs of compiler options (from compat.exp) by
  defining the environment variable COMPAT_OPTIONS as:
  COMPAT_OPTIONS="[list [list {tst1} {alt1}]...[list {tstn} {altn}]]"
  where tsti and alti are lists of options, with "tsti" used by the
  compiler under test and "alti" used by the alternate compiler. The
  characters "[", "{", "}", and "]" are really in the string, as in
  "[list [list {-g} {-O3}] [list {-O3} {-g}]]".

  To use an alternate compiler, define the environment variable for the
  language you're testing; e.g., for C++ define ALT_CXX_UNDER_TEST to be
  the full pathname of an installed compiler.  That will be written to
  the site.exp file used by DejaGNU.  The default is to build each test
  with the compiler under test using the first of each pair of compiler
  options from COMPAT_OPTIONS.  When ALT_CXX_UNDER_TEST is "same", each
  test is built using the compiler under test but with combinations of
  the options from COMPAT_OPTIONS.  If the alternate compiler needs a
  different setting for LD_LIBRARY_PATH, specify it in the environment
  varible ALT_LD_LIBRARY_PATH.  If it needs different options to build
  a shared object, define those in ALT_SHOBJ_OPTIONS.

  Normally, only a small amount of compatibility tests is run.
  Setting RUN_ALL_COMPAT_TESTS=1 in the environment before running the
  testsuite enables running all compatibility tests, but might take
  significantly longer than it takes without this variable.

To run only the C++ compatibility tests, optionally specifying an
alternate compiler and/or sets of options, do from $objdir/gcc:

  rm site.exp
  make -k ALT_CXX_UNDER_TEST=<prefix>/bin/g++ \
    ALT_LD_LIBRARY_PATH=<prefix>/lib \
    COMPAT_OPTIONS="list as shown above" check-c++ \
    RUNTESTFLAGS="compat.exp"

A test that fails when the pieces are compiled with different compilers
but passes when the pieces are compiled with the same compiler
demonstrates incompatibility of the generated code or runtime support.
A test that fails for the alternate compiler but passes for the compiler
under test probably tests for a fix that is not present in the alternate
compiler.

There's a lot more that can be done with these tests.

  Increase the robustness of the existing tests:

   -  recognize target triples in dg-options strings in test files

   -  continue to split up tests that rely on language runtime support
      and add them to this set of tests

   -  make sure they work for testing with a simulator

   -  multilibs support?

  Build test pieces into more combinations that will be likely to find
  unusual errors:

   -  link using the alternate compiler; this requires support for
      LD_LIBRARY_PATH for the alternate compiler in the support for each
      language

   -  build pieces of the test into static libraries; this requires
      knowledge about how to build a static library for each target

  Continue to add tests to g++.dg/compat, based on existing tests for
  language runtime support and for test cases in PRs that didn't used to
  fit into the test suite.

  Set up similar tests in libstdc++-v3/testsuite.

  Set up similar tests for additional languages and libraries.

Janis Johnson, <janis187@us.ibm.com>
October 2002


Copyright (C) 2002-2013 Free Software Foundation, Inc.

Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved.

This file contains a list of notes for those writing testcases and those
writing expect scripts.  At present, they're in random order.

Verbosity Levels

- each level adds verbosity so level 2 prints all level 1 and level 2 stuff.

1) Print a one-liner indicating the testcase being run (and maybe special
   compiler options).

2) Print compiler and program invocation including arguments and their output.
   Proc's gcc_load and gcc_start handle the latter two.

3) Print detailed testcase analysis like "Looking for pattern ...", etc.

4) Maximum verbosity.  Print anything else of interest.

send_log conventions

Various messages are stored in gcc.log by the testing framework and we
try to augment them with some of our own.  The format of the framework
messages are:

PASS: blah blah ...
FAIL: blah blah ...

so we use

XXXX: blah blah ...

Current messages are:

EXEC: program being executed (so compiler path and args are recorded)
STAT: intermediate pass/fail statistics

Test scripts must ignore the compiler messages "path prefix never used"
and "linker input file unused".  Don't let their appearance cause a testcase
to fail.  See lib/dg.exp for the exact regsub to use.

If you're unclear about which directory a testcase should be installed in,
ask gcc-local.

Have the text of a fail message be the same as that for pass.
IE: have

        if ...success...
                pass "pr 1234"
        else
                fail "pr 1234"

not

        if ...success...
                pass "pr 1234 passed"
        else
                fail "pr 1234 failed"


This lets test-tool (which drives the nightly tests) do a better job
at tracking which tests have digressed or been fixed.

DO NOT PUT NON-PORTABLE TESTCASES IN gcc.c-torture.

ANY TARGET SPECIFIC TESTCASE MUST HAVE APPROPRIATE CODE TO PREVENT IT FROM
CAUSING A `FAILURE' ON UNSUPPORTED PLATFORMS.

The "torture" tests are meant to be generic tests that can run on any
target.  So you have to be careful about endianness, assumptions about
sizes of datatypes, etc etc.

For tests that merely need to compile, put them in the "compile" directory.

For tests which should give an error, put them in the "noncompile" directory
and update noncompile.exp appropriately (see examples in noncompile.exp).

For IEEE FP specific tests, put them in execute/ieee.

For execution tests, put them in execute.

If a test does not fit into the torture framework, use the dg framework.


Copyright (C) 1997-2013 Free Software Foundation, Inc.

Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved.