Release notes for Valgrind
~~~~~~~~~~~~~~~~~~~~~~~~~~
If you are building a binary package of Valgrind for distribution,
please read README_PACKAGERS.  It contains some important information.

If you are developing Valgrind, please read README_DEVELOPERS.  It contains
some useful information.

For instructions on how to build/install, see the end of this file.

If you have problems, consult the FAQ to see if there are workarounds.


Executive Summary
~~~~~~~~~~~~~~~~~
Valgrind is a framework for building dynamic analysis tools. There are
Valgrind tools that can automatically detect many memory management
and threading bugs, and profile your programs in detail. You can also
use Valgrind to build new tools.

The Valgrind distribution currently includes six production-quality
tools: a memory error detector, two thread error detectors, a cache
and branch-prediction profiler, a call-graph generating cache abd
branch-prediction profiler, and a heap profiler. It also includes
three experimental tools: a heap/stack/global array overrun detector,
a different kind of heap profiler, and a SimPoint basic block vector
generator.

Valgrind is closely tied to details of the CPU, operating system and to
a lesser extent, compiler and basic C libraries. This makes it difficult
to make it portable.  Nonetheless, it is available for the following
platforms:

- X86/Linux
- AMD64/Linux
- PPC32/Linux
- PPC64/Linux
- ARM/Linux
- x86/macOS
- AMD64/macOS
- S390X/Linux
- MIPS32/Linux
- MIPS64/Linux
- X86/Solaris
- AMD64/Solaris

Note that AMD64 is just another name for x86_64, and Valgrind runs fine
on Intel processors.  Also note that the core of macOS is called
"Darwin" and this name is used sometimes.

Valgrind is licensed under the GNU General Public License, version 2.
Read the file COPYING in the source distribution for details.

However: if you contribute code, you need to make it available as GPL
version 2 or later, and not 2-only.


Documentation
~~~~~~~~~~~~~
A comprehensive user guide is supplied.  Point your browser at
$PREFIX/share/doc/valgrind/manual.html, where $PREFIX is whatever you
specified with --prefix= when building.


Building and installing it
~~~~~~~~~~~~~~~~~~~~~~~~~~
To install from the GIT repository:

  0. Clone the code from GIT, following the instructions at
     http://www.valgrind.org/downloads/repository.html.

  1. cd into the source directory.

  2. Run ./autogen.sh to setup the environment (you need the standard
     autoconf tools to do so).

  3. Continue with the following instructions...

To install from a tar.bz2 distribution:

  4. Run ./configure, with some options if you wish.  The only interesting
     one is the usual --prefix=/where/you/want/it/installed.

  5. Run "make".

  6. Run "make install", possibly as root if the destination permissions
     require that.

  7. See if it works.  Try "valgrind ls -l".  Either this works, or it
     bombs out with some complaint.  In that case, please let us know
     (see www.valgrind.org).

Important!  Do not move the valgrind installation into a place
different from that specified by --prefix at build time.  This will
cause things to break in subtle ways, mostly when Valgrind handles
fork/exec calls.


The Valgrind Developers

Status
~~~~~~

As of Jan 2014 the trunk contains a port to AArch64 ARMv8 -- loosely,
the 64-bit ARM architecture.  Currently it supports integer and FP
instructions and can run anything generated by gcc-4.8.2 -O3.  The
port is under active development.

Current limitations, as of mid-May 2014.

* limited support of vector (SIMD) instructions.  Initial target is
  support for instructions created by gcc-4.8.2 -O3
  (via autovectorisation).  This is complete.

* Integration with the built in GDB server:
   - works ok (breakpoint, attach to a process blocked in a syscall, ...)
   - still to do:
      arm64 xml register description files (allowing shadow registers
                                            to be looked at).
      cpsr transfer to/from gdb to be looked at (see also arm equivalent code)

* limited syscall support

There has been extensive testing of the baseline simulation of integer
and FP instructions.  Memcheck is also believed to work, at least for
small examples.  Other tools appear to at least not crash when running
/bin/date.

Enough syscalls and instructions are supported for substantial
programs to work.  Firefox 26 is able to start up and quit.  The noise
level from Memcheck is low enough to make it practical to use for real
debugging.


Building
~~~~~~~~

You could probably build it directly on a target OS, using the normal
non-cross scheme

  ./autogen.sh ; ./configure --prefix=.. ; make ; make install

Development so far was however done by cross compiling, viz:

  export CC=aarch64-linux-gnu-gcc
  export LD=aarch64-linux-gnu-ld
  export AR=aarch64-linux-gnu-ar

  ./autogen.sh
  ./configure --prefix=`pwd`/Inst --host=aarch64-unknown-linux \
              --enable-only64bit
  make -j4
  make -j4 install

Doing this assumes that the install path (`pwd`/Inst) is valid on
both host and target, which isn't normally the case.  To avoid
this limitation, do instead:

  ./configure --prefix=/install/path/on/target \
              --host=aarch64-unknown-linux \
              --enable-only64bit
  make -j4
  make -j4 install DESTDIR=/a/temp/dir/on/host
  # and then copy the contents of DESTDIR to the target.

See README.android for more examples of cross-compile building.


Implementation tidying-up/TODO notes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

UnwindStartRegs -- what should that contain?


vki-arm64-linux.h: vki_sigaction_base
I really don't think that __vki_sigrestore_t sa_restorer
should be present.  Adding it surely puts sa_mask at a wrong
offset compared to (kernel) reality.  But not having it causes
compilation of m_signals.c to fail in hard to understand ways,
so adding it temporarily.


m_trampoline.S: what's the unexecutable-insn value? 0xFFFFFFFF
is there at the moment, but 0x00000000 is probably what it should be.
Also, fix indentation/tab-vs-space stuff


./include/vki/vki-arm64-linux.h: uses __uint128_t.  Should change
it to __vki_uint128_t, but what's the defn of that?


m_debuginfo/priv_storage.h: need proper defn of DiCfSI


readdwarf.c: is this correct?
#elif defined(VGP_arm64_linux)
#  define FP_REG         29    //???
#  define SP_REG         31    //???
#  define RA_REG_DEFAULT 30    //???


vki-arm64-linux.h:
re linux-3.10.5/include/uapi/asm-generic/sembuf.h
I'd say the amd64 version has padding it shouldn't have.  Check?


syswrap-linux.c run_a_thread_NORETURN assembly sections
seems like tst->os_state.exitcode has word type
in which case the ppc64_linux use of lwz to read it, is wrong


syswrap-linux.c ML_(do_fork_clone)
assuming that VGP_arm64_linux is the same as VGP_arm_linux here


dispatch-arm64-linux.S: FIXME: set up FP control state before
entering generated code.  Also fix screwy indentation.


dispatcher-ery general: what's a good (predictor-friendly) way to
branch to a register?


in vki-arm64-scnums.h
//#if __BITS_PER_LONG == 64 && !defined(__SYSCALL_COMPAT)
Probably want to reenable that and clean up accordingly


putIRegXXorZR: figure out a way that the computed value is actually
used, so as to keep any memory reads that might generate it, alive.
(else the simulation can lose exceptions).  At least, for writes to
the zero register generated by loads .. or .. can anything other
integer instructions, that write to a register, cause exceptions?


loads/stores: generate stack alignment checks as necessary


fix barrier insns: ISB, DMB


fix atomic loads/stores


FMADD/FMSUB/FNMADD/FNMSUB: generate and use the relevant fused
IROps so as to avoid double rounding


ARM64Instr_Call getRegUsage: re-check relative to what
getAllocableRegs_ARM64 makes available


Make dispatch-arm64-linux.S save any callee-saved Q regs
I think what is required is to save D8-D15 and nothing more than that.


wrapper for __NR3264_fstat -- correct?


PRE(sys_clone): get rid of references to vki_modify_ldt_t and the
definition of it in vki-arm64-linux.h.  Ditto for 32 bit arm.


sigframe-arm64-linux.c: build_sigframe: references to nonexistent
siguc->uc_mcontext.trap_no, siguc->uc_mcontext.error_code have been
replaced by zero.  Also in synth_ucontext.


m_debugger.c:
uregs.pstate   = LibVEX_GuestARM64_get_nzcv(vex); /* is this correct? */
Is that remotely correct?


host_arm64_defs.c: emit_ARM64INstr:
ARM64in_VDfromX and ARM64in_VQfromXX: use simple top-half zeroing
MOVs to vector registers instead of INS Vd.D[0], Xreg, to avoid false
dependencies on the top half of the register.  (Or at least check
the semantics of INS Vd.D[0] to see if it zeroes out the top.)


preferredVectorSubTypeFromSize: review perf effects and decide
on a types-for-subparts policy


fold_IRExpr_Unop: add a reduction rule for this
1Sto64(CmpNEZ64( Or64(GET:I64(1192),GET:I64(1184)) ))
vis 1Sto64(CmpNEZ64(x)) --> CmpwNEZ64(x)


check insn selection for memcheck-only primops:
Left64 CmpwNEZ64 V128to64 V128HIto64 1Sto64 CmpNEZ64 CmpNEZ32
widen_z_8_to_64 1Sto32 Left32 32HLto64 CmpwNEZ32 CmpNEZ8


isel: get rid of various cases where zero is put into a register
and just use xzr instead.  Especially for CmpNEZ64/32.  And for
writing zeroes into the CC thunk fields.


/* Keep this list in sync with that in iselNext below */
/* Keep this list in sync with that for Ist_Exit above */
uh .. they are not in sync


very stupid:
imm64  x23, 0xFFFFFFFFFFFFFFA0
17 F4 9F D2 F7 FF BF F2 F7 FF DF F2 F7 FF FF F2


valgrind.h: fix VALGRIND_ALIGN_STACK/VALGRIND_RESTORE_STACK,
also add CFI annotations


could possibly bring r29 into use, which be useful as it is
callee saved


ubfm/sbfm etc: special case cases that are simple shifts, as iropt
can't always simplify the general-case IR to a shift in such cases.


LDP,STP (immediate, simm7) (FP&VEC)
should zero out hi parts of dst registers in the LDP case


DUP insns: use Iop_Dup8x16, Iop_Dup16x8, Iop_Dup32x4
rather than doing it "by hand"


Any place where ZeroHI64ofV128 is used in conjunction with
FP vector IROps: find a way to make sure that arithmetic on
the upper half of the values is "harmless."


math_MINMAXV: use real Iop_Cat{Odd,Even}Lanes ops rather than
inline scalar code


chainXDirect_ARM64: use direct jump forms when possible

How to cross-compile and run on Android.  Please read to the end,
since there are important details further down regarding crash
avoidance and GPU support.

These notes were last updated on 4 Nov 2014, for Valgrind SVN
revision 14689/2987.

These instructions are known to work, or have worked at some time in
the past, for:

arm:
  Android 4.0.3 running on a (rooted, AOSP build) Nexus S.
  Android 4.0.3 running on Motorola Xoom.
  Android 4.0.3 running on android arm emulator.
  Android 4.1   running on android emulator.
  Android 2.3.4 on Nexus S worked at some time in the past.

x86:
  Android 4.0.3 running on android x86 emulator.

mips32:
  Android 4.1.2 running on android mips emulator.
  Android 4.2.2 running on android mips emulator.
  Android 4.3   running on android mips emulator.
  Android 4.0.4 running on BROADCOM bcm7425

arm64:
  Android 4.5 (?) running on ARM Juno

On android-arm, GDBserver might insert breaks at wrong addresses.
Feedback on this welcome.

Other configurations and toolchains might work, but haven't been tested.
Feedback is welcome.

Toolchain:

  For arm32, x86 and mips32 you need the android-ndk-r6 native
    development kit.  r6b and r7 give a non-completely-working build;
    see http://code.google.com/p/android/issues/detail?id=23203
    For the android emulator, the versions needed and how to install
    them are described in README.android_emulator.

    You can get android-ndk-r6 from
    http://dl.google.com/android/ndk/android-ndk-r6-linux-x86.tar.bz2

  For arm64 (aarch64) you need the android-ndk-r10c NDK, from
    http://dl.google.com/android/ndk/android-ndk-r10c-linux-x86_64.bin

Install the NDK somewhere.  Doesn't matter where.  Then:


# Modify this (obviously).  Note, this "export" command is only done
# so as to reduce the amount of typing required.  None of the commands
# below read it as part of their operation.
#
export NDKROOT=/path/to/android-ndk-r<version>


# Then cd to the root of your Valgrind source tree.
#
cd /path/to/valgrind/source/tree


# After this point, you don't need to modify anything.  Just copy and
# paste the commands below.


# Set up toolchain paths.
#
# For ARM
export AR=$NDKROOT/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-ar
export LD=$NDKROOT/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-ld
export CC=$NDKROOT/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc

# For x86
export AR=$NDKROOT/toolchains/x86-4.4.3/prebuilt/linux-x86/bin/i686-android-linux-ar
export LD=$NDKROOT/toolchains/x86-4.4.3/prebuilt/linux-x86/bin/i686-android-linux-ld
export CC=$NDKROOT/toolchains/x86-4.4.3/prebuilt/linux-x86/bin/i686-android-linux-gcc

# For MIPS32
export AR=$NDKROOT/toolchains/mipsel-linux-android-4.8/prebuilt/linux-x86_64/bin/mipsel-linux-android-ar
export LD=$NDKROOT/toolchains/mipsel-linux-android-4.8/prebuilt/linux-x86_64/bin/mipsel-linux-android-ld
export CC=$NDKROOT/toolchains/mipsel-linux-android-4.8/prebuilt/linux-x86_64/bin/mipsel-linux-android-gcc

# For ARM64 (AArch64)
export AR=$NDKROOT/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-ar
export LD=$NDKROOT/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-ld
export CC=$NDKROOT/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc


# Do configuration stuff.  Don't mess with the --prefix in the
# configure command below, even if you think it's wrong.
# You may need to set the --with-tmpdir path to something
# different if /sdcard doesn't work on the device -- this is
# a known cause of difficulties.

# The below re-generates configure, Makefiles, ...
# This is not needed if you start from a release tarball.
./autogen.sh

# for ARM
CPPFLAGS="--sysroot=$NDKROOT/platforms/android-3/arch-arm" \
   CFLAGS="--sysroot=$NDKROOT/platforms/android-3/arch-arm" \
   ./configure --prefix=/data/local/Inst \
   --host=armv7-unknown-linux --target=armv7-unknown-linux \
   --with-tmpdir=/sdcard
# note: on android emulator, android-14 platform was also tested and works.
# It is not clear what this platform nr really is.

# for x86
CPPFLAGS="--sysroot=$NDKROOT/platforms/android-9/arch-x86" \
   CFLAGS="--sysroot=$NDKROOT/platforms/android-9/arch-x86 -fno-pic" \
   ./configure --prefix=/data/local/Inst \
   --host=i686-android-linux --target=i686-android-linux \
   --with-tmpdir=/sdcard

# for MIPS32
CPPFLAGS="--sysroot=$NDKROOT/platforms/android-18/arch-mips" \
   CFLAGS="--sysroot=$NDKROOT/platforms/android-18/arch-mips" \
   ./configure --prefix=/data/local/Inst \
   --host=mipsel-linux-android --target=mipsel-linux-android \
   --with-tmpdir=/sdcard

# for ARM64 (AArch64)
CPPFLAGS="--sysroot=$NDKROOT/platforms/android-21/arch-arm64" \
   CFLAGS="--sysroot=$NDKROOT/platforms/android-21/arch-arm64" \
   ./configure --prefix=/data/local/Inst \
   --host=aarch64-unknown-linux --target=aarch64-unknown-linux \
   --with-tmpdir=/sdcard


# At the end of the configure run, a few lines of details
# are printed.  Make sure that you see these two lines:
#
# For ARM:
#          Platform variant: android
#     Primary -DVGPV string: -DVGPV_arm_linux_android=1
#
# For x86:
#          Platform variant: android
#     Primary -DVGPV string: -DVGPV_x86_linux_android=1
#
# For mips32:
#          Platform variant: android
#     Primary -DVGPV string: -DVGPV_mips32_linux_android=1
#
# For ARM64 (AArch64):
#          Platform variant: android
#     Primary -DVGPV string: -DVGPV_arm64_linux_android=1
#
# If you see anything else at this point, something is wrong, and
# either the build will fail, or will succeed but you'll get something
# which won't work.


# Build, and park the install tree in `pwd`/Inst
#
make -j4
make -j4 install DESTDIR=`pwd`/Inst


# To get the install tree onto the device:
# (I don't know why it's not "adb push Inst /data/local", but this
# formulation does appear to put the result in /data/local/Inst.)
#
adb push Inst /


# To run (on the device).  There are two things you need to consider:
#
# (1) if you are running on the Android emulator, Valgrind may crash
#     at startup.  This is because the emulator (for ARM) may not be
#     simulating a hardware TLS register.  To get around this, run
#     Valgrind with:
#       --kernel-variant=android-no-hw-tls
#
# (2) if you are running a real device, you need to tell Valgrind
#     what GPU it has, so Valgrind knows how to handle custom GPU
#     ioctls.  You can choose one of the following:
#       --kernel-variant=android-gpu-sgx5xx     # PowerVR SGX 5XX series
#       --kernel-variant=android-gpu-adreno3xx  # Qualcomm Adreno 3XX series
#     If you don't choose one, the program will still run, but Memcheck
#     may report false errors after the program performs GPU-specific ioctls.
#
# Anyway: to run on the device:
#
/data/local/Inst/bin/valgrind [kernel variant args] [the usual args etc]


# Once you're up and running, a handy modify-V-rebuild-reinstall
# command line (on the host, of course) is
#
mq -j2 && mq -j2 install DESTDIR=`pwd`/Inst && adb push Inst /
#
# where 'mq' is an alias for 'make --quiet'.


# One common cause of runs failing at startup is the inability of
# Valgrind to find a suitable temporary directory.  On the device,
# there doesn't seem to be any one location which we always have
# permission to write to.  The instructions above use /sdcard.  If
# that doesn't work for you, and you're Valgrinding one specific
# application which is already installed, you could try using its
# temporary directory, in /data/data, for example
# /data/data/org.mozilla.firefox_beta.
#
# Using /system/bin/logcat on the device is helpful for diagnosing
# these kinds of problems.

How to install and run an android emulator.

mkdir android # or any other place you prefer
cd android

# download java JDK
# http://www.oracle.com/technetwork/java/javase/downloads/index.html
# download android SDK
# http://developer.android.com/sdk/index.html
# download android NDK
# http://developer.android.com/sdk/ndk/index.html

# versions I used:
#  jdk-7u4-linux-i586.tar.gz
#  android-ndk-r8-linux-x86.tar.bz2
#  android-sdk_r18-linux.tgz

# install jdk
tar xzf jdk-7u4-linux-i586.tar.gz

# install sdk
tar xzf android-sdk_r18-linux.tgz

# install ndk
tar xjf android-ndk-r8-linux-x86.tar.bz2


# setup PATH to use the installed software:
export SDKROOT=$HOME/android/android-sdk-linux
export PATH=$PATH:$SDKROOT/tools:$SDKROOT/platform-tools
export NDKROOT=$HOME/android/android-ndk-r8

# install android platforms you want by starting:
android
# (from $SDKROOT/tools)

# select the platforms you need
# I selected and installed:
#   Android 4.0.3 (API 15)
# Upgraded then to the newer version available:
#     Android sdk 20
#     Android platform tools 12

# then define a virtual device:
Tools -> Manage AVDs...
# I define an AVD Name with 64 Mb SD Card, (4.0.3, api 15)
# rest is default


# compile and make install Valgrind, following README.android


# Start your android emulator (it takes some time).
# You can use adb shell to get a shell on the device
# and see it is working. Note that I usually get
# one or two time out from adb shell before it works
adb shell

# Once the emulator is ready, push your Valgrind to the emulator:
adb push Inst /


# IMPORTANT: when running Valgrind, you may need give it the flag
#
#    --kernel-variant=android-no-hw-tls
#
# since otherwise it may crash at startup.
# See README.android for details.


# if you need to debug:
# You have on the android side a gdbserver
# on the device side:
gdbserver :1234 your_exe

# on the host side:
adb forward tcp:1234 tcp:1234
$HOME/android/android-ndk-r8/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-gdb your_exe
target remote :1234

Supported platforms
-------------------
- MIPS32 and MIPS64 platforms are currently supported.
- Both little-endian and big-endian cores are supported.
- MIPS DSP ASE on MIPS32 platforms is supported.


Building V for MIPS
-------------------
- Native build is available for all supported platforms. The build system
expects that native GCC is configured correctly and optimized for the platform.
Yet, this may not be the case with some Debian distributions which configure
GCC to compile to "mips1" by default. Depending on a target platform, using
CFLAGS="-mips32r2", CFLAGS="-mips32" or CFLAGS="-mips64" or
CFLAGS="-mips64 -mabi=64" will do the trick and compile Valgrind correctly.

- Use of cross-toolchain is supported as well.
- Example of configure line and additional configure options:

   $ ./configure --host=mipsel-linux-gnu --prefix=<path_to_install_directory>

 * --host=mips-linux-gnu is necessary only if Valgrind is built on platform
   other then MIPS, tools for building MIPS application have to be in PATH.

 * --host=mips-linux-gnu is necessary if you compile it with cross toolchain
   compiler for big endian platform.

 * --host=mipsel-linux-gnu is necessary if you compile it with cross toolchain
   compiler for little endian platform.

 * --build=mips-linux is needed if you want to build it for MIPS32 on 64-bit
   MIPS system.

 * If you are compiling Valgrind for mips32 with gcc version older then
   gcc (GCC) 4.5.1, you must specify CFLAGS="-mips32r2 -mplt", e.g.

   ./configure --prefix=<path_to_install_directory>
   CFLAGS="-mips32r2 -mplt"


Limitations
-----------
- Some gdb tests will fail when gdb (GDB) older than 7.5 is used and gdb is
  not compiled with '--with-expat=yes'.
- You can not compile tests for DSP ASE if you are using gcc (GCC) older
  then 4.6.1 due to a bug in the toolchain.
- Older GCC may have issues with some inline assembly blocks. Get a toolchain
  based on newer GCC versions, if possible.

Requirements
------------
- You need GCC 3.4 or later to compile the s390 port.
- To run valgrind a z10 machine or any later model is recommended.
  Older machine models down to and including z990 may work but have
  not been tested extensively.


Limitations
-----------
- 31-bit client programs are not supported.
- Hexadecimal floating point is not supported.
- Transactional memory is not supported.
- Instructions operating on vector registers are not supported.
- memcheck, cachegrind, drd, helgrind, massif, lackey, and none are
  supported.
- On machine models predating z10, cachegrind will assume a z10 cache
  architecture. Otherwise, cachegrind will query the hosts cache system
  and use those parameters.
- callgrind and all experimental tools are currently not supported.
- Some gcc versions use mvc to copy 4/8 byte values. This will affect
  certain debug messages. For example, memcheck will complain about
  4 one-byte reads/writes instead of just a single read/write.
- The transactional-execution facility is not supported; it is masked
  off from HWCAP.


Hardware facilities
-------------------
Valgrind does not require that the host machine has the same hardware
facilities as the machine for which the client program was compiled.
This is convenient. If possible, the JIT compiler will translate the
client instructions according to the facilities available on the host.
This means, though, that probing for hardware facilities by issuing
instructions from that facility and observing whether SIGILL is thrown
may not work. As a consequence, programs that attempt to do so may
behave differently. It is believed that this is a rare use case.


Recommendations
---------------
Applications should be compiled with -fno-builtin to avoid
false positives due to builtin string operations when running memcheck.


Reading Material
----------------
(1) Linux for zSeries ELF ABI Supplement
    http://refspecs.linuxfoundation.org/ELF/zSeries/index.html
(2) z/Architecture Principles of Operation
    http://publibfi.boulder.ibm.com/epubs/pdf/dz9zr010.pdf
(3) z/Architecture Reference Summary
    http://publibfi.boulder.ibm.com/epubs/pdf/dz9zs008.pdf

Requirements
------------
- You need a recent Solaris-like OS to compile this port. Solaris 11 or
  any illumos-based distribution should work, Solaris 10 is not supported.
  Running `uname -r` has to print '5.11'.
- Recent GCC tools are required, GCC 3 will probably not work. GCC version
  4.5 (or higher) is recommended.
- Solaris ld has to be the first linker in the PATH. GNU ld cannot be used.
  There is currently no linker check in the configure script but the linking
  phase fails if GNU ld is used. Recent Solaris/illumos distributions are ok.
- A working combination of autotools is required: aclocal, autoheader,
  automake and autoconf have to be found in the PATH. You should be able to
  install pkg:/developer/build/automake and pkg:/developer/build/autoconf
  packages to fulfil this requirement.
- System header files are required. On Solaris, these can be installed with:
    # pkg install system/header
- GNU make is also required. On Solaris, this can be quickly achieved with:
    $ PATH=/usr/gnu/bin:$PATH; export PATH
- For remote debugging support, working GDB is required (see below).
- For running regression tests, GNU sed, grep, awk, diff are required.
  This can be quickly achieved on Solaris by prepending /usr/gnu/bin to PATH.


Compilation
-----------
Please follow the generic instructions in the README file.

The configure script detects a canonical host to determine which version of
Valgrind should be built. If the system compiler by default produces 32-bit
binaries then only a 32-bit version of Valgrind will be built. To enable
compilation of both 64-bit and 32-bit versions on such a system, issue the
configure script as follows:
./configure CC='gcc -m64' CXX='g++ -m64'


Oracle Solaris and illumos support
----------------------------------
One of the main goal of this port is to support both Oracle Solaris and
illumos kernels. This is a very hard task because Solaris kernel traditionally
does not provide a stable syscall interface and because Valgrind contains
several parts that are closely tied to the underlying kernel. For these
reasons, the port needs to detect which syscall interfaces are present. This
detection cannot be done easily at run time and is currently implemented as
a set of configure tests. This means that a binary version of this port can be
executed only on a kernel that is compatible with a kernel that was used
during the configure and compilation time.

Main currently-known incompatibilities:
- Solaris 11 (released in November 2011) removed a large set of syscalls where
  *at variant of the syscall was also present, for example, open() versus
  openat(AT_FDCWD) [1]
- syscall number for unlinkat() is 76 on Solaris 11, but 65 on illumos [2]
- illumos (in April 2013) changed interface of the accept() and pipe()
  syscalls [3]
- posix_spawn() functionality is backed up by true spawn() syscall on Solaris 11.4
  whereas illumos and Solaris 11.3 leverage vfork()
- illumos and older Solaris use utimesys() syscall whereas newer Solaris
  uses utimensat()

[1] http://docs.oracle.com/cd/E26502_01/html/E28556/gkzlf.html#gkzip
[2] https://www.illumos.org/issues/521
[3] https://github.com/illumos/illumos-gate/commit/5dbfd19ad5fcc2b779f40f80fa05c1bd28fd0b4e


Limitations
-----------
- The port is Work-In-Progress, many things may not work or they can be subtly
  broken.
- Coredumps produced by Valgrind do not contain all information available,
  especially microstate accounting and processor bindings.
- Accessing contents of /proc/self/psinfo is not thread-safe.  That is because
  Valgrind emulates this file on behalf of the client programs.  Entire
  open() - read() - close() sequence on this file needs to be performed
  atomically.
- Fork limitations: vfork() is translated to fork(), forkall() is not
  supported.
- Valgrind does not track definedness of some eflags (OF, SF, ZF, AF, CF, PF)
  individually for each flag. After a syscall is finished, when a carry flag
  is set and defined, all other mentioned flags will be also defined even
  though they might be undefined before making the syscall.
- System call "execve" with a file descriptor which points to a hardlink
  is currently not supported. That is because from the opened file descriptor
  itself it is not possible to reverse map the intended pathname.
  Examples are fexecve(3C) and isaexec(3C).
- Program headers PT_SUNW_SYSSTAT and PT_SUNW_SYSSTAT_ZONE are not supported.
  That is, programs linked with mapfile directive RESERVE_SEGMENT and attribute
  TYPE equal to SYSSTAT or SYSSTAT_ZONE will cause Valgrind exit. It is not
  possible for Valgrind to arrange mapping of a kernel shared page at the
  address specified in the mapfile for the guest application. There is currently
  no such mechanism in Solaris. Hacky workarounds are possible, though.
- When a thread has no stack then all system calls will result in Valgrind
  crash, even though such system calls use just parameters passed in registers.
  This should happen only in pathological situations when a thread is created
  with custom mmap'ed stack and this stack is then unmap'ed during thread
  execution.


Remote debugging support
------------------------
Solaris port of GDB has a major flaw which prevents remote debugging from
working correctly. Fortunately this flaw has an easy fix [4]. Unfortunately
it is not present in the current GDB 7.6.2. This boils down to several
options:
- Use GDB shipped with Solaris 11.2 which has this flaw fixed.
- Wait until GDB 7.7 becomes available (there won't be other 7.6.x releases).
- Build GDB 7.6.2 with the fix by yourself using the following steps:
    # pkg install developer/gnu-binutils
    $ wget http://ftp.gnu.org/gnu/gdb/gdb-7.6.2.tar.gz
    $ gzip -dc gdb-7.6.2.tar.gz | tar xf -
    $ cd gdb-7.6.2
    $ patch -p1 -i /path/to/valgrind-solaris/solaris/gdb-sol-thread.patch
    $ export LIBS="-lncurses"
    $ export CC="gcc -m64"
    $ ./configure --with-x=no --with-curses --with-libexpat-prefix=/usr/lib
    $ gmake && gmake install

[4] https://sourceware.org/ml/gdb-patches/2013-12/msg00573.html


TODO list
---------
- Fix few remaining failing tests.
- Add more Solaris-specific tests (especially for the door and spawn
  syscalls).
- Provide better error reporting for various subsyscalls.
- Implement storing of extra register state in signal frame.
- Performance comparison against other platforms.
- Prevent SIGPIPE when writing to a socket (coregrind/m_libcfile.c).
- Implement ticket locking for fair scheduling (--fair-sched=yes).
- Implement support in DRD and Helgrind tools for thr_join() with thread == 0.
- Add support for accessing thread-local variables via gdb (auxprogs/getoff.c).
  Requires research on internal libc TLS representation.
- VEX supports AVX, BMI and AVX2. Investigate if they can be enabled on
  Solaris/illumos.
- Investigate support for more flags in AT_SUN_AUXFLAGS.
- Fix Valgrind crash when a thread has no stack and syswrap-main.c accesses
  all possible syscall parameters. Enable helgrind/tests/stackteardown.c
  to see this in effect. Would require awareness of syscall parameter semantics.
- Correctly print arguments of DW_CFA_ORCL_arg_loc in show_CF_instruction() when
  it is implemented in libdwarf.
- Handle a situation when guest program sets SC_CANCEL_FLG in schedctl and
  Valgrind needs to invoke a syscall on its own.


Summary of Solaris 11 Kernel Interfaces Used
--------------------------------------------
Valgrind uses directly the following kernel interfaces (not exhaustive list).
Then, of course, it has very intimate knowledge of all syscalls, many ioctls
and some door calls because it has wrappers around them.
- Syscalls:
  . clock_gettime
  . close
  . connect
  . execve
  . exit
  . faccessat
  . fcntl
  . forksys
  . fstatat
  . getcwd
  . getdents
  . geteuid
  . getgid
  . getgroups
  . getpeername
  . getpid
  . getrlimit
  . getsockname
  . getsockopt
  . gettimeofday
  . kill
  . lseek
  . lwp_create
  . lwp_exit
  . lwp_self
  . lwp_sigqueue
  . mknodat
  . mmap
  . mprotect
  . munmap
  . openat
  . pipe
  . pollsys
  . pread
  . prgpsys
  . pwrite
  . read
  . readlinkat
  . renameat
  . rt_sigprocmask
  . send
  . setrlimit
  . setsockopt
  . sigaction
  . sigreturn
  . sigtimedwait
  . so_socket
  . spawn
  . uname
  . unlinkat
  . waitsys
  . write
- Signal frames. Valgrind decomposes and synthetizes signal frames.
- Flag sc_sigblock flag in the schedctl structure by replacing
  function block_all_signals() from libc. The replacement emulates lwp_sigmask
  syscall. More details in coregrind/vg_preloaded.c.
- Initial stack layout for the main thread is synthetized.
- procfs agent thread and other procfs commands for manipulating the process.
- mmapobj syscall is emulated because it gets in the way of the address space
  manager's control.


Contacts
--------
Please send bug reports and any questions about the port to:
Ivo Raisr <ivosh@ivosh.net>
Petr Pavlu <setup@dagobah.cz>

Building and not installing it
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To run Valgrind without having to install it, run coregrind/valgrind
with the VALGRIND_LIB environment variable set, where <dir> is the root
of the source tree (and must be an absolute path).  Eg:

  VALGRIND_LIB=~/grind/head4/.in_place ~/grind/head4/coregrind/valgrind

This allows you to compile and run with "make" instead of "make install",
saving you time.

Or, you can use the 'vg-in-place' script which does that for you.

I recommend compiling with "make --quiet" to further reduce the amount of
output spewed out during compilation, letting you actually see any errors,
warnings, etc.


Building a distribution tarball
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To build a distribution tarball from the valgrind sources:

  make dist

In addition to compiling, linking and packaging everything up, the command
will also attempt to build the documentation.

If you only want to test whether the generated tarball is complete and runs
regression tests successfully, building documentation is not needed.

  make dist BUILD_ALL_DOCS=no

If you insist on building documentation some embarrassing instructions
can be found in docs/README.


Running the regression tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To build and run all the regression tests, run "make [--quiet] regtest".

To run a subset of the regression tests, execute:

  perl tests/vg_regtest <name>

where <name> is a directory (all tests within will be run) or a single
.vgtest test file, or the name of a program which has a like-named .vgtest
file.  Eg:

  perl tests/vg_regtest memcheck
  perl tests/vg_regtest memcheck/tests/badfree.vgtest
  perl tests/vg_regtest memcheck/tests/badfree


Running the performance tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To build and run all the performance tests, run "make [--quiet] perf".

To run a subset of the performance suite, execute:

  perl perf/vg_perf <name>

where <name> is a directory (all tests within will be run) or a single
.vgperf test file, or the name of a program which has a like-named .vgperf
file.  Eg:

  perl perf/vg_perf perf/
  perl perf/vg_perf perf/bz2.vgperf
  perl perf/vg_perf perf/bz2

To compare multiple versions of Valgrind, use the --vg= option multiple
times.  For example, if you have two Valgrinds next to each other, one in
trunk1/ and one in trunk2/, from within either trunk1/ or trunk2/ do this to
compare them on all the performance tests:

  perl perf/vg_perf --vg=../trunk1 --vg=../trunk2 perf/


Debugging Valgrind with GDB
~~~~~~~~~~~~~~~~~~~~~~~~~~~
To debug the valgrind launcher program (<prefix>/bin/valgrind) just
run it under gdb in the normal way.

Debugging the main body of the valgrind code (and/or the code for
a particular tool) requires a bit more trickery but can be achieved
without too much problem by following these steps:

(1) Set VALGRIND_LAUNCHER to point to the valgrind executable.  Eg:

      export VALGRIND_LAUNCHER=/usr/local/bin/valgrind

    or for an uninstalled version in a source directory $DIR:

      export VALGRIND_LAUNCHER=$DIR/coregrind/valgrind

(2) Run gdb on the tool executable.  Eg:

      gdb /usr/local/lib/valgrind/ppc32-linux/lackey

    or

      gdb $DIR/.in_place/x86-linux/memcheck

(3) Do "handle SIGSEGV SIGILL nostop noprint" in GDB to prevent GDB from
    stopping on a SIGSEGV or SIGILL:

    (gdb) handle SIGILL SIGSEGV nostop noprint

(4) Set any breakpoints you want and proceed as normal for gdb. The
    macro VG_(FUNC) is expanded to vgPlain_FUNC, so If you want to set
    a breakpoint VG_(do_exec), you could do like this in GDB:

    (gdb) b vgPlain_do_exec

(5) Run the tool with required options (the --tool option is required
    for correct setup), e.g.

    (gdb) run --tool=lackey pwd

Steps (1)--(3) can be put in a .gdbinit file, but any directory names must
be fully expanded (ie. not an environment variable).

A different and possibly easier way is as follows:

(1) Run Valgrind as normal, but add the flag --wait-for-gdb=yes.  This
    puts the tool executable into a wait loop soon after it gains
    control.  This delays startup for a few seconds.

(2) In a different shell, do "gdb /proc/<pid>/exe <pid>", where
    <pid> you read from the output printed by (1).  This attaches
    GDB to the tool executable, which should be in the abovementioned
    wait loop.

(3) Do "cont" to continue.  After the loop finishes spinning, startup
    will continue as normal.  Note that comment (3) above re passing
    signals applies here too.


Self-hosting
~~~~~~~~~~~~
This section explains :
  (A) How to configure Valgrind to run under Valgrind.
      Such a setup is called self hosting, or outer/inner setup.
  (B) How to run Valgrind regression tests in a 'self-hosting' mode,
      e.g. to verify Valgrind has no bugs such as memory leaks.
  (C) How to run Valgrind performance tests in a 'self-hosting' mode,
      to analyse and optimise the performance of Valgrind and its tools.

(A) How to configure Valgrind to run under Valgrind:

(1) Check out 2 trees, "Inner" and "Outer".  Inner runs the app
    directly.  Outer runs Inner.

(2) Configure Inner with --enable-inner and build as usual.

(3) Configure Outer normally and build+install as usual.
    Note: You must use a "make install"-ed valgrind.
    Do *not* use vg-in-place for the Outer valgrind.

(4) Choose a very simple program (date) and try

    outer/.../bin/valgrind --sim-hints=enable-outer --trace-children=yes  \
       --smc-check=all-non-file \
       --run-libc-freeres=no --tool=cachegrind -v \
       inner/.../vg-in-place --vgdb-prefix=./inner --tool=none -v prog

If you omit the --trace-children=yes, you'll only monitor Inner's launcher
program, not its stage2. Outer needs --run-libc-freeres=no, as otherwise
it will try to find and run __libc_freeres in the inner, while libc is not
used by the inner. Inner needs --vgdb-prefix=./inner to avoid inner
gdbserver colliding with outer gdbserver.
Currently, inner does *not* use the client request
VALGRIND_DISCARD_TRANSLATIONS for the JITted code or the code patched for
translation chaining. So the outer needs --smc-check=all-non-file to
detect the modified code.

Debugging the whole thing might imply to use up to 3 GDB:
  * a GDB attached to the Outer valgrind, allowing
    to examine the state of Outer.
  * a GDB using Outer gdbserver, allowing to
    examine the state of Inner.
  * a GDB using Inner gdbserver, allowing to
    examine the state of prog.

The whole thing is fragile, confusing and slow, but it does work well enough
for you to get some useful performance data.  Inner has most of
its output (ie. those lines beginning with "==<pid>==") prefixed with a '>',
which helps a lot. However, when running regression tests in an Outer/Inner
setup, this prefix causes the reg test diff to fail. Give
--sim-hints=no-inner-prefix to the Inner to disable the production
of the prefix in the stdout/stderr output of Inner.

The allocators in coregrind/m_mallocfree.c and VEX/priv/main_util.h are
annotated with client requests so Memcheck can be used to find leaks
and use after free in an Inner Valgrind.

The Valgrind "big lock" is annotated with helgrind client requests
so Helgrind and DRD can be used to find race conditions in an Inner
Valgrind.

All this has not been tested much, so don't be surprised if you hit problems.

When using self-hosting with an outer Callgrind tool, use '--pop-on-jump'
(on the outer). Otherwise, Callgrind has much higher memory requirements.

(B) Regression tests in an outer/inner setup:

 To run all the regression tests with an outer memcheck, do :
   perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
                         --all

 To run a specific regression tests with an outer memcheck, do:
   perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
                         none/tests/args.vgtest

 To run regression tests with another outer tool:
   perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
                         --outer-tool=helgrind --all

 --outer-args allows to give specific arguments to the outer tool,
 replacing the default one provided by vg_regtest.

Note: --outer-valgrind must be a "make install"-ed valgrind.
Do *not* use vg-in-place.

When an outer valgrind runs an inner valgrind, a regression test
produces one additional file <testname>.outer.log which contains the
errors detected by the outer valgrind.  E.g. for an outer memcheck, it
contains the leaks found in the inner, for an outer helgrind or drd,
it contains the detected race conditions.

The file tests/outer_inner.supp contains suppressions for
the irrelevant or benign errors found in the inner.

An regression test running in the inner (e.g. memcheck/tests/badrw) will
cause the inner to report an error, which is expected and checked
as usual when running the regtests in an outer/inner setup.
However, the outer will often also observe an error, e.g. a jump
using uninitialised data, or a read/write outside the bounds of a heap
block. When the outer reports such an error, it will output the
inner host stacktrace. To this stacktrace, it will append the
stacktrace of the inner guest program. For example, this is an error
reported by the outer when the inner runs the badrw regtest:
  ==8119== Invalid read of size 2
  ==8119==    at 0x7F2EFD7AF: ???
  ==8119==    by 0x7F2C82EAF: ???
  ==8119==    by 0x7F180867F: ???
  ==8119==    by 0x40051D: main (badrw.c:5)
  ==8119==    by 0x7F180867F: ???
  ==8119==    by 0x1BFF: ???
  ==8119==    by 0x3803B7F0: _______VVVVVVVV_appended_inner_guest_stack_VVVVVVVV_______ (m_execontext.c:332)
  ==8119==    by 0x40055C: main (badrw.c:22)
  ==8119==  Address 0x55cd03c is 4 bytes before a block of size 16 alloc'd
  ==8119==    at 0x2804E26D: vgPlain_arena_malloc (m_mallocfree.c:1914)
  ==8119==    by 0x2800BAB4: vgMemCheck_new_block (mc_malloc_wrappers.c:368)
  ==8119==    by 0x2800BC87: vgMemCheck_malloc (mc_malloc_wrappers.c:403)
  ==8119==    by 0x28097EAE: do_client_request (scheduler.c:1861)
  ==8119==    by 0x28097EAE: vgPlain_scheduler (scheduler.c:1425)
  ==8119==    by 0x280A7237: thread_wrapper (syswrap-linux.c:103)
  ==8119==    by 0x280A7237: run_a_thread_NORETURN (syswrap-linux.c:156)
  ==8119==    by 0x3803B7F0: _______VVVVVVVV_appended_inner_guest_stack_VVVVVVVV_______ (m_execontext.c:332)
  ==8119==    by 0x4C294C4: malloc (vg_replace_malloc.c:298)
  ==8119==    by 0x40051D: main (badrw.c:5)
In the above, the first stacktrace starts with the inner host stacktrace,
which in this case is some JITted code. Such code sometimes contains IPs
that points in the inner guest code (0x40051D: main (badrw.c:5)).
After the separator, we have the inner guest stacktrace.
The second stacktrace gives the stacktrace where the heap block that was
overrun was allocated. We see it was allocated by the inner valgrind
in the client arena (first part of the stacktrace). The second part is
the guest stacktrace that did the allocation.


(C) Performance tests in an outer/inner setup:

 To run all the performance tests with an outer cachegrind, do :
    perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf

 To run a specific perf test (e.g. bz2) in this setup, do :
    perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf/bz2

 To run all the performance tests with an outer callgrind, do :
    perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \
                      --outer-tool=callgrind perf

Note: --outer-valgrind must be a "make install"-ed valgrind.
Do *not* use vg-in-place.

 To compare the performance of multiple Valgrind versions, do :
    perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \
      --outer-tool=callgrind \
      --vg=../inner_xxxx --vg=../inner_yyyy perf
  (where inner_xxxx and inner_yyyy are the toplevel directories of
  the versions to compare).
  Cachegrind and cg_diff are particularly handy to obtain a delta
  between the two versions.

When the outer tool is callgrind or cachegrind, the following
output files will be created for each test:
   <outertoolname>.out.<inner_valgrind_dir>.<tt>.<perftestname>.<pid>
   <outertoolname>.outer.log.<inner_valgrind_dir>.<tt>.<perftestname>.<pid>
 (where tt is the two letters abbreviation for the inner tool(s) run).

For example, the command
    perl perf/vg_perf \
      --outer-valgrind=../outer_trunk/install/bin/valgrind \
      --outer-tool=callgrind \
      --vg=../inner_tchain --vg=../inner_trunk perf/many-loss-records

produces the files
    callgrind.out.inner_tchain.no.many-loss-records.18465
    callgrind.outer.log.inner_tchain.no.many-loss-records.18465
    callgrind.out.inner_tchain.me.many-loss-records.21899
    callgrind.outer.log.inner_tchain.me.many-loss-records.21899
    callgrind.out.inner_trunk.no.many-loss-records.21224
    callgrind.outer.log.inner_trunk.no.many-loss-records.21224
    callgrind.out.inner_trunk.me.many-loss-records.22916
    callgrind.outer.log.inner_trunk.me.many-loss-records.22916


Printing out problematic blocks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you want to print out a disassembly of a particular block that
causes a crash, do the following.

Try running with "--vex-guest-chase-thresh=0 --trace-flags=10000000
--trace-notbelow=999999".  This should print one line for each block
translated, and that includes the address.

Then re-run with 999999 changed to the highest bb number shown.
This will print the one line per block, and also will print a
disassembly of the block in which the fault occurred.

This file documents various "processes" that are used by Valgrind
developers for development and release activities.
This file contains one section for each process.
A summary of each process is given here. Each process is described
more in details afterwards.


* Update of the NEWS file: NEWS describes fixed bugs and new features.
  It is updated and committed together with the code fixing the bug/implementing
  the feature.

* Major release production:
  See docs/internals/release-HOWTO.txt (currently a bit out of date)

* Various guidelines/recommended usage for valgrind GIT
  See docs/internals/git-HOWTO.txt

* Minor/correction release production: TBD


Processes detailed descriptions:

Update of the NEWS file.
========================
  The NEWS file gives for each release:
    - the list of fixed bugs,
    - a short description of each functional change,
    - a short description of each technical change impacting the users.

  The update of the NEWS file should be committed together with the
  code change (or as part of the last committed change) that fixes the
  bug or implements the new feature/technical change.
  The documentation (e.g. user manual) should also be committed as part of
  the code change.

  Fixing a bug
  ------------
  When fixing a bug, add a line in the 'FIXED BUGS' section of
  the NEWS file.  Keep the list of bugs sorted by bugzilla entry number.

  Once you have commit the change, update the bug status in bugzilla,
  adding in the comment the revision number of the commit fixing the bug.

  If a bug is not entered in bugzilla (not encouraged), use "n-i-bz"
  and add the bug line at the end of the bug list.

  The file docs/internals/X_Y_BUGSTATUS.txt (where X_Y is the last
  major release e.g. 3_9) contains information/statuses for some bugs.
  If a bug is fixed, remove the (possible) bug info from this file.

  Implementing a change
  ---------------------
  When implementing a functional or 'user impacting' technical change,
  add a short description of the change in the relevant sub-section
  (e.g. TOOL CHANGES, PLATFORM CHANGES, ...).


  Some special cases:
  -------------------
  Some changes or bug fixes are very big and might be implemented over
  a significant period. In such a case, update the NEWS as part of the
  final commit.
  If relevant, you might already update the NEWS file as part of
  earlier commits, using the word 'PARTIAL' to indicate that the change or
  bug fix is not complete yet.

  Some bugs are reported more than once in bugzilla.
  Also document in NEWS that such duplicated bugs have been fixed, using a line
  such as:
     308333 == 307106
  to indicate that the bug 308333 is a duplicate of 307106, and was thus
  fixed in the commit that fixed 307106.
  Change also the status of the duplicated bug  in bugzilla,
  indicating in the comment the commit revision that fixed the 'master bug'.

So, install ports for autoconf, automake and gmake.
$ sh autogen.sh
$ ./configure --prefix=/where/ever
$ gmake
$ gmake install

Sun Aug 19 20:26:48 UTC 2007 PS_STRINGS
	Valgrind barfs all over the place on setproctitle.

	This also manifests itself in a corrupted environment in
	children of child processes when --trace-children=yes is used.

	To cope correctly Valgrind must install the modified argv/envp
	pointers in the ps-strings area, and mark it as accessible.

Dealing with missing system call or ioctl wrappers in Valgrind
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You're probably reading this because Valgrind bombed out whilst
running your program, and advised you to read this file.  The good
news is that, in general, it's easy to write the missing syscall or
ioctl wrappers you need, so that you can continue your debugging.  If
you send the resulting patches to me, then you'll be doing a favour to
all future Valgrind users too.

Note that an "ioctl" is just a special kind of system call, really; so
there's not a lot of need to distinguish them (at least conceptually)
in the discussion that follows.

All this machinery is in coregrind/m_syswrap.


What are syscall/ioctl wrappers?  What do they do?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Valgrind does what it does, in part, by keeping track of everything your
program does.  When a system call happens, for example a request to read
part of a file, control passes to the Linux kernel, which fulfills the
request, and returns control to your program.  The problem is that the
kernel will often change the status of some part of your program's memory
as a result, and tools (instrumentation plug-ins) may need to know about
this.

Syscall and ioctl wrappers have two jobs:

1. Tell a tool what's about to happen, before the syscall takes place.  A
   tool could perform checks beforehand, eg. if memory about to be written
   is actually writeable.  This part is useful, but not strictly
   essential.

2. Tell a tool what just happened, after a syscall takes place.  This is
   so it can update its view of the program's state, eg. that memory has
   just been written to.  This step is essential.

The "happenings" mostly involve reading/writing of memory.

So, let's look at an example of a wrapper for a system call which
should be familiar to many Unix programmers.


The syscall wrapper for time()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The wrapper for the time system call looks like this:

  PRE(sys_time)
  {
     /* time_t time(time_t *t); */
     PRINT("sys_time ( %p )",ARG1);
     PRE_REG_READ1(long, "time", int *, t);
     if (ARG1 != 0) {
        PRE_MEM_WRITE( "time(t)", ARG1, sizeof(vki_time_t) );
     }
  }

  POST(sys_time)
  {
     if (ARG1 != 0) {
        POST_MEM_WRITE( ARG1, sizeof(vki_time_t) );
     }
  }

The first thing we do happens before the syscall occurs, in the PRE() function.
The PRE() function typically starts with invoking to the PRINT() macro. This
PRINT() macro implements support for the --trace-syscalls command line option.
Next, the tool is told the return type of the syscall, that the syscall has
one argument, the type of the syscall argument and that the argument is being
read from a register:

     PRE_REG_READ1(long, "time", int *, t);

Next, if a non-NULL buffer is passed in as the argument, tell the tool that the
buffer is about to be written to:

     if (ARG1 != 0) {
        PRE_MEM_WRITE( "time", ARG1, sizeof(vki_time_t) );
     }

Finally, the really important bit, after the syscall occurs, in the POST()
function:  if, and only if, the system call was successful, tell the tool that
the memory was written:

     if (ARG1 != 0) {
        POST_MEM_WRITE( ARG1, sizeof(vki_time_t) );
     }

The POST() function won't be called if the syscall failed, so you
don't need to worry about checking that in the POST() function.
(Note: this is sometimes a bug; some syscalls do return results when
they "fail" - for example, nanosleep returns the amount of unslept
time if interrupted. TODO: add another per-syscall flag for this
case.)

Note that we use the type 'vki_time_t'.  This is a copy of the kernel
type, with 'vki_' prefixed.  Our copies of such types are kept in the
appropriate vki*.h file(s).  We don't include kernel headers or glibc headers
directly.


Writing your own syscall wrappers (see below for ioctl wrappers)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If Valgrind tells you that system call NNN is unimplemented, do the
following:

1.  Find out the name of the system call:

       grep NNN /usr/include/asm/unistd*.h

    This should tell you something like  __NR_mysyscallname.
    Copy this entry to include/vki/vki-scnums-$(VG_PLATFORM).h.


2.  Do 'man 2 mysyscallname' to get some idea of what the syscall
    does.  Note that the actual kernel interface can differ from this,
    so you might also want to check a version of the Linux kernel
    source.

    NOTE: any syscall which has something to do with signals or
    threads is probably "special", and needs more careful handling.
    Post something to valgrind-developers if you aren't sure.


3.  Add a case to the already-huge collection of wrappers in
    the coregrind/m_syswrap/syswrap-*.c files.
    For each in-memory parameter which is read or written by
    the syscall, do one of

      PRE_MEM_READ( ... )
      PRE_MEM_RASCIIZ( ... )
      PRE_MEM_WRITE( ... )

    for  that parameter.  Then do the syscall.  Then, if the syscall
    succeeds, issue suitable POST_MEM_WRITE( ... ) calls.
    (There's no need for POST_MEM_READ calls.)

    Also, add it to the syscall_table[] array; use one of GENX_, GENXY
    LINX_, LINXY, PLAX_, PLAXY.
    GEN* for generic syscalls (in syswrap-generic.c), LIN* for linux
    specific ones (in syswrap-linux.c) and PLA* for the platform
    dependent ones (in syswrap-$(PLATFORM)-linux.c).
    The *XY variant if it requires a PRE() and POST() function, and
    the *X_ variant if it only requires a PRE()
    function.

    If you find this difficult, read the wrappers for other syscalls
    for ideas.  A good tip is to look for the wrapper for a syscall
    which has a similar behaviour to yours, and use it as a
    starting point.

    If you need structure definitions and/or constants for your syscall,
    copy them from the kernel headers into include/vki.h and co., with
    the appropriate vki_*/VKI_* name mangling.  Don't #include any
    kernel headers.  And certainly don't #include any glibc headers.

    Test it.

    Note that a common error is to call POST_MEM_WRITE( ... )
    with 0 (NULL) as the first (address) argument.  This usually means
    your logic is slightly inadequate.  It's a sufficiently common bug
    that there's a built-in check for it, and you'll get a "probably
    sanity check failure" for the syscall wrapper you just made, if this
    is the case.


4.  Once happy, send us the patch.  Pretty please.




Writing your own ioctl wrappers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Is pretty much the same as writing syscall wrappers, except that all
the action happens within PRE(ioctl) and POST(ioctl).

There's a default case, sometimes it isn't correct and you have to write a
more specific case to get the right behaviour.

As above, please create a bug report and attach the patch as described
on http://www.valgrind.org.


Writing your own door call wrappers (Solaris only)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Unlike syscalls or ioctls, door calls transfer data between two userspace
programs, albeit through a kernel interface. Programs may use completely
proprietary semantics in the data buffers passed between them.
Therefore it may not be possible to capture these semantics within
a Valgrind door call or door return wrapper.

Nevertheless, for system or well-known door services it would be beneficial
to have a door call and a door return wrapper. Writing such wrapper is pretty
much the same as writing ioctl wrappers. Please take a few moments to study
the following picture depicting how a door client and a door server interact
through the kernel interface in a typical scenario:


door client thread          kernel       door server thread
invokes door_call()                     invokes door_return()
-------------------------------------------------------------------
                               <----  PRE(sys_door, DOOR_RETURN)
PRE(sys_door, DOOR_CALL)  --->
                               ---->  POST(sys_door, DOOR_RETURN)
                                           ----> server_procedure()
                                           <----
                               <----  PRE(sys_door, DOOR_RETURN)
POST(sys_door, DOOR_CALL) <---

The first PRE(sys_door, DOOR_RETURN) is invoked with data_ptr=NULL
and data_size=0. That's because it has not received any data from
a door call, yet.

Semantics are described by the following functions
in coregring/m_syswrap/syswrap-solaris.c module:
o For a door call wrapper the following attributes of 'params' argument:
  - data_ptr (and associated data_size) as input buffer (request);
      described in door_call_pre_mem_params_data()
  - rbuf (and associated rsize) as output buffer (response);
      described in door_call_post_mem_params_rbuf()
o For a door return wrapper the following parameters:
  - data_ptr (and associated data_size) as input buffer (request);
      described in door_return_post_mem_data()
  - data_ptr (and associated data_size) as output buffer (response);
      described in door_return_pre_mem_data()

There's a default case which may not be correct and you have to write a
more specific case to get the right behaviour. Unless Valgrind's option
'--sim-hints=lax-doors' is specified, the default case also spits a warning.

As above, please create a bug report and attach the patch as described
on http://www.valgrind.org.

Greetings, packaging person!  This information is aimed at people
building binary distributions of Valgrind.

Thanks for taking the time and effort to make a binary distribution of
Valgrind.  The following notes may save you some trouble.

-- If your toolchain (compiler, linker) support lto, using the configure
   option --enable-lto=yes will produce a smaller/faster valgrind
   (up to 10%).

-- Do not ship your Linux distro with a completely stripped
   /lib/ld.so.  At least leave the debugging symbol names on -- line
   number info isn't necessary.  If you don't want to leave symbols on
   ld.so, alternatively you can have your distro install ld.so's
   debuginfo package by default, or make ld.so.debuginfo be a
   requirement of your Valgrind RPM/DEB/whatever.

   Reason for this is that Valgrind's Memcheck tool needs to intercept
   calls to, and provide replacements for, some symbols in ld.so at
   startup (most importantly strlen).  If it cannot do that, Memcheck
   shows a large number of false positives due to the highly optimised
   strlen (etc) routines in ld.so.  This has caused some trouble in
   the past.  As of version 3.3.0, on some targets (ppc32-linux,
   ppc64-linux), Memcheck will simply stop at startup (and print an
   error message) if such symbols are not present, because it is
   infeasible to continue.

   It's not like this is going to cost you much space.  We only need
   the symbols for ld.so (a few K at most).  Not the debug info and
   not any debuginfo or extra symbols for any other libraries.


-- (Unfortunate but true) When you configure to build with the
   --prefix=/foo/bar/xyzzy option, the prefix /foo/bar/xyzzy gets
   baked into valgrind.  The consequence is that you _must_ install
   valgrind at the location specified in the prefix.  If you don't,
   it may appear to work, but will break doing some obscure things,
   particularly doing fork() and exec().

   So you can't build a relocatable RPM / whatever from Valgrind.


-- Don't strip the debug info off lib/valgrind/$platform/vgpreload*.so
   in the installation tree.  Either Valgrind won't work at all, or it
   will still work if you do, but will generate less helpful error
   messages.  Here's an example:

   Mismatched free() / delete / delete []
      at 0x40043249: free (vg_clientfuncs.c:171)
      by 0x4102BB4E: QGArray::~QGArray(void) (tools/qgarray.cpp:149)
      by 0x4C261C41: PptDoc::~PptDoc(void) (include/qmemarray.h:60)
      by 0x4C261F0E: PptXml::~PptXml(void) (pptxml.cc:44)
      Address 0x4BB292A8 is 0 bytes inside a block of size 64 alloc'd
      at 0x4004318C: __builtin_vec_new (vg_clientfuncs.c:152)
      by 0x4C21BC15: KLaola::readSBStream(int) const (klaola.cc:314)
      by 0x4C21C155: KLaola::stream(KLaola::OLENode const *) (klaola.cc:416)
      by 0x4C21788F: OLEFilter::convert(QCString const &) (olefilter.cc:272)

   This tells you that some memory allocated with new[] was freed with
   free().

   Mismatched free() / delete / delete []
      at 0x40043249: (inside vgpreload_memcheck.so)
      by 0x4102BB4E: QGArray::~QGArray(void) (tools/qgarray.cpp:149)
      by 0x4C261C41: PptDoc::~PptDoc(void) (include/qmemarray.h:60)
      by 0x4C261F0E: PptXml::~PptXml(void) (pptxml.cc:44)
      Address 0x4BB292A8 is 0 bytes inside a block of size 64 alloc'd
      at 0x4004318C: (inside vgpreload_memcheck.so)
      by 0x4C21BC15: KLaola::readSBStream(int) const (klaola.cc:314)
      by 0x4C21C155: KLaola::stream(KLaola::OLENode const *) (klaola.cc:416)
      by 0x4C21788F: OLEFilter::convert(QCString const &) (olefilter.cc:272)

   This isn't so helpful.  Although you can tell there is a mismatch,
   the names of the allocating and deallocating functions are no longer
   visible.  The same kind of thing occurs in various other messages
   from valgrind.


-- Don't strip symbols from lib/valgrind/* in the installation tree.
   Doing so will likely cause problems.  Removing the line number info is
   probably OK (at least for some of the files in that directory), although
   that has not been tested by the Valgrind developers.


-- Please test the final installation works by running it on something
   huge.  I suggest checking that it can start and exit successfully
   both Firefox and OpenOffice.org.  I use these as test programs, and I
   know they fairly thoroughly exercise Valgrind.  The command lines to use
   are:

   valgrind -v --trace-children=yes firefox

   valgrind -v --trace-children=yes soffice


If you find any more hints/tips for packaging, please report
it as a bugreport. See http://www.valgrind.org for details.