1\documentclass[dvipdfm,11pt]{article} 2\usepackage[dvipdfm]{hyperref} % Upgraded url package 3\parskip=.1in 4 5\begin{document} 6\markright{MPICH Logging} 7\title{MPICH Logging\\ 8Version 0.1\\ 9DRAFT of \today\\ 10Mathematics and Computer Science Division\\ 11Argonne National Laboratory} 12 13\author{David Ashton} 14 15 16\maketitle 17 18\cleardoublepage 19 20\pagenumbering{arabic} 21\pagestyle{headings} 22 23 24\section{Introduction} 25\label{sec:introduction} 26 27This manual assumes that MPICH has already been installed. For 28instructions on how to install MPICH, see the MPICH Installer's Guide, 29or the README in the top-level MPICH directory. This manual will 30explain how the internal logging macros are generated and how the user 31can generate log files viewable in Jumpshot. 32 33 34\section{Configuring mpich to create log files} 35\label{sec:configuring} 36 37When users run configure they can specify logging options. There are three 38configure options to control logging. 39 40\begin{description} 41\item[\texttt{--enable-timing=<timing\_type>}]\mbox{}\\ 42Add this option to enable timing. The two options for timing\_type are 43\texttt{log} and \texttt{log\_detailed}. The \texttt{log} option will log 44only the MPI functions. The 45\texttt{log\_detailed} will log every function in mpich. This option gives 46fine grained logging information and also creates large log files. It must 47be used in conjunction with a timer-type that can log very short intervals 48on the order of 100's of nanoseconds. 49 50\item[\texttt{--with-logging=<logger>}]\mbox{}\\ 51Specify the logging library to use. Currently the only logger option is \texttt{rlog}. 52 53\item[\texttt{--enable-timer-type=<timer\_type>}]\mbox{}\\ 54Specify the timer type. The options are 55\begin{itemize} 56\item \texttt{gethrtime} - 57Solaris timer (Solaris systems only) 58\item \texttt{clock\_gettime} - 59Posix timer (where available) 60\item \texttt{gettimeofday} - 61Most Unix systems 62\item \texttt{linux86\_cycle} - 63Linux x86 cycle counter* 64\item \texttt{linuxalpha\_cycle} - 65Like linux86\_cycle, but for Linux Alpha* 66\item \texttt{gcc\_ia64\_cycle} - 67IA64 cycle counter* 68\end{itemize} 69* Note that CPU cycle counters count cycles, not elapsed time. 70Because processor frequencies are variable, especially with modern 71power-aware hardware, these are not always reliable for timing and so 72should only be used if you're sure you know what you're doing. 73 74\end{description} 75 76Here is an example: 77\begin{verbatim} 78mpich/configure 79 --enable-timing=log 80 --with-logging=rlog 81 --enable-timer-type=gettimeofday 82 ... 83\end{verbatim} 84 85\section{Generating log files} 86\label{sec:genlogs} 87Run your mpi application to create intermediate \texttt{.irlog} files. 88 89\begin{verbatim} 90mpicc myapp.c -o myapp 91mpiexec -n 3 myapp 92\end{verbatim} 93There will be .irlog files created for each process: 94\begin{verbatim} 95log0.irlog 96log1.irlog 97log2.irlog 98\end{verbatim} 99 100\section{RLOG tools} 101\label{sec:tools} 102For performance reasons each process produces a local intermediate log file 103that needs to be merged into a single rlog file. Use the rlog tools to merge 104the \texttt{.irlog} files into an \texttt{.rlog} file. The rlog tools are 105found in \texttt{mpich\_build/src/util/logging/rlog}. Currently they are not 106copied to the install directory. 107 108\begin{description} 109\item[\texttt{irlog2rlog}]\mbox{}\\ 110This tool combines the intermediate \texttt{.irlog} files into a single 111\texttt{.rlog} file. The usage is: ``\texttt{irlog2rlog outname.rlog 112input0.irlog input1.irlog ...}'' A shortcut is provided: ``\texttt{irlog2rlog 113outname.rlog <num\_files>}''. Execute \texttt{irlog2rlog} without any 114parameters to see the usage options. 115 116\item[\texttt{printrlog}]\mbox{}\\ 117This tool prints the contents of an \texttt{.rlog} file. 118 119\item[\texttt{printirlog}]\mbox{}\\ 120This tool prints the contents of an \texttt{.irlog} file. 121\end{description} 122 123Continuing the example from the previous section: 124\begin{verbatim} 125irlog2rlog myapp.rlog 3 126\end{verbatim} 127will convert \texttt{log0.irlog}, \texttt{log1.irlog} and \texttt{log2.irlog} 128to \texttt{myapp.rlog}. 129 130\section{Viewing log files} 131This section describes how to view a log file 132 133\texttt{.rlog} files can be printed from a command shell using the 134\texttt{printrlog} tool but the more interesting way to view the log files 135is from Jumpshot. Jumpshot displays slog2 files and has a built in converter 136to convert \texttt{.rlog} files to \texttt{.slog2} files. Start Jumpshot and 137open your \texttt{.rlog} file. Jumpshot will ask you if you want to convert 138the file and you say yes. 139 140\section{Logging state code generation} 141\label{sec:genstates} 142 143This section can be skipped by users. It describes the internal scripts used 144to develop the logging macros. 145 146This is how the \texttt{maint/genstates} script works: 147 148\begin{enumerate} 149\item \texttt{autogen.sh} creates \texttt{genstates} from 150\texttt{genstates.in} replacing \texttt{@PERL@} with the appropriate path to 151perl and then runs \texttt{genstates}. 152 153\item \texttt{genstates} finds all \texttt{.i}, \texttt{.h} and \texttt{.c} files 154in the mpich directory tree, searches for \texttt{\_STATE\_DECL} in each 155file and builds a list of all the MPID\_STATEs. It validates that the states 156start in a \texttt{\_STATE\_DECL} statement, followed by a \texttt{FUNC\_ENTER} 157statement, and then at least one \texttt{FUNC\_EXIT} statement. Errors are printed 158out if the code does not follow this format except for macros. State declarations 159in macros are assumed to be correct. 160 161\item \texttt{genstates} finds all the \texttt{describe\_states.txt} 162files anywhere in the mpich tree. \texttt{describe\_states.txt} files are 163optional and are used to set the output name of the state and its associated 164color. 165 166\item The \texttt{describe\_states.txt} file format is this: 167\begin{verbatim} 168MPID_STATE_XXX <user string for the state> <optional rgb color> 169\end{verbatim} 170 Here is an example line: 171\begin{verbatim} 172 MPID_STATE_MPI_SEND MPI_Send 0 0 255 173\end{verbatim} 174If you don't specify a state in a \texttt{describe\_states.txt} file then 175the state user name will be automatically created by stripping off the 176\texttt{MPID\_STATE\_} prefix and the color will be assigned a random value. 177 178\item \texttt{genstates} ouputs \texttt{mpich/src/include/mpiallstates.h} 179with this \texttt{enum} in it: 180\begin{verbatim} 181enum MPID_TIMER_STATE 182{ 183 MPID_STATE_XXX, 184 ... 185}; 186\end{verbatim} 187 188\item \texttt{genstates} outputs 189\texttt{mpich/src/util/logging/describe\_states.c} with the 190\texttt{MPIR\_Describe\_timer\_states()} function in it. Currently, only 191the rlog version of \texttt{MPIR\_Describe\_timer\_states()} is generated. 192 193\end{enumerate} 194 195\end{document} 196