1 2\section{Use of TCGMSG global operation routines} 3 4In some cases (notably workstation clusters) the global array tools 5use a ``data-server'' process on each node in addition to the compute 6processes. Data-server processes don't follow the same flow of 7execution of compute processes, so TCGMSG global operations 8(\verb+brdcst+, \verb+igop+, and \verb+dgop+) will hang when invoked. 9The global array toolkit provides ``wrapper'' functions 10(\verb+ga_brdcst+, \verb+ga_igop+, and \verb+ga_dgop+) which properly 11exclude data server processes from the global communication and must 12be used instead of the corresponding TCGMSG functions. 13 14\section{Interaction between GA and message-passing} 15 16The limited buffering available on the IBM SP-1/2 means that GA and 17message-passing operations cannot interleave as readily as they do on 18other machines. Basically, in transitioning from GA to message 19passing or vice versa the application must call {\tt ga\_sync()}. 20 21%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 22\section{The memory allocator --- MA} 23%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 24 25\sloppy 26For detailed information on routines see the MA man pages by adding 27{\tt \$(NWCHEM\_TOP)/src/man/ma/man} % $ 28to your {\tt MANPATH}. 29 30\fussy 31 32MA is a library of routines that comprises a dynamic memory allocator 33for use by C, FORTRAN, or mixed-language applications. FORTRAN 34applications require such a library because the language does not 35support dynamic memory allocation. C applications can benefit from 36using MA instead of the ordinary {\tt malloc()} and {\tt free()} 37routines because of the extra features MA provides: both heap and 38stack memory management disciplines, debugging and verification 39support, usage statistics, and quantitative memory availability 40information. MA is designed to be portable across a large variety of 41platforms. 42 43\section{Typing} 44 45\section{MA data types} 46 47All MA memory is typed. Data is allocated in units of integer, 48logical, double precision, etc., words. The type of data is specified 49in arguments using predefined Fortran parameters (or macros in C). 50\begin{description} 51\item{\verb+MT_INT+} --- integer 52\item{\verb+MT_DBL+} --- double precision 53\item{\verb+MT_LOG+} --- logical 54\item{\verb+MT_CHAR+} --- character\verb+*+1 55\end{description} 56 57\section{List of routines} 58 59All MA routines are shown below, grouped by category and listed 60alphabetically within each category. The FORTRAN interface is given 61({\em or the plan is to include it eventually}), 62refer to the man pages for the C interface or information on the 63arguments. 64 65Initialization: 66\begin{itemize} 67\item {\tt MA\_init(datatype, nominal\_stack, nominal\_heap)} 68\item {\tt MA\_sizeof(datatype1, nelem1, datatype2)} 69\item {\tt MA\_sizeof\_overhead(datatype)} 70\end{itemize} 71 72Allocation: 73\begin{itemize} 74\item {\tt MA\_alloc\_get(datatype, nelem, name, memhandle, index)} 75\item {\tt MA\_allocate\_heap(datatype, nelem, name, memhandle)} 76\item {\tt MA\_get\_index(memhandle, index)} 77\item {\tt MA\_get\_pointer()} --- C only 78\item {\tt MA\_inquire\_avail(datatype)} 79\item {\tt MA\_inquire\_heap(datatype)} 80\item {\tt MA\_inquire\_stack(datatype)} 81\item {\tt MA\_push\_get(datatype, nelem, name, memhandle, index)} 82\item {\tt MA\_push\_stack(datatype, nelem, name, memhandle)} 83\end{itemize} 84 85Deallocation: 86\begin{itemize} 87\item {\tt MA\_chop\_stack(memhandle)} 88\item {\tt MA\_free\_heap(memhandle)} 89\item {\tt MA\_pop\_stack(memhandle)} 90\end{itemize} 91 92Debugging: 93\begin{itemize} 94\item {\tt MA\_set\_auto\_verify()} 95\item {\tt MA\_set\_error\_print()} 96\item {\tt MA\_set\_hard\_fail()} 97\item {\tt MA\_summarize\_allocated\_blocks()} 98\item {\tt MA\_verify\_allocator\_stuff()} 99\end{itemize} 100 101Iteration Over Allocated Blocks: 102\begin{itemize} 103\item {\tt MA\_get\_next\_memhandle(ithandle, memhandle)} 104\item {\tt MA\_init\_memhandle\_iterator(ithandle)} 105\end{itemize} 106 107Statistics: 108\begin{itemize} 109\item {\tt MA\_print\_stats(oprintroutines)} 110\end{itemize} 111 112 113\section{Errors} 114 115Errors considered fatal by MA result in program termination. Errors 116considered nonfatal by MA cause the MA routine to return an error 117value to the caller. For most boolean functions, false is returned 118upon failure and true is returned upon success. (The boolean 119functions for which the return value means something other than 120success or failure are {\tt MA\_set\_auto\_verify()}, {\tt 121 MA\_set\_error\_print()}, and {\tt MA\_set\_hard\_fail()}.) Integer 122functions return zero upon failure; depending on the function, zero 123may or may not be distinguishable as an exceptional value. 124 125An application can force MA to treat all errors as fatal via 126{\tt MA\_set\_hard\_fail()}. 127 128If a fatal error occurs, an error message is printed on the standard 129error (stderr). By default, error messages are also printed for 130nonfatal errors. An application can force MA to print or not print 131error messages for nonfatal errors via {\tt MA\_set\_error\_print()}. 132 133\section{Files} 134 135To access required MA definitions, C applications should include 136{\tt macdecls.h} and FORTRAN applications should include 137{\tt mafdecls.fh}. 138 139\section{Implementation} 140 141Memory layout definitions: 142\begin{itemize} 143\item segment = heap\_region stack\_region 144\item region = block block block \ldots 145\item block = AD gap1 guard1 client\_space guard2 gap2 146\end{itemize} 147 148A segment of memory is obtained from the OS upon initialization. The 149low end of the segment is managed as a heap; the heap region grows 150from low addresses to high addresses. The high end of the segment is 151managed as a stack; the stack region grows from high addresses to low 152addresses. 153 154Each region consists of a series of contiguous blocks, one per 155allocation request, and possibly some unused space. Blocks in the 156heap region are either in use by the client (allocated and not yet 157deallocated) or not in use by the client (allocated and already 158deallocated). A block on the rightmost end of the heap region becomes 159part of the unused space upon deallocation. Blocks in the stack 160region are always in use by the client, because when a stack block is 161deallocated, it becomes part of the unused space. 162 163A block consists of the client space, i.e., the range of memory 164available for use by the application; guard words adjacent to each end 165of the client space to help detect improper memory access by the 166client; bookkeeping info (in an "allocation descriptor," AD); and two 167gaps, each zero or more bytes long, to satisfy alignment constraints 168(specifically, to ensure that AD and client\_space are aligned 169properly). 170 171 172