1
2\section{Use of TCGMSG global operation routines}
3
4In some cases (notably workstation clusters) the global array tools
5use a ``data-server'' process on each node in addition to the compute
6processes.  Data-server processes don't follow the same flow of
7execution of compute processes, so TCGMSG global operations
8(\verb+brdcst+, \verb+igop+, and \verb+dgop+) will hang when invoked.
9The global array toolkit provides ``wrapper'' functions
10(\verb+ga_brdcst+, \verb+ga_igop+, and \verb+ga_dgop+) which properly
11exclude data server processes from the global communication and must
12be used instead of the corresponding TCGMSG functions.
13
14\section{Interaction between GA and message-passing}
15
16The limited buffering available on the IBM SP-1/2 means that GA and
17message-passing operations cannot interleave as readily as they do on
18other machines.  Basically, in transitioning from GA to message
19passing or vice versa the application must call {\tt ga\_sync()}.
20
21%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
22\section{The memory allocator --- MA}
23%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
24
25\sloppy
26For detailed information on routines see the MA man pages by adding
27{\tt \$(NWCHEM\_TOP)/src/man/ma/man} % $
28to your {\tt MANPATH}.
29
30\fussy
31
32MA is a library of routines that comprises a dynamic memory allocator
33for use by C, FORTRAN, or mixed-language applications.  FORTRAN
34applications require such a library because the language does not
35support dynamic memory allocation.  C applications can benefit from
36using MA instead of the ordinary {\tt malloc()} and {\tt free()}
37routines because of the extra features MA provides: both heap and
38stack memory management disciplines, debugging and verification
39support, usage statistics, and quantitative memory availability
40information.  MA is designed to be portable across a large variety of
41platforms.
42
43\section{Typing}
44
45\section{MA data types}
46
47All MA memory is typed.  Data is allocated in units of integer,
48logical, double precision, etc., words.  The type of data is specified
49in arguments using predefined Fortran parameters (or macros in C).
50\begin{description}
51\item{\verb+MT_INT+} --- integer
52\item{\verb+MT_DBL+} --- double precision
53\item{\verb+MT_LOG+} --- logical
54\item{\verb+MT_CHAR+} --- character\verb+*+1
55\end{description}
56
57\section{List of routines}
58
59All MA routines are shown below, grouped by category and listed
60alphabetically within each category.  The FORTRAN interface is given
61({\em or the plan is to include it eventually}),
62refer to the man pages for the C interface or information on the
63arguments.
64
65Initialization:
66\begin{itemize}
67\item {\tt MA\_init(datatype, nominal\_stack, nominal\_heap)}
68\item {\tt MA\_sizeof(datatype1, nelem1, datatype2)}
69\item {\tt MA\_sizeof\_overhead(datatype)}
70\end{itemize}
71
72Allocation:
73\begin{itemize}
74\item {\tt MA\_alloc\_get(datatype, nelem, name, memhandle, index)}
75\item {\tt MA\_allocate\_heap(datatype, nelem, name, memhandle)}
76\item {\tt MA\_get\_index(memhandle, index)}
77\item {\tt MA\_get\_pointer()} --- C only
78\item {\tt MA\_inquire\_avail(datatype)}
79\item {\tt MA\_inquire\_heap(datatype)}
80\item {\tt MA\_inquire\_stack(datatype)}
81\item {\tt MA\_push\_get(datatype, nelem, name, memhandle, index)}
82\item {\tt MA\_push\_stack(datatype, nelem, name, memhandle)}
83\end{itemize}
84
85Deallocation:
86\begin{itemize}
87\item {\tt MA\_chop\_stack(memhandle)}
88\item {\tt MA\_free\_heap(memhandle)}
89\item {\tt MA\_pop\_stack(memhandle)}
90\end{itemize}
91
92Debugging:
93\begin{itemize}
94\item {\tt MA\_set\_auto\_verify()}
95\item {\tt MA\_set\_error\_print()}
96\item {\tt MA\_set\_hard\_fail()}
97\item {\tt MA\_summarize\_allocated\_blocks()}
98\item {\tt MA\_verify\_allocator\_stuff()}
99\end{itemize}
100
101Iteration Over Allocated Blocks:
102\begin{itemize}
103\item {\tt MA\_get\_next\_memhandle(ithandle, memhandle)}
104\item {\tt MA\_init\_memhandle\_iterator(ithandle)}
105\end{itemize}
106
107Statistics:
108\begin{itemize}
109\item {\tt MA\_print\_stats(oprintroutines)}
110\end{itemize}
111
112
113\section{Errors}
114
115Errors considered fatal by MA result in program termination.  Errors
116considered nonfatal by MA cause the MA routine to return an error
117value to the caller.  For most boolean functions, false is returned
118upon failure and true is returned upon success.  (The boolean
119functions for which the return value means something other than
120success or failure are {\tt MA\_set\_auto\_verify()}, {\tt
121  MA\_set\_error\_print()}, and {\tt MA\_set\_hard\_fail()}.)  Integer
122functions return zero upon failure; depending on the function, zero
123may or may not be distinguishable as an exceptional value.
124
125An application can force MA to treat all errors as fatal via
126{\tt MA\_set\_hard\_fail()}.
127
128If a fatal error occurs, an error message is printed on the standard
129error (stderr).  By default, error messages are also printed for
130nonfatal errors.  An application can force MA to print or not print
131error messages for nonfatal errors via {\tt MA\_set\_error\_print()}.
132
133\section{Files}
134
135To access required MA definitions, C applications should include
136{\tt macdecls.h} and FORTRAN applications should include
137{\tt mafdecls.fh}.
138
139\section{Implementation}
140
141Memory layout definitions:
142\begin{itemize}
143\item segment = heap\_region stack\_region
144\item region = block block block \ldots
145\item block = AD gap1 guard1 client\_space guard2 gap2
146\end{itemize}
147
148A segment of memory is obtained from the OS upon initialization.  The
149low end of the segment is managed as a heap; the heap region grows
150from low addresses to high addresses.  The high end of the segment is
151managed as a stack; the stack region grows from high addresses to low
152addresses.
153
154Each region consists of a series of contiguous blocks, one per
155allocation request, and possibly some unused space.  Blocks in the
156heap region are either in use by the client (allocated and not yet
157deallocated) or not in use by the client (allocated and already
158deallocated).  A block on the rightmost end of the heap region becomes
159part of the unused space upon deallocation.  Blocks in the stack
160region are always in use by the client, because when a stack block is
161deallocated, it becomes part of the unused space.
162
163A block consists of the client space, i.e., the range of memory
164available for use by the application; guard words adjacent to each end
165of the client space to help detect improper memory access by the
166client; bookkeeping info (in an "allocation descriptor," AD); and two
167gaps, each zero or more bytes long, to satisfy alignment constraints
168(specifically, to ensure that AD and client\_space are aligned
169properly).
170
171
172