1%% 2%% 3 4\chapter{Bacula Memory Management} 5\label{blb:TheChapterStart7} 6\index{Management!Bacula Memory} 7\index{Bacula Memory Management} 8 9\section{General} 10\index{General} 11 12This document describes the memory management routines that are used in Bacula 13and is meant to be a technical discussion for developers rather than part of 14the user manual. 15 16Since Bacula may be called upon to handle filenames of varying and more or 17less arbitrary length, special attention needs to be used in the code to 18ensure that memory buffers are sufficiently large. There are four 19possibilities for memory usage within {\bf Bacula}. Each will be described in 20turn. They are: 21 22\begin{bitemize} 23\item Statically allocated memory. 24\item Dynamically allocated memory using malloc() and free(). 25\item Non-pooled memory. 26\item Pooled memory. 27 \end{bitemize} 28 29\subsection{Statically Allocated Memory} 30\index{Statically Allocated Memory} 31\index{Memory!Statically Allocated} 32 33Statically allocated memory is of the form: 34 35\footnotesize 36\begin{bVerbatim} 37char buffer[MAXSTRING]; 38\end{bVerbatim} 39\normalsize 40 41The use of this kind of memory is discouraged except when you are 100\% sure 42that the strings to be used will be of a fixed length. One example of where 43this is appropriate is for {\bf Bacula} resource names, which are currently 44limited to 127 characters (MAX\_NAME\_LENGTH). Although this maximum size may 45change, particularly to accommodate Unicode, it will remain a relatively small 46value. 47 48\subsection{Dynamically Allocated Memory} 49\index{Dynamically Allocated Memory} 50\index{Memory!Dynamically Allocated} 51 52Dynamically allocated memory is obtained using the standard malloc() routines. 53As in: 54 55\footnotesize 56\begin{bVerbatim} 57char *buf; 58buf = malloc(256); 59\end{bVerbatim} 60\normalsize 61 62This kind of memory can be released with: 63 64\footnotesize 65\begin{bVerbatim} 66free(buf); 67\end{bVerbatim} 68\normalsize 69 70It is recommended to use this kind of memory only when you are sure that you 71know the memory size needed and the memory will be used for short periods of 72time -- that is it would not be appropriate to use statically allocated 73memory. An example might be to obtain a large memory buffer for reading and 74writing files. When {\bf SmartAlloc} is enabled, the memory obtained by 75malloc() will automatically be checked for buffer overwrite (overflow) during 76the free() call, and all malloc'ed memory that is not released prior to 77termination of the program will be reported as Orphaned memory. 78 79\subsection{Pooled and Non-pooled Memory} 80\index{Memory!Pooled and Non-pooled} 81\index{Pooled and Non-pooled Memory} 82 83In order to facility the handling of arbitrary length filenames and to 84efficiently handle a high volume of dynamic memory usage, we have implemented 85routines between the C code and the malloc routines. The first is called 86``Pooled'' memory, and is memory, which once allocated and then released, is 87not returned to the system memory pool, but rather retained in a Bacula memory 88pool. The next request to acquire pooled memory will return any free memory 89block. In addition, each memory block has its current size associated with the 90block allowing for easy checking if the buffer is of sufficient size. This 91kind of memory would normally be used in high volume situations (lots of 92malloc()s and free()s) where the buffer length may have to frequently change 93to adapt to varying filename lengths. 94 95The non-pooled memory is handled by routines similar to those used for pooled 96memory, allowing for easy size checking. However, non-pooled memory is 97returned to the system rather than being saved in the Bacula pool. This kind 98of memory would normally be used in low volume situations (few malloc()s and 99free()s), but where the size of the buffer might have to be adjusted 100frequently. 101 102\paragraph*{Types of Memory Pool:} 103 104Currently there are three memory pool types: 105 106\begin{bitemize} 107\item PM\_NOPOOL -- non-pooled memory. 108\item PM\_FNAME -- a filename pool. 109\item PM\_MESSAGE -- a message buffer pool. 110\item PM\_EMSG -- error message buffer pool. 111\end{bitemize} 112 113\paragraph*{Getting Memory:} 114 115To get memory, one uses: 116 117\footnotesize 118\begin{bVerbatim} 119void *get_pool_memory(pool); 120\end{bVerbatim} 121\normalsize 122 123where {\bf pool} is one of the above mentioned pool names. The size of the 124memory returned will be determined by the system to be most appropriate for 125the application. 126 127If you wish non-pooled memory, you may alternatively call: 128 129\footnotesize 130\begin{bVerbatim} 131void *get_memory(size_t size); 132\end{bVerbatim} 133\normalsize 134 135The buffer length will be set to the size specified, and it will be assigned 136to the PM\_NOPOOL pool (no pooling). 137 138\paragraph*{Releasing Memory:} 139 140To free memory acquired by either of the above two calls, use: 141 142\footnotesize 143\begin{bVerbatim} 144void free_pool_memory(void *buffer); 145\end{bVerbatim} 146\normalsize 147 148where buffer is the memory buffer returned when the memory was acquired. If 149the memory was originally allocated as type PM\_NOPOOL, it will be released to 150the system, otherwise, it will be placed on the appropriate Bacula memory pool 151free chain to be used in a subsequent call for memory from that pool. 152 153\paragraph*{Determining the Memory Size:} 154 155To determine the memory buffer size, use: 156 157\footnotesize 158\begin{bVerbatim} 159size_t sizeof_pool_memory(void *buffer); 160\end{bVerbatim} 161\normalsize 162 163\paragraph*{Resizing Pool Memory:} 164 165To resize pool memory, use: 166 167\footnotesize 168\begin{bVerbatim} 169void *realloc_pool_memory(void *buffer); 170\end{bVerbatim} 171\normalsize 172 173The buffer will be reallocated, and the contents of the original buffer will 174be preserved, but the address of the buffer may change. 175 176\paragraph*{Automatic Size Adjustment:} 177 178To have the system check and if necessary adjust the size of your pooled 179memory buffer, use: 180 181\footnotesize 182\begin{bVerbatim} 183void *check_pool_memory_size(void *buffer, size_t new-size); 184\end{bVerbatim} 185\normalsize 186 187where {\bf new-size} is the buffer length needed. Note, if the buffer is 188already equal to or larger than {\bf new-size} no buffer size change will 189occur. However, if a buffer size change is needed, the original contents of 190the buffer will be preserved, but the buffer address may change. Many of the 191low level Bacula subroutines expect to be passed a pool memory buffer and use 192this call to ensure the buffer they use is sufficiently large. 193 194\paragraph*{Releasing All Pooled Memory:} 195 196In order to avoid orphaned buffer error messages when terminating the program, 197use: 198 199\footnotesize 200\begin{bVerbatim} 201void close_memory_pool(); 202\end{bVerbatim} 203\normalsize 204 205to free all unused memory retained in the Bacula memory pool. Note, any memory 206not returned to the pool via free\_pool\_memory() will not be released by this 207call. 208 209\paragraph*{Pooled Memory Statistics:} 210 211For debugging purposes and performance tuning, the following call will print 212the current memory pool statistics: 213 214\footnotesize 215\begin{bVerbatim} 216void print_memory_pool_stats(); 217\end{bVerbatim} 218\normalsize 219 220an example output is: 221 222\footnotesize 223\begin{bVerbatim} 224Pool Maxsize Maxused Inuse 225 0 256 0 0 226 1 256 1 0 227 2 256 1 0 228\end{bVerbatim} 229\normalsize 230