1Bareos Memory Management 2======================== 3 4General 5------- 6 7This document describes the memory management routines that are used in 8Bareos and is meant to be a technical discussion for developers rather 9than part of the user manual. 10 11Since Bareos may be called upon to handle filenames of varying and more 12or less arbitrary length, special attention needs to be used in the code 13to ensure that memory buffers are sufficiently large. There are four 14possibilities for memory usage within **Bareos**. Each will be described 15in turn. They are: 16 17- Statically allocated memory. 18 19- Dynamically allocated memory using malloc() and free(). 20 21- Non-pooled memory. 22 23- Pooled memory. 24 25Statically Allocated Memory 26~~~~~~~~~~~~~~~~~~~~~~~~~~~ 27 28Statically allocated memory is of the form: 29 30:: 31 32 char buffer[MAXSTRING]; 33 34The use of this kind of memory is discouraged except when you are 100% 35sure that the strings to be used will be of a fixed length. One example 36of where this is appropriate is for **Bareos** resource names, which are 37currently limited to 127 characters (MAX_NAME_LENGTH). Although this 38maximum size may change, particularly to accommodate Unicode, it will 39remain a relatively small value. 40 41Dynamically Allocated Memory 42~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 43 44Dynamically allocated memory is obtained using the standard malloc() 45routines. As in: 46 47:: 48 49 char *buf; 50 buf = malloc(256); 51 52This kind of memory can be released with: 53 54:: 55 56 free(buf); 57 58It is recommended to use this kind of memory only when you are sure that 59you know the memory size needed and the memory will be used for short 60periods of time – that is it would not be appropriate to use statically 61allocated memory. An example might be to obtain a large memory buffer 62for reading and writing files. When **SmartAlloc** is enabled, the 63memory obtained by malloc() will automatically be checked for buffer 64overwrite (overflow) during the free() call, and all malloc’ed memory 65that is not released prior to termination of the program will be 66reported as Orphaned memory. 67 68Pooled and Non-pooled Memory 69~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 70 71In order to facility the handling of arbitrary length filenames and to 72efficiently handle a high volume of dynamic memory usage, we have 73implemented routines between the C code and the malloc routines. The 74first is called “Pooled” memory, and is memory, which once allocated and 75then released, is not returned to the system memory pool, but rather 76retained in a Bareos memory pool. The next request to acquire pooled 77memory will return any free memory block. In addition, each memory block 78has its current size associated with the block allowing for easy 79checking if the buffer is of sufficient size. This kind of memory would 80normally be used in high volume situations (lots of malloc()s and 81free()s) where the buffer length may have to frequently change to adapt 82to varying filename lengths. 83 84The non-pooled memory is handled by routines similar to those used for 85pooled memory, allowing for easy size checking. However, non-pooled 86memory is returned to the system rather than being saved in the Bareos 87pool. This kind of memory would normally be used in low volume 88situations (few malloc()s and free()s), but where the size of the buffer 89might have to be adjusted frequently. 90 91Types of Memory Pool: 92''''''''''''''''''''' 93 94Currently there are three memory pool types: 95 96- PM_NOPOOL – non-pooled memory. 97 98- PM_FNAME – a filename pool. 99 100- PM_MESSAGE – a message buffer pool. 101 102- PM_EMSG – error message buffer pool. 103 104Getting Memory: 105''''''''''''''' 106 107To get memory, one uses: 108 109:: 110 111 void *get_pool_memory(pool); 112 113where **pool** is one of the above mentioned pool names. The size of the 114memory returned will be determined by the system to be most appropriate 115for the application. 116 117If you wish non-pooled memory, you may alternatively call: 118 119:: 120 121 void *get_memory(size_t size); 122 123The buffer length will be set to the size specified, and it will be 124assigned to the PM_NOPOOL pool (no pooling). 125 126Releasing Memory: 127''''''''''''''''' 128 129To free memory acquired by either of the above two calls, use: 130 131:: 132 133 void free_pool_memory(void *buffer); 134 135where buffer is the memory buffer returned when the memory was acquired. 136If the memory was originally allocated as type PM_NOPOOL, it will be 137released to the system, otherwise, it will be placed on the appropriate 138Bareos memory pool free chain to be used in a subsequent call for memory 139from that pool. 140 141Determining the Memory Size: 142'''''''''''''''''''''''''''' 143 144To determine the memory buffer size, use: 145 146:: 147 148 size_t sizeof_pool_memory(void *buffer); 149 150Resizing Pool Memory: 151''''''''''''''''''''' 152 153To resize pool memory, use: 154 155:: 156 157 void *realloc_pool_memory(void *buffer); 158 159The buffer will be reallocated, and the contents of the original buffer 160will be preserved, but the address of the buffer may change. 161 162Automatic Size Adjustment: 163'''''''''''''''''''''''''' 164 165To have the system check and if necessary adjust the size of your pooled 166memory buffer, use: 167 168:: 169 170 void *check_pool_memory_size(void *buffer, size_t new-size); 171 172where **new-size** is the buffer length needed. Note, if the buffer is 173already equal to or larger than **new-size** no buffer size change will 174occur. However, if a buffer size change is needed, the original contents 175of the buffer will be preserved, but the buffer address may change. Many 176of the low level Bareos subroutines expect to be passed a pool memory 177buffer and use this call to ensure the buffer they use is sufficiently 178large. 179 180Releasing All Pooled Memory: 181'''''''''''''''''''''''''''' 182 183In order to avoid orphaned buffer error messages when terminating the 184program, use: 185 186:: 187 188 void close_memory_pool(); 189 190to free all unused memory retained in the Bareos memory pool. Note, any 191memory not returned to the pool via free_pool_memory() will not be 192released by this call. 193 194Pooled Memory Statistics: 195''''''''''''''''''''''''' 196 197For debugging purposes and performance tuning, the following call will 198print the current memory pool statistics: 199 200:: 201 202 void print_memory_pool_stats(); 203 204an example output is: 205 206:: 207 208 Pool Maxsize Maxused Inuse 209 0 256 0 0 210 1 256 1 0 211 2 256 1 0 212