1Bareos Memory Management
2========================
3
4General
5-------
6
7This document describes the memory management routines that are used in
8Bareos and is meant to be a technical discussion for developers rather
9than part of the user manual.
10
11Since Bareos may be called upon to handle filenames of varying and more
12or less arbitrary length, special attention needs to be used in the code
13to ensure that memory buffers are sufficiently large. There are four
14possibilities for memory usage within **Bareos**. Each will be described
15in turn. They are:
16
17-  Statically allocated memory.
18
19-  Dynamically allocated memory using malloc() and free().
20
21-  Non-pooled memory.
22
23-  Pooled memory.
24
25Statically Allocated Memory
26~~~~~~~~~~~~~~~~~~~~~~~~~~~
27
28Statically allocated memory is of the form:
29
30::
31
32    char buffer[MAXSTRING];
33
34The use of this kind of memory is discouraged except when you are 100%
35sure that the strings to be used will be of a fixed length. One example
36of where this is appropriate is for **Bareos** resource names, which are
37currently limited to 127 characters (MAX_NAME_LENGTH). Although this
38maximum size may change, particularly to accommodate Unicode, it will
39remain a relatively small value.
40
41Dynamically Allocated Memory
42~~~~~~~~~~~~~~~~~~~~~~~~~~~~
43
44Dynamically allocated memory is obtained using the standard malloc()
45routines. As in:
46
47::
48
49    char *buf;
50    buf = malloc(256);
51
52This kind of memory can be released with:
53
54::
55
56    free(buf);
57
58It is recommended to use this kind of memory only when you are sure that
59you know the memory size needed and the memory will be used for short
60periods of time – that is it would not be appropriate to use statically
61allocated memory. An example might be to obtain a large memory buffer
62for reading and writing files. When **SmartAlloc** is enabled, the
63memory obtained by malloc() will automatically be checked for buffer
64overwrite (overflow) during the free() call, and all malloc’ed memory
65that is not released prior to termination of the program will be
66reported as Orphaned memory.
67
68Pooled and Non-pooled Memory
69~~~~~~~~~~~~~~~~~~~~~~~~~~~~
70
71In order to facility the handling of arbitrary length filenames and to
72efficiently handle a high volume of dynamic memory usage, we have
73implemented routines between the C code and the malloc routines. The
74first is called “Pooled” memory, and is memory, which once allocated and
75then released, is not returned to the system memory pool, but rather
76retained in a Bareos memory pool. The next request to acquire pooled
77memory will return any free memory block. In addition, each memory block
78has its current size associated with the block allowing for easy
79checking if the buffer is of sufficient size. This kind of memory would
80normally be used in high volume situations (lots of malloc()s and
81free()s) where the buffer length may have to frequently change to adapt
82to varying filename lengths.
83
84The non-pooled memory is handled by routines similar to those used for
85pooled memory, allowing for easy size checking. However, non-pooled
86memory is returned to the system rather than being saved in the Bareos
87pool. This kind of memory would normally be used in low volume
88situations (few malloc()s and free()s), but where the size of the buffer
89might have to be adjusted frequently.
90
91Types of Memory Pool:
92'''''''''''''''''''''
93
94Currently there are three memory pool types:
95
96-  PM_NOPOOL – non-pooled memory.
97
98-  PM_FNAME – a filename pool.
99
100-  PM_MESSAGE – a message buffer pool.
101
102-  PM_EMSG – error message buffer pool.
103
104Getting Memory:
105'''''''''''''''
106
107To get memory, one uses:
108
109::
110
111    void *get_pool_memory(pool);
112
113where **pool** is one of the above mentioned pool names. The size of the
114memory returned will be determined by the system to be most appropriate
115for the application.
116
117If you wish non-pooled memory, you may alternatively call:
118
119::
120
121    void *get_memory(size_t size);
122
123The buffer length will be set to the size specified, and it will be
124assigned to the PM_NOPOOL pool (no pooling).
125
126Releasing Memory:
127'''''''''''''''''
128
129To free memory acquired by either of the above two calls, use:
130
131::
132
133    void free_pool_memory(void *buffer);
134
135where buffer is the memory buffer returned when the memory was acquired.
136If the memory was originally allocated as type PM_NOPOOL, it will be
137released to the system, otherwise, it will be placed on the appropriate
138Bareos memory pool free chain to be used in a subsequent call for memory
139from that pool.
140
141Determining the Memory Size:
142''''''''''''''''''''''''''''
143
144To determine the memory buffer size, use:
145
146::
147
148    size_t sizeof_pool_memory(void *buffer);
149
150Resizing Pool Memory:
151'''''''''''''''''''''
152
153To resize pool memory, use:
154
155::
156
157    void *realloc_pool_memory(void *buffer);
158
159The buffer will be reallocated, and the contents of the original buffer
160will be preserved, but the address of the buffer may change.
161
162Automatic Size Adjustment:
163''''''''''''''''''''''''''
164
165To have the system check and if necessary adjust the size of your pooled
166memory buffer, use:
167
168::
169
170    void *check_pool_memory_size(void *buffer, size_t new-size);
171
172where **new-size** is the buffer length needed. Note, if the buffer is
173already equal to or larger than **new-size** no buffer size change will
174occur. However, if a buffer size change is needed, the original contents
175of the buffer will be preserved, but the buffer address may change. Many
176of the low level Bareos subroutines expect to be passed a pool memory
177buffer and use this call to ensure the buffer they use is sufficiently
178large.
179
180Releasing All Pooled Memory:
181''''''''''''''''''''''''''''
182
183In order to avoid orphaned buffer error messages when terminating the
184program, use:
185
186::
187
188    void close_memory_pool();
189
190to free all unused memory retained in the Bareos memory pool. Note, any
191memory not returned to the pool via free_pool_memory() will not be
192released by this call.
193
194Pooled Memory Statistics:
195'''''''''''''''''''''''''
196
197For debugging purposes and performance tuning, the following call will
198print the current memory pool statistics:
199
200::
201
202    void print_memory_pool_stats();
203
204an example output is:
205
206::
207
208    Pool  Maxsize  Maxused  Inuse
209       0      256        0      0
210       1      256        1      0
211       2      256        1      0
212