1.. SPDX-License-Identifier: GPL-2.0 2 3======================= 4DAMON-based Reclamation 5======================= 6 7DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to 8be used for proactive and lightweight reclamation under light memory pressure. 9It doesn't aim to replace the LRU-list based page_granularity reclamation, but 10to be selectively used for different level of memory pressure and requirements. 11 12Where Proactive Reclamation is Required? 13======================================== 14 15On general memory over-committed systems, proactively reclaiming cold pages 16helps saving memory and reducing latency spikes that incurred by the direct 17reclaim of the process or CPU consumption of kswapd, while incurring only 18minimal performance degradation [1]_ [2]_ . 19 20Free Pages Reporting [3]_ based memory over-commit virtualization systems are 21good example of the cases. In such systems, the guest VMs reports their free 22memory to host, and the host reallocates the reported memory to other guests. 23As a result, the memory of the systems are fully utilized. However, the 24guests could be not so memory-frugal, mainly because some kernel subsystems and 25user-space applications are designed to use as much memory as available. Then, 26guests could report only small amount of memory as free to host, results in 27memory utilization drop of the systems. Running the proactive reclamation in 28guests could mitigate this problem. 29 30How It Works? 31============= 32 33DAMON_RECLAIM finds memory regions that didn't accessed for specific time 34duration and page out. To avoid it consuming too much CPU for the paging out 35operation, a speed limit can be configured. Under the speed limit, it pages 36out memory regions that didn't accessed longer time first. System 37administrators can also configure under what situation this scheme should 38automatically activated and deactivated with three memory pressure watermarks. 39 40Interface: Module Parameters 41============================ 42 43To use this feature, you should first ensure your system is running on a kernel 44that is built with ``CONFIG_DAMON_RECLAIM=y``. 45 46To let sysadmins enable or disable it and tune for the given system, 47DAMON_RECLAIM utilizes module parameters. That is, you can put 48``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write 49proper values to ``/sys/modules/damon_reclaim/parameters/<parameter>`` files. 50 51Note that the parameter values except ``enabled`` are applied only when 52DAMON_RECLAIM starts. Therefore, if you want to apply new parameter values in 53runtime and DAMON_RECLAIM is already enabled, you should disable and re-enable 54it via ``enabled`` parameter file. Writing of the new values to proper 55parameter values should be done before the re-enablement. 56 57Below are the description of each parameter. 58 59enabled 60------- 61 62Enable or disable DAMON_RECLAIM. 63 64You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. 65Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do 66no real monitoring and reclamation due to the watermarks-based activation 67condition. Refer to below descriptions for the watermarks parameter for this. 68 69commit_inputs 70------------- 71 72Make DAMON_RECLAIM reads the input parameters again, except ``enabled``. 73 74Input parameters that updated while DAMON_RECLAIM is running are not applied 75by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values 76of parametrs except ``enabled`` again. Once the re-reading is done, this 77parameter is set as ``N``. If invalid parameters are found while the 78re-reading, DAMON_RECLAIM will be disabled. 79 80min_age 81------- 82 83Time threshold for cold memory regions identification in microseconds. 84 85If a memory region is not accessed for this or longer time, DAMON_RECLAIM 86identifies the region as cold, and reclaims it. 87 88120 seconds by default. 89 90quota_ms 91-------- 92 93Limit of time for the reclamation in milliseconds. 94 95DAMON_RECLAIM tries to use only up to this time within a time window 96(quota_reset_interval_ms) for trying reclamation of cold pages. This can be 97used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the 98limit is disabled. 99 10010 ms by default. 101 102quota_sz 103-------- 104 105Limit of size of memory for the reclamation in bytes. 106 107DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time 108window (quota_reset_interval_ms) and makes no more than this limit is tried. 109This can be used for limiting consumption of CPU and IO. If this value is 110zero, the limit is disabled. 111 112128 MiB by default. 113 114quota_reset_interval_ms 115----------------------- 116 117The time/size quota charge reset interval in milliseconds. 118 119The charget reset interval for the quota of time (quota_ms) and size 120(quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than 121quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms 122milliseconds. 123 1241 second by default. 125 126wmarks_interval 127--------------- 128 129Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is 130enabled but inactive due to its watermarks rule. 131 132wmarks_high 133----------- 134 135Free memory rate (per thousand) for the high watermark. 136 137If free memory of the system in bytes per thousand bytes is higher than this, 138DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks 139the watermarks. 140 141wmarks_mid 142---------- 143 144Free memory rate (per thousand) for the middle watermark. 145 146If free memory of the system in bytes per thousand bytes is between this and 147the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and 148the reclaiming. 149 150wmarks_low 151---------- 152 153Free memory rate (per thousand) for the low watermark. 154 155If free memory of the system in bytes per thousand bytes is lower than this, 156DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the 157watermarks. In the case, the system falls back to the LRU-list based page 158granularity reclamation logic. 159 160sample_interval 161--------------- 162 163Sampling interval for the monitoring in microseconds. 164 165The sampling interval of DAMON for the cold memory monitoring. Please refer to 166the DAMON documentation (:doc:`usage`) for more detail. 167 168aggr_interval 169------------- 170 171Aggregation interval for the monitoring in microseconds. 172 173The aggregation interval of DAMON for the cold memory monitoring. Please 174refer to the DAMON documentation (:doc:`usage`) for more detail. 175 176min_nr_regions 177-------------- 178 179Minimum number of monitoring regions. 180 181The minimal number of monitoring regions of DAMON for the cold memory 182monitoring. This can be used to set lower-bound of the monitoring quality. 183But, setting this too high could result in increased monitoring overhead. 184Please refer to the DAMON documentation (:doc:`usage`) for more detail. 185 186max_nr_regions 187-------------- 188 189Maximum number of monitoring regions. 190 191The maximum number of monitoring regions of DAMON for the cold memory 192monitoring. This can be used to set upper-bound of the monitoring overhead. 193However, setting this too low could result in bad monitoring quality. Please 194refer to the DAMON documentation (:doc:`usage`) for more detail. 195 196monitor_region_start 197-------------------- 198 199Start of target memory region in physical address. 200 201The start physical address of memory region that DAMON_RECLAIM will do work 202against. That is, DAMON_RECLAIM will find cold memory regions in this region 203and reclaims. By default, biggest System RAM is used as the region. 204 205monitor_region_end 206------------------ 207 208End of target memory region in physical address. 209 210The end physical address of memory region that DAMON_RECLAIM will do work 211against. That is, DAMON_RECLAIM will find cold memory regions in this region 212and reclaims. By default, biggest System RAM is used as the region. 213 214kdamond_pid 215----------- 216 217PID of the DAMON thread. 218 219If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else, 220-1. 221 222nr_reclaim_tried_regions 223------------------------ 224 225Number of memory regions that tried to be reclaimed by DAMON_RECLAIM. 226 227bytes_reclaim_tried_regions 228--------------------------- 229 230Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM. 231 232nr_reclaimed_regions 233-------------------- 234 235Number of memory regions that successfully be reclaimed by DAMON_RECLAIM. 236 237bytes_reclaimed_regions 238----------------------- 239 240Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM. 241 242nr_quota_exceeds 243---------------- 244 245Number of times that the time/space quota limits have exceeded. 246 247Example 248======= 249 250Below runtime example commands make DAMON_RECLAIM to find memory regions that 251not accessed for 30 seconds or more and pages out. The reclamation is limited 252to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too 253much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do 254nothing if the system's free memory rate is more than 50%, but start the real 255works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and 256therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to 257do nothing again, so that we can fall back to the LRU-list based page 258granularity reclamation. :: 259 260 # cd /sys/modules/damon_reclaim/parameters 261 # echo 30000000 > min_age 262 # echo $((1 * 1024 * 1024 * 1024)) > quota_sz 263 # echo 1000 > quota_reset_interval_ms 264 # echo 500 > wmarks_high 265 # echo 400 > wmarks_mid 266 # echo 200 > wmarks_low 267 # echo Y > enabled 268 269.. [1] https://research.google/pubs/pub48551/ 270.. [2] https://lwn.net/Articles/787611/ 271.. [3] https://www.kernel.org/doc/html/latest/vm/free_page_reporting.html 272