Revision tags: v8.1.1, v7.2.6, v8.0.5 |
|
#
34a68001 |
| 05-Sep-2023 |
Andrei Gudkov <gudkov.andrei@huawei.com> |
migration/calc-dirty-rate: millisecond-granularity period
This patch allows to measure dirty page rate for sub-second intervals of time. An optional argument is introduced -- calc-time-unit. For exa
migration/calc-dirty-rate: millisecond-granularity period
This patch allows to measure dirty page rate for sub-second intervals of time. An optional argument is introduced -- calc-time-unit. For example: {"execute": "calc-dirty-rate", "arguments": {"calc-time": 500, "calc-time-unit": "millisecond"} }
Millisecond granularity allows to make predictions whether migration will succeed or not. To do this, calculate dirty rate with calc-time set to max allowed downtime (e.g. 300ms), convert measured rate into volume of dirtied memory, and divide by network throughput. If the value is lower than max allowed downtime, then migration will converge.
Measurement results for single thread randomly writing to a 1/4/24GiB memory region:
+----------------+-----------------------------------------------+ | calc-time | dirty rate MiB/s | | (milliseconds) +----------------+---------------+--------------+ | | theoretical | page-sampling | dirty-bitmap | | | (at 3M wr/sec) | | | +----------------+----------------+---------------+--------------+ | 1GiB | +----------------+----------------+---------------+--------------+ | 100 | 6996 | 7100 | 3192 | | 200 | 4606 | 4660 | 2655 | | 300 | 3305 | 3280 | 2371 | | 400 | 2534 | 2525 | 2154 | | 500 | 2041 | 2044 | 1871 | | 750 | 1365 | 1341 | 1358 | | 1000 | 1024 | 1052 | 1025 | | 1500 | 683 | 678 | 684 | | 2000 | 512 | 507 | 513 | +----------------+----------------+---------------+--------------+ | 4GiB | +----------------+----------------+---------------+--------------+ | 100 | 10232 | 8880 | 4070 | | 200 | 8954 | 8049 | 3195 | | 300 | 7889 | 7193 | 2881 | | 400 | 6996 | 6530 | 2700 | | 500 | 6245 | 5772 | 2312 | | 750 | 4829 | 4586 | 2465 | | 1000 | 3865 | 3780 | 2178 | | 1500 | 2694 | 2633 | 2004 | | 2000 | 2041 | 2031 | 1789 | +----------------+----------------+---------------+--------------+ | 24GiB | +----------------+----------------+---------------+--------------+ | 100 | 11495 | 8640 | 5597 | | 200 | 11226 | 8616 | 3527 | | 300 | 10965 | 8386 | 2355 | | 400 | 10713 | 8370 | 2179 | | 500 | 10469 | 8196 | 2098 | | 750 | 9890 | 7885 | 2556 | | 1000 | 9354 | 7506 | 2084 | | 1500 | 8397 | 6944 | 2075 | | 2000 | 7574 | 6402 | 2062 | +----------------+----------------+---------------+--------------+
Theoretical values are computed according to the following formula: size * (1 - (1-(4096/size))^(time*wps)) / (time * 2^20), where size is in bytes, time is in seconds, and wps is number of writes per second.
Signed-off-by: Andrei Gudkov <gudkov.andrei@huawei.com> Reviewed-by: Hyman Huang <yong.huang@smartx.com> Message-Id: <d802e6b8053eb60fbec1a784cf86f67d9528e0a8.1693895970.git.gudkov.andrei@huawei.com> Signed-off-by: Hyman Huang <yong.huang@smartx.com>
show more ...
|
Revision tags: v8.1.1, v7.2.6, v8.0.5 |
|
#
34a68001 |
| 05-Sep-2023 |
Andrei Gudkov <gudkov.andrei@huawei.com> |
migration/calc-dirty-rate: millisecond-granularity period
This patch allows to measure dirty page rate for sub-second intervals of time. An optional argument is introduced -- calc-time-unit. For exa
migration/calc-dirty-rate: millisecond-granularity period
This patch allows to measure dirty page rate for sub-second intervals of time. An optional argument is introduced -- calc-time-unit. For example: {"execute": "calc-dirty-rate", "arguments": {"calc-time": 500, "calc-time-unit": "millisecond"} }
Millisecond granularity allows to make predictions whether migration will succeed or not. To do this, calculate dirty rate with calc-time set to max allowed downtime (e.g. 300ms), convert measured rate into volume of dirtied memory, and divide by network throughput. If the value is lower than max allowed downtime, then migration will converge.
Measurement results for single thread randomly writing to a 1/4/24GiB memory region:
+----------------+-----------------------------------------------+ | calc-time | dirty rate MiB/s | | (milliseconds) +----------------+---------------+--------------+ | | theoretical | page-sampling | dirty-bitmap | | | (at 3M wr/sec) | | | +----------------+----------------+---------------+--------------+ | 1GiB | +----------------+----------------+---------------+--------------+ | 100 | 6996 | 7100 | 3192 | | 200 | 4606 | 4660 | 2655 | | 300 | 3305 | 3280 | 2371 | | 400 | 2534 | 2525 | 2154 | | 500 | 2041 | 2044 | 1871 | | 750 | 1365 | 1341 | 1358 | | 1000 | 1024 | 1052 | 1025 | | 1500 | 683 | 678 | 684 | | 2000 | 512 | 507 | 513 | +----------------+----------------+---------------+--------------+ | 4GiB | +----------------+----------------+---------------+--------------+ | 100 | 10232 | 8880 | 4070 | | 200 | 8954 | 8049 | 3195 | | 300 | 7889 | 7193 | 2881 | | 400 | 6996 | 6530 | 2700 | | 500 | 6245 | 5772 | 2312 | | 750 | 4829 | 4586 | 2465 | | 1000 | 3865 | 3780 | 2178 | | 1500 | 2694 | 2633 | 2004 | | 2000 | 2041 | 2031 | 1789 | +----------------+----------------+---------------+--------------+ | 24GiB | +----------------+----------------+---------------+--------------+ | 100 | 11495 | 8640 | 5597 | | 200 | 11226 | 8616 | 3527 | | 300 | 10965 | 8386 | 2355 | | 400 | 10713 | 8370 | 2179 | | 500 | 10469 | 8196 | 2098 | | 750 | 9890 | 7885 | 2556 | | 1000 | 9354 | 7506 | 2084 | | 1500 | 8397 | 6944 | 2075 | | 2000 | 7574 | 6402 | 2062 | +----------------+----------------+---------------+--------------+
Theoretical values are computed according to the following formula: size * (1 - (1-(4096/size))^(time*wps)) / (time * 2^20), where size is in bytes, time is in seconds, and wps is number of writes per second.
Signed-off-by: Andrei Gudkov <gudkov.andrei@huawei.com> Reviewed-by: Hyman Huang <yong.huang@smartx.com> Message-Id: <d802e6b8053eb60fbec1a784cf86f67d9528e0a8.1693895970.git.gudkov.andrei@huawei.com> Signed-off-by: Hyman Huang <yong.huang@smartx.com>
show more ...
|
Revision tags: v8.1.0, v8.1.0-rc4, v8.1.0-rc3, v7.2.5, v8.0.4, v8.1.0-rc2, v8.1.0-rc1, v8.1.0-rc0, v8.0.3, v7.2.4, v8.0.2, v8.0.1, v7.2.3, v7.2.2, v8.0.0, v8.0.0-rc4, v8.0.0-rc3, v7.2.1, v8.0.0-rc2, v8.0.0-rc1, v8.0.0-rc0, v7.2.0, v7.2.0-rc4, v7.2.0-rc3, v7.2.0-rc2, v7.2.0-rc1, v7.2.0-rc0, v7.1.0, v7.1.0-rc4, v7.1.0-rc3, v7.1.0-rc2, v7.1.0-rc1, v7.1.0-rc0 |
|
#
8244166d |
| 25-Jun-2022 |
Hyman Huang(黄勇) <huangy81@chinatelecom.cn> |
migration/dirtyrate: Refactor dirty page rate calculation
abstract out dirty log change logic into function global_dirty_log_change.
abstract out dirty page rate calculation logic via dirty-ring in
migration/dirtyrate: Refactor dirty page rate calculation
abstract out dirty log change logic into function global_dirty_log_change.
abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate.
abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it from DirtyStat.
rename set_sample_page_period to dirty_stat_wait, which is well-understood and will be reused in dirtylimit.
handle cpu hotplug/unplug scenario during measurement of dirty page rate.
export util functions outside migration.
Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> Reviewed-by: Peter Xu <peterx@redhat.com> Message-Id: <7b6f6f4748d5b3d017b31a0429e630229ae97538.1656177590.git.huangy81@chinatelecom.cn> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
show more ...
|
Revision tags: v7.0.0, v7.0.0-rc4, v7.0.0-rc3, v7.0.0-rc2, v7.0.0-rc1, v7.0.0-rc0, v6.1.1, v6.2.0, v6.2.0-rc4, v6.2.0-rc3, v6.2.0-rc2, v6.2.0-rc1, v6.2.0-rc0, v6.0.1, v6.1.0, v6.1.0-rc4, v6.1.0-rc3, v6.1.0-rc2, v6.1.0-rc1, v6.1.0-rc0 |
|
#
71864ead |
| 29-Jun-2021 |
Hyman Huang(黄勇) <huangy81@chinatelecom.cn> |
migration/dirtyrate: introduce struct and adjust DirtyRateStat
introduce "DirtyRateMeasureMode" to specify what method should be used to calculate dirty rate, introduce "DirtyRateVcpu" to store dirt
migration/dirtyrate: introduce struct and adjust DirtyRateStat
introduce "DirtyRateMeasureMode" to specify what method should be used to calculate dirty rate, introduce "DirtyRateVcpu" to store dirty rate for each vcpu.
use union to store stat data of specific mode
Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> Message-Id: <661c98c40f40e163aa58334337af8f3ddf41316a.1624040308.git.huangy81@chinatelecom.cn> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
show more ...
|
#
7afa08cd |
| 07-Jun-2021 |
Hyman Huang(黄勇) <huangy81@chinatelecom.cn> |
migration/dirtyrate: make sample page count configurable
introduce optional sample-pages argument in calc-dirty-rate, making sample page count per GB configurable so that more accurate dirtyrate can
migration/dirtyrate: make sample page count configurable
introduce optional sample-pages argument in calc-dirty-rate, making sample page count per GB configurable so that more accurate dirtyrate can be calculated.
Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> Message-Id: <3103453a3b2796f929269c99a6ad81a9a7f1f405.1623027729.git.huangy81@chinatelecom.cn> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Wrapped a couple of long lines
show more ...
|
Revision tags: v6.0.0, v6.0.0-rc5, v6.0.0-rc4, v6.0.0-rc3, v6.0.0-rc2, v6.0.0-rc1, v6.0.0-rc0, v5.2.0, v5.2.0-rc4, v5.2.0-rc3, v5.2.0-rc2, v5.2.0-rc1, v5.2.0-rc0 |
|
#
eca58224 |
| 16-Sep-2020 |
Chuan Zheng <zhengchuan@huawei.com> |
migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid()
Implement is_sample_period_valid() to check if the sample period is vaild and do set_sample_page_period() to slee
migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid()
Implement is_sample_period_valid() to check if the sample period is vaild and do set_sample_page_period() to sleep specific time between sample actions.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: David Edmondson <david.edmondson@oracle.com> Reviewed-by: Li Qiang <liq3ea@gmail.com> Message-Id: <1600237327-33618-10-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
show more ...
|
#
f82583cd |
| 16-Sep-2020 |
Chuan Zheng <zhengchuan@huawei.com> |
migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE
In order to sample real RAM, skip ramblock with size below MIN_RAMBLOCK_SIZE which is set as 128M.
Signed-off-by: Chuan
migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE
In order to sample real RAM, skip ramblock with size below MIN_RAMBLOCK_SIZE which is set as 128M.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Reviewed-by: David Edmondson <david.edmondson@oracle.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Li Qiang <liq3ea@gmail.com> Message-Id: <1600237327-33618-9-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
show more ...
|
#
c9a58d71 |
| 16-Sep-2020 |
Chuan Zheng <zhengchuan@huawei.com> |
migration/dirtyrate: Add dirtyrate statistics series functions
Add dirtyrate statistics functions to record/update dirtyrate info.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Reviewed-by: Dr
migration/dirtyrate: Add dirtyrate statistics series functions
Add dirtyrate statistics functions to record/update dirtyrate info.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Li Qiang <liq3ea@gmail.com> Message-Id: <1600237327-33618-5-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
show more ...
|
#
a2635f0a |
| 16-Sep-2020 |
Chuan Zheng <zhengchuan@huawei.com> |
migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info
Add RamblockDirtyInfo to store sampled page info of each ramblock.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Reviewed-
migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info
Add RamblockDirtyInfo to store sampled page info of each ramblock.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: David Edmondson <david.edmondson@oracle.com> Reviewed-by: Li Qiang <liq3ea@gmail.com> Message-Id: <1600237327-33618-4-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
show more ...
|
#
4240dcee |
| 16-Sep-2020 |
Chuan Zheng <zhengchuan@huawei.com> |
migration/dirtyrate: setup up query-dirtyrate framwork
Add get_dirtyrate_thread() functions to setup query-dirtyrate framework.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Signed-off-by: Yan
migration/dirtyrate: setup up query-dirtyrate framwork
Add get_dirtyrate_thread() functions to setup query-dirtyrate framework.
Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Signed-off-by: YanYing Zhuang <ann.zhuangyanying@huawei.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: David Edmondson <david.edmondson@oracle.com> Reviewed-by: Li Qiang <liq3ea@gmail.com> Message-Id: <1600237327-33618-2-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
show more ...
|