1Using the original execution date of: 2017-05-03 13:33:24Z 2 3Current cluster status: 4Node controller-1 (2): UNCLEAN (offline) 5Online: [ controller-0 controller-2 ] 6RemoteOnline: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] 7 8 messaging-0 (ocf::pacemaker:remote): Started controller-0 9 messaging-1 (ocf::pacemaker:remote): Started controller-1 (UNCLEAN) 10 messaging-2 (ocf::pacemaker:remote): Started controller-0 11 galera-0 (ocf::pacemaker:remote): Started controller-1 (UNCLEAN) 12 galera-1 (ocf::pacemaker:remote): Started controller-0 13 galera-2 (ocf::pacemaker:remote): Started controller-1 (UNCLEAN) 14 Clone Set: rabbitmq-clone [rabbitmq] 15 Started: [ messaging-0 messaging-1 messaging-2 ] 16 Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 ] 17 Master/Slave Set: galera-master [galera] 18 Masters: [ galera-0 galera-1 ] 19 Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ] 20 Master/Slave Set: redis-master [redis] 21 redis (ocf::heartbeat:redis): Slave controller-1 (UNCLEAN) 22 Masters: [ controller-0 ] 23 Slaves: [ controller-2 ] 24 Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] 25 ip-192.168.24.6 (ocf::heartbeat:IPaddr2): Started controller-0 26 ip-10.0.0.102 (ocf::heartbeat:IPaddr2): Started controller-0 27 ip-172.17.1.14 (ocf::heartbeat:IPaddr2): Started controller-1 (UNCLEAN) 28 ip-172.17.1.17 (ocf::heartbeat:IPaddr2): Started controller-1 (UNCLEAN) 29 ip-172.17.3.15 (ocf::heartbeat:IPaddr2): Started controller-0 30 ip-172.17.4.11 (ocf::heartbeat:IPaddr2): Started controller-1 (UNCLEAN) 31 Clone Set: haproxy-clone [haproxy] 32 haproxy (systemd:haproxy): Started controller-1 (UNCLEAN) 33 Started: [ controller-0 controller-2 ] 34 Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] 35 openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 36 stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 37 stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 38 stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-1 (UNCLEAN) 39 40Transition Summary: 41 * Fence (reboot) messaging-1 'resources are active and the connection is unrecoverable' 42 * Fence (reboot) controller-1 'peer is no longer part of the cluster' 43 * Stop messaging-1 (controller-1) due to node availability 44 * Move galera-0 ( controller-1 -> controller-2 ) 45 * Stop galera-2 ( controller-1 ) due to node availability 46 * Stop rabbitmq:2 (messaging-1) due to node availability 47 * Stop redis:0 ( Slave controller-1 ) due to node availability 48 * Move ip-172.17.1.14 ( controller-1 -> controller-2 ) 49 * Move ip-172.17.1.17 ( controller-1 -> controller-2 ) 50 * Move ip-172.17.4.11 ( controller-1 -> controller-2 ) 51 * Stop haproxy:0 (controller-1) due to node availability 52 * Restart stonith-fence_ipmilan-525400bbf613 ( controller-0 ) due to resource definition change 53 * Restart stonith-fence_ipmilan-525400b4f6bd ( controller-0 ) due to resource definition change 54 * Move stonith-fence_ipmilan-5254005bdbb5 ( controller-1 -> controller-2 ) 55 56Executing cluster transition: 57 * Pseudo action: galera-0_stop_0 58 * Pseudo action: redis-master_pre_notify_stop_0 59 * Resource action: stonith-fence_ipmilan-525400bbf613 stop on controller-0 60 * Resource action: stonith-fence_ipmilan-525400b4f6bd stop on controller-0 61 * Pseudo action: stonith-fence_ipmilan-5254005bdbb5_stop_0 62 * Fencing controller-1 (reboot) 63 * Pseudo action: messaging-1_stop_0 64 * Pseudo action: galera-2_stop_0 65 * Pseudo action: redis_post_notify_stop_0 66 * Resource action: redis notify on controller-0 67 * Resource action: redis notify on controller-2 68 * Pseudo action: redis-master_confirmed-pre_notify_stop_0 69 * Pseudo action: redis-master_stop_0 70 * Pseudo action: haproxy-clone_stop_0 71 * Fencing messaging-1 (reboot) 72 * Resource action: galera-0 start on controller-2 73 * Pseudo action: rabbitmq_post_notify_stop_0 74 * Pseudo action: rabbitmq-clone_stop_0 75 * Resource action: galera monitor=10000 on galera-0 76 * Pseudo action: redis_stop_0 77 * Pseudo action: redis-master_stopped_0 78 * Pseudo action: haproxy_stop_0 79 * Pseudo action: haproxy-clone_stopped_0 80 * Resource action: stonith-fence_ipmilan-525400bbf613 start on controller-0 81 * Resource action: stonith-fence_ipmilan-525400bbf613 monitor=60000 on controller-0 82 * Resource action: stonith-fence_ipmilan-525400b4f6bd start on controller-0 83 * Resource action: stonith-fence_ipmilan-525400b4f6bd monitor=60000 on controller-0 84 * Resource action: stonith-fence_ipmilan-5254005bdbb5 start on controller-2 85 * Resource action: galera-0 monitor=20000 on controller-2 86 * Resource action: rabbitmq notify on messaging-2 87 * Resource action: rabbitmq notify on messaging-0 88 * Pseudo action: rabbitmq_notified_0 89 * Pseudo action: rabbitmq_stop_0 90 * Pseudo action: rabbitmq-clone_stopped_0 91 * Pseudo action: redis-master_post_notify_stopped_0 92 * Pseudo action: ip-172.17.1.14_stop_0 93 * Pseudo action: ip-172.17.1.17_stop_0 94 * Pseudo action: ip-172.17.4.11_stop_0 95 * Resource action: stonith-fence_ipmilan-5254005bdbb5 monitor=60000 on controller-2 96 * Resource action: redis notify on controller-0 97 * Resource action: redis notify on controller-2 98 * Pseudo action: redis-master_confirmed-post_notify_stopped_0 99 * Resource action: ip-172.17.1.14 start on controller-2 100 * Resource action: ip-172.17.1.17 start on controller-2 101 * Resource action: ip-172.17.4.11 start on controller-2 102 * Pseudo action: redis_notified_0 103 * Resource action: ip-172.17.1.14 monitor=10000 on controller-2 104 * Resource action: ip-172.17.1.17 monitor=10000 on controller-2 105 * Resource action: ip-172.17.4.11 monitor=10000 on controller-2 106Using the original execution date of: 2017-05-03 13:33:24Z 107 108Revised cluster status: 109Online: [ controller-0 controller-2 ] 110OFFLINE: [ controller-1 ] 111RemoteOnline: [ galera-0 galera-1 messaging-0 messaging-2 ] 112RemoteOFFLINE: [ galera-2 messaging-1 ] 113 114 messaging-0 (ocf::pacemaker:remote): Started controller-0 115 messaging-1 (ocf::pacemaker:remote): Stopped 116 messaging-2 (ocf::pacemaker:remote): Started controller-0 117 galera-0 (ocf::pacemaker:remote): Started controller-2 118 galera-1 (ocf::pacemaker:remote): Started controller-0 119 galera-2 (ocf::pacemaker:remote): Stopped 120 Clone Set: rabbitmq-clone [rabbitmq] 121 Started: [ messaging-0 messaging-2 ] 122 Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 messaging-1 ] 123 Master/Slave Set: galera-master [galera] 124 Masters: [ galera-0 galera-1 ] 125 Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ] 126 Master/Slave Set: redis-master [redis] 127 Masters: [ controller-0 ] 128 Slaves: [ controller-2 ] 129 Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] 130 ip-192.168.24.6 (ocf::heartbeat:IPaddr2): Started controller-0 131 ip-10.0.0.102 (ocf::heartbeat:IPaddr2): Started controller-0 132 ip-172.17.1.14 (ocf::heartbeat:IPaddr2): Started controller-2 133 ip-172.17.1.17 (ocf::heartbeat:IPaddr2): Started controller-2 134 ip-172.17.3.15 (ocf::heartbeat:IPaddr2): Started controller-0 135 ip-172.17.4.11 (ocf::heartbeat:IPaddr2): Started controller-2 136 Clone Set: haproxy-clone [haproxy] 137 Started: [ controller-0 controller-2 ] 138 Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] 139 openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 140 stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 141 stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 142 stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-2 143 144