FIX: While synchronizing instance power states, found XX instances in the database and XX instances on the hypervisor
Check kibana logs for
"While synchronizing instance power states, found"
Check libvirt VMs vs. nova DB
COMPUTE_NODE=com1-dev # get shutoff VMs on compute node VMS_COMPUTE=$(ssh ${COMPUTE_NODE} docker exec -i nova_libvirt virsh list --state-shutoff --uuid | sed '/^$/d' | sort) # echo "${VMS_COMPUTE}" # get shutoff VMs from nova DB VMS_NOVA=$(ssh os-admin-dev "source /etc/kolla/admin-openrc.sh; openstack server list --all --host ${COMPUTE_NODE} -c ID -f value --status SHUTOFF" | sort) # echo "${VMS_NOVA}" # diff shutoff VMs comm -3 <(echo "${VMS_COMPUTE}") <(echo "${VMS_NOVA}")
Remove shutdown VM from libvirt