1. How to locate the virtual machine startup failure problem?

Log in to the physical host where the virtual machine is located and use virsh list to find the name of the virtual machine, as shown below, the virtual machine is instance-00001840 (all the examples that follow are based on this virtual machine(1):

Check the creation log of the virtual machine to find the corresponding point in time and check the cause of failure:

/var/log/libvirt/libvirtd.log

/var/log/libvirt/qemu/instance-00001840.log

If the log is a few days ago, then it may have been dumped, for example, the log of the failure to create a virtual machine on 2019-01-09 has been dumped to logdump-56-20190109235801.tar.gz, which you need to unzip and view by yourself:

2. Common failure reasons and processing methods:

1. Insufficient memory leads to failure, take instance-00001840 VM for example, the /var/log/libvirt/qemu/instance-00001840.log log is as follows:

Failure reason keyword: Cannot allocate memory

Failure cause keyword: insufficent free host memory pages avaiable to allocate guest RAM

Solution: Insufficent free host memory pages avaiable to allocate guest RAM, please create to other free host or free the memory of other VMs or processes on this host.

2. The virtual machine disk file system is read-only causing failure, /var/log/libvirt/qemu/instance-00001840.log log as follows:

Failure cause keyword: Read-only file system

Solution: Contact the interface person for dsware storage to handle the problem.

3. Vhost NIC does not exist causing failure, /var/log/libvirt/qemu/instance-00001840.log log as follows:

Failure keyword: Failed to connect socket: No such file or directory. 

Solution: VM NIC device /var/run/vhost-user/port1 does not exist, contact nova or neutron interface to deal with it.

4. The virtual machine failed to start due to exhaustion of system resources, /var/log/libvirt/qemu/instance-00001840.log is as follows:

Key log:

libvirt: error : libvirtd quit during handshake: Input/output error.

Solution: release system resources (find the component that is problematically draining system resources and fix the bug. this time it is caused by a problem with one of the network domain services)

5. Hibernation wakeup host main frequency is different resulting in virtual machine startup failure, /var/log/libvirt/qemu/instance-00001840.log log as follows:

Key log:

qemu-kvm: warning: TSC frequency mistach between VM (xxxx kHz) and host (xxxx kHz) and TSC scaling unavailable

failed to set MSR 0x340d4c1 to 0x4b564d03

kvm_put_msr: Assertion 'ret == cpu->kvm_msr_buf->nmsrs' failed

Solution: Heterogeneous hibernation wakeups (with different host frequencies) are not supported, you need to wake up the hibernated VM on an isomorphic host.

6. BIOS does not have interrupt remapping enabled causing VM startup failure, /var/log/libvirt/qemu/instance-00001840.log log as follows:

Key log:

vfio error: 0000:b1:00.1 failed to setup container for group 92: failed to set iommu for container: operation not permitted

Solution: Reboot the host and turn on interrupt remapping in bios.

7. machine.slice group cpuset.cpus is empty causing the virtual machine startup to fail, /var/log/libvirt/qemu/instance-00001840.log log as follows:

Key log: Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d8\x2dcore.scope/tasks': No space left on device

Solution: This time it is due to deploying the host with hyperthreading turned on, testing with hyperthreading turned off, restarting the host and setting the cpuset cgroup resulting in an empty group. We need to find the FC or the person who deployed the domain to reset the group isolation.

8. vBMC_agent service anomaly caused virtual machine startup failure.

Key log: parse isa-ipmi failed

Solution: Restart the vBMC_agent service.

9. Large page memory setting limit causes virtual machine startup failure.

Key log: error: kvm run failed Bad address

Solution: Check whether the memory and hugetlb groups of the virtual machine are set to the upper memory limit, if so, cancel the upper limit.

If you have requirement of network equipments, please contact us www.hi-network.com  (Email: [email protected])