Mike D’s Blog

This post is sponsored by IT Knowledge Exchange. Visit ITKnowledgeExchange.com today to ask your toughest virtualization questions and get answers from your peers.

I got a question from a customer as well as a partner recently about whether or not they should use Perfmon in a Windows VM. I emailed our internal performance team to make sure that my answer of “no – it could give you bad results” was an ok answer. Turns out they’re in the process of writing a KB article on the subject and with their permissions I’ve pasted the draft of that KB article below. NOTE: This applies to ESX 3.5 and prior. Talk to your local VMware SE to get some insight into how things are changing. Unfortunately, I can’t talk about future offerings that haven’t been publicly talked about (press, VMworld, etc) in a blog posting.

———– The Draft KB ————

Do not use guest OS performance tools to monitor virtual machine performance

Details

There is often a difference between performance measurements taken from within a virtual machine’s guest operating system and those taken by VMware Infrastructure monitoring tools, such as VirtualCenter and esxtop.

The guest OS performance counters do not take into account the host’s hardware or the external constraints that might be set on the virtual machine, for example, resource reservation limits of a parent resource pool. As a result, resource utilization measured from within the guest OS might not reflect the true overall resource utilization measured at the physical hardware level. For example, consider an ESX/ESXi host running five virtual machines, each of which reports 100 percent CPU utilization by the guest OS tools. The guest does not know it is sharing the host CPU resources with other guests. On the other hand, the VirtualCenter Server and other virtual infrastructure monitoring tools are aware of these distinctions and report that each individual virtual machine is using 20 percent of the available CPU resources on the ESX/ESXi host. The host is 100 percent utilized and divides available resources among all virtual machines.

Inaccurate statistics can also the result of the virtualization system offloading certain tasks from the guest and virtualization overhead requirements. For example, the guest tools are not aware of the overhead memory required by the VMkernel.

Solution

To ensure the performance numbers are accurate for a virtual machine, perform functional testing on the virtual machine. The internal workings of the virtual machine should not be known by the tester or the performance tools. Follow these guidelines:

• Use performance tools that are located outside of the virtual machine you are monitoring.

• If possible, run the performance tools on a host where the virtual machine is not running.
• For load tests, generate the load outside of the virtual machine being measured.
• Do not use benchmarking tools. Instead, measure real applications.