I/O bottlenecks in virtual environments

On November 10, 2009, in Uncategorized, by Chris Wallack

I/O bottlenecks in virtual environments

via Virtualization Pro by Eric Siebert on 10/16/09

I saw in a poll taken for a recent SearchServerVirtualization.com article that input/output, or I/O bottlenecks are the number one challenge for VMware users. This didn’t really surprise me because I/O bottlenecks are sometimes not that obvious and can be difficult to find and troubleshoot. Why is that? Because most users don’t understand how to look for them or how to interpret the data that is presented to them from monitoring utilities that would indicate a bottleneck. Some of the most common causes of I/O bottlenecks are improperly architected/configured hosts and network/storage devices, too many high disk I/O VM’s on a single host or LUN and excessive use of vSMP.

So how do you find I/O bottlenecks? You start with the built-in performance monitoring tools in the vSphere Client on the Performance tab. The esxtop command line utility can also be very helpful for looking at real-time metrics.

Using the Windows built-in Performance Monitor (perfmon) utility is usually not recommended, because some statistics may not be accurate because they are not aware of the underlying virtualization layer. VMware, however, has recently added some VM specific performance counters to perfmon that are aware of the virtualization layer and provide accurate metrics. There is also a utility introduced in ESX 3.5 called vscsiStats that can be used for storage performance analysis.

Understanding how to properly use these built-in tools to collect data and interpret it can take some practice. There are a number of free and paid third-party tools that can help you with this such as Reflex Systems VMC, Veeam Monitor, Vizioncore vFoglight, eG Monitor, Hyper9 and Netuitive Si.

If you’d like to try and learn more about performance troubleshooting that will help you identify I/O bottlenecks, check out some of the below resources: