How to Spot & Solving IO Bottleneck problem in Vmware Server 2.x with SSD HDD

In this article, I will try to point my solution for solving IO Bottleneck in Vmware Server 2.x

Firstly, why use virtual solution in the first place? Well there are many subjective reasons, and but for me my main reason is Electrical Consumption. Running multiple guest OS under the same physical Server will greatly reduce electrical consumption. At the moment, our server ( AMD X6 with 8GB RAM) are hosting 10 Guest OS : 3 pfsense, 5 ubuntu server, 1 IPCOP, 1 Windows XP. I can save up to 90 %.

Some of my friend ask me, there is gonna be performance hit by running virtualization. Of course there is, but, in real world application, it’s very rare to see your server running in 100% utilization, since most of the time, it’s doing idle/nothing. Performance hits can be spotted when it start to slow down / under perform, most of it are caused by bottleneck in CPU/Memory/IO or hardware problem.

How to Spot Bottleneck :

If you run vmware server under linux, you can use TOP command to monitor server load. Here is an example:

root@vmserver002:~# top
top – 11:06:56 up 5 days, 8 min, 2 users, load average: 1.42, 1.75, 1.70
Tasks: 244 total, 1 running, 243 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.9%us, 15.7%sy, 0.0%ni, 80.6%id, 0.3%wa, 0.1%hi, 0.4%si, 0.0%st

Mem: 8193488k total, 8118824k used, 74664k free, 38152k buffers
Swap: 0k total, 0k used, 0k free, 6919920k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

3995 root 20 0 1270m 56m 39m S 54 0.7 1772:31 vmware-vmx
3427 root 20 0 697m 251m 236m S 21 3.1 1167:04 vmware-vmx

3434 root 20 0 559m 66m 52m S 17 0.8 1739:46 vmware-vmx
4726 root 20 0 697m 205m 191m S 13 2.6 763:06.36 vmware-vmx

3443 root 20 0 707m 140m 118m S 8 1.8 304:12.98 vmware-vmx
3927 root 20 0 972m 63m 48m S 8 0.8 205:20.49 vmware-vmx

3957 root 20 0 558m 18m 6860 S 8 0.2 654:20.00 vmware-vmx
4734 root 20 0 843m 342m 320m S 8 4.3 267:09.66 vmware-vmx

4626 root 20 0 432m 129m 106m S 6 1.6 218:49.46 vmware-vmx
3492 root 20 0 706m 83m 60m S 4 1.0 184:30.86 vmware-vmx

3284 root 20 0 139m 54m 12m S 2 0.7 53:59.05 vmware-hostd
3440 root 20 0 0 0 0 S 2 0.0 62:27.59 vmware-rtc

As you can see, I run 10 guest OS, and the CPU load are +- 20% (80% are idle). And this is a production server, not testing server.

Cpu(s): 2.9%us, 15.7%sy, 0.0%ni, 80.6%id, 0.3%wa, 0.1%hi, 0.4%si, 0.0%st

legend:

us : % CPU used for user space application
sy : % CPU used for system space application
ni : % CPU used for application with nice attribute
id : % CPU not used / IDLE
wa : % CPU in waiting due to bottleneck in IO

Pay attention to the "wa" attribute. When I use regular IDE/SATA HDD, the "%wa" is most of the time reach double digit and the total of "%us" + "%sy" is hardly ever reach 10%, that means, my server is under IO bottleneck. Ever since I switch to SSD HDD, which are pretty cheap nowadays, "%wa" is always very low, just around 4-5% at peak load (at system boot or when starting guest OS simultaneously.

So, upgrade your hard drive to SSD. Your system may reside at the old IDE/SATA hard disk, but make sure the storage for guest OS are reside in the SSD HDD. Also consider investing in main board with SATA 3 / SATA 6 port. My main board is still SATA 3, with both the main HDD and SSD HDD are in the same controller, and it’s not a real issue.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: