Configuring vSphere 4.1 VM to Host DRS Affinity Rules

Introduction

VMWare vSphere’s DRS (Distributed Resource Scheduler) is mainly used for load balancing virtual machines (VMs) on a cluster. While most virtualization admins will want to run DRS in fully automated mode - i.e., vSphere decides on its own which VM is assigned to which ESX Server - there may be certain instances when you would want to enforce some conditions by setting what are known as DRS Affinity Rules.

Instances wherein you would want to enforce DRS Affinity Rules

So what are some of those instances wherein you would want to dictate which VM or VMs should (or should not) be assigned to a particular ESX Server or group of ESX Servers?
Licensing Issues
Some applications running on your VMs may have licensing peculiarities such as:

• Those that require the application to be run on only one CPU; a restriction that can have complications if you have a single-CPU server along with a bunch of dual-CPU servers.
• Those that restrict the application to one specific server with a specific serial number.

Availability Requirements
You might want to prevent a group of VMs from running on particular ESX servers.
Performance Requirements
You might want to assign some VMs to your newly acquired multiple-CPU server.

How to configure VM to Host DRS Affinity Rules

Let’s now proceed to see how you can keep a single VM or a group of VMs to either a single ESX server or a group of ESX servers.
Open your vSphere Client and find the cluster on which the rules will be enforced.
In the screenshot below, we’ll be enforcing the rules on DRS Cluster 1, which contains four ESX servers and a bunch of virtual machines.

Right-click on the cluster in question and, in the corresponding context menu, select Edit Settings.

First, we’ll create a Virtual Machines DRS Group and assign virtual machines to it.
To do that, go to the left-hand-side panel and select DRS Groups Manager. Next, go to the Virtual Machines DRS Groups panel and click that panel’s Add button.

You can now select VMs that you’d like to add to your VM group.

Click the “>>” button to add the selected VMs to the VM group. Once you’re done, give your group a name (e.g. vCenter-VM-Group), then click the OK button.

Back at the DRS Groups Manager, you can then create a Host DRS Group. This is the host DRS group where you’ll be assigning your newly created virtual machines DRS group.

Go to the Host DRS Groups panel and click the corresponding Add button.

You’ll then see a similar window as the one where you added VMs to a group (see two screenshots back). Just like what you did with the VMs, select hosts that you want to add to the host DRS group and click the “>>” button.
Note: Assigning the VM group to a host group is optional. It is possible to assign your VM group to a single host or ESX server.
Now, give the host group a name (e.g. ESX-DR-Group-1) and click the OK button.

So now you have both a Virtual Machines DRS Group and a Host DRS Group. Notice that the screenshot below shows the DRS Groups Manager tab. This is also exactly what you’ll see if you select the DRS Groups Manager in the left-hand-side panel.

Create DRS Affinity Rule

You are finally ready to create your DRS Affinity Rule. Click the Rule tab and come up with a name for your rule (e.g. Pin-vCenter-VMs-to-ESX-Group-1).
Now, for the following drop-down lists, select the following items:

• Type = Virtual Machines to Hosts
• Cluster VM Group = the Virtual Machines DRS Group you created (e.g. vCenter-VM-Group)
• Cluster Host Group = the Host DRS Group you created (e.g. ESX-DR-Group-1). This is where the VM group is supposed to run.

Notice that we skipped one drop-down list. In the screenshot, it’s the one that says “Must run on hosts in group”. That drop-down list is where you’re supposed to select the actual rule that will apply to the VM group-host group pair.
Expand that list, select the rule you wish to implement, and click OK.

That should create your VM to Host DRS Affinity Rule.
You’ll see something like this (see screenshot below) in the succeeding window. You’ll have to click the “+” symbol to expand the VM to Host Rule and reveal the corresponding Cluster VM Group and Cluster Host Group on which the rule will apply.

That’s it. You have just created a vSphere 4.1 VM to Host DRS Affinity Rule.

Understanding Memory Compression in vSphere 4.1

Introduction

vSphere 4.1 comes with a new performance-enhancing feature called memory compression. It can give your system a performance boost when it starts running low on RAM. Just like swap memory, memory compression is not equal to real memory but can help to prevent degradation of performance when you're running low on memory. For comparison, memory compression is faster than swap on disk but slower than real memory.

Where to configure Memory Compression in vSphere 4.1

To start, go to your vSphere Client and select an ESX server whose memory compression settings you’d like to configure. Now, navigate to that server’s Configuration tab and, in the Software panel, click the Advanced Settings link.

In the Advanced Settings window, open the Mem set of settings by clicking that name in the left-hand-side panel. The settings we’re looking for are found near the bottom, so you’ll have to scroll down until you reach the first setting that starts with Mem.Mem.Zip.

All settings related to memory compression start with Mem.Mem.Zip. For instance, there’s Mem.Mem.ZipEnable, which allows you to Enable (set to 1) or Disable (set to 0) Memory Compression. Other memory compression settings include:

• Mem.Mem.ZipMaxPct
• Mem.Mem.ZipMaxAllocPct
• Mem.Mem.ZipLowMemMaxSwapOut
• Mem.Mem.ZipBalloonXferPct
• and so on

Below each setting, you’ll see a short description of what each setting does as well as their corresponding Minimum and Maximum values.
All settings take effect after you click the OK button.
Now that you know where to make changes to the Memory Compression settings in case you really need to, let me show you where you should go to find out if those settings really need any tweaking.

How to check performance information related to Memory Compression

Again, make sure the right ESX Server is selected, then go to that server’s Performance tab. Now, expand the drop-down list right beside the label that says Switch to: and then click Memory.

Initially, the next window won’t contain any stats regarding memory compression. You’ll have to add them first. To do that, click the link that says Chart Options. I’ve enclosed that with a yellow circle in the screenshot above.

In the succeeding window, go to the Counters panel and scroll down that vertical scroll bar. The counters related to memory compression are the following:

• Compression rate
• Decompression rate, and
• Compressed (which refers to the amount of compressed memory)

Each time you select a counter, a short description about it will be given in the Counter Description panel.

After clicking the OK button, you’ll be brought back to the previous window where you’ll then see the newly added memory compression stats in the graph as well as in the Performance Chart Legend.

The graphs representing the newly added memory compression-related counters are not yet visible in the screenshot shown above. That’s because, as mentioned earlier, memory compression kicks in only during times when the system is running low on RAM.

Summary

This article provides an overview of memory compression, a new feature in vSphere 4.1. We explained how memory compression helps to prevent degradation of performance when you're running low on memory, and how it's not as fast as real memory but a lot faster than swap (on disk). Additionally, we covered configuring memory compression on your vSphere virtual infrastructure.

What is esxtop?

Simply put, esxtop is VMware's command-line tool to analyze performance.

Where did it come from?

If you've used the Linux or Unix top command, esxtop may already be familiar to you. Esxtop is VMware's version of this popular "top" command that run on an ESX or ESXi server. Both top and esxtop run only at the command line of a server. You would access these commands by either going directly to a server console or by connecting to a server remotely via SSH (or telnet, if enabled).

What does esxtop do?

Esxtop is used to analyze real-time performance data from an individual ESX or ESXi server. You bring it up by going to the physical server console or remote console via SSH, logging in, and typing esxtop.
Most people would say that esxtop analyzes CPU, Memory, Disk, and Network statistics. More specifically, you'll find that esxtop has 8 different "displays" that show CPU, interrupt, memory, network, disk adapter, disk interface, disk VM, and power management, as you see in the screenshot below.

In the graphic, you can see that each of these different displays are brought up by pressing the corresponding letter for each display (i.e., press "d" for the "disk adapter" display).
Once in a display, you'll always have uptime and CPU trending info at the top of each screen. From there, you'll have a column view that shows stats for different objects. For example, in the CPU view you have processes running on the host down the left side (which could correspond to a particular VM) and columns across the top with different stats for each of those processes.

In the graphic above, you see the CPU display rolled up to just show virtual machines. For each VM there is information about the percent of CPU used allocated. One of the most valuable stats is "CPU Ready" (which you can learn more about here, or in my vSphere Performance video below).

What's the difference between esxtop on ESX and ESXi?

You'll find that you can run esxtop on either ESX or ESXi without any trouble. The only difference will be that esxtop on ESX will show information about the service console. On ESX, you'll also find a nice man page for esxtop that you can access by typing man esxtop. In fact, I have used that man page to figure out what I want to do with esxtop then gone over to my ESXi server to do it.

What's the difference between esxtop and resxtop?

You'll see both esxtop and resxtop mentioned when talking about esxtop. The only difference is that esxtop is run locally on an ESX or ESXi server. Resxtop, on the other hand, is "remote esxtop" and it is only used to run esxtop on another server. For example, you would use resxtop to monitor performance of an ESXi server if you were currently logged into a vMA (vSphere Management Assistant) virtual appliance.

When running resxtop, you will have to specify the ESX or ESXi server hostname, username, and password, as you see below:

How do you control esxtop columns and sort order?

Besides looking at different displays in esxtop you can also do two other things to manipulate what you see:

1. Adding or removing fields - when using esxtop you want to make sure that your SSH appliance is the full width of your monitor to get as much on the screen as you can. It is common for the many columns fields provided by esxtop to run off the screen and for you to not be able to view them. In fact, the fields that are off the screen may be the fields that you want, so resizing the console window or removing fields may be necessary to get the info you are looking for. You can add or remove the fields that are shown with the f or F key. This will take you to a screen like in the graphic below that allows you to select (using letters) which fields you want to remove or add. An asterisk next to a field means that it is currently being displayed. When you press the corresponding letter for that field, the field is either added or removed.


2. Changing field order - you can also change the order that the fields are displayed in. This is done by pressing the o or O key which will bring up the order screen below. From there, similar to adding or removing fields, you will use letters to move fields up and down in the order of display.

What are the best resources to learn more about esxtop?

In this post my goal was to give you an overview of what esxtop is and the basics of how it works. However, esxtop is a powerful advanced performance monitoring tool, and there is a lot more that you can do with it.

Upgrading to vSphere 4.1 Overview

vSphere 4.1 Upgrade Considerations

Here's what you need to take into consideration when upgrading to vSphere 4.1.

Downtimes

Some of your end users might be affected during the vSphere upgrade. You should therefore figure out ways to reduce or perhaps even eliminate possible downtime. If you have two ESX servers, for example, then you can employ VMotion and shared storage, which will enable you to move running virtual machines from one server to another without disturbing your end users.
Basically, you can move running virtual machines to, say, ESX server 1 and perform the upgrade on ESX server 2. When you're done, you can then move the virtual machines unto ESX server 2 and then perform the upgrade on ESX server 1.

Backups

If vCenter is running as a VM, then you should take a snapshot of that VM before carrying out the upgrade. So that if something bad happens along the way, you can easily revert to that snapshot. In addition, you might want to backup your vCenter server as well as your ESX or ESXi servers in order to retain custom configurations you might have on them.

vSphere 4.1 Upgrade Prerequisites

An upgrade to vSphere 4.1 also requires the following:

• vCenter running on a 64-bit server with a 64-bit OS.
• In case your vCenter doesn’t run a 64-bit server, you can use the VMware vCenter Data Migration Tool to remedy the situation. More about this tool in a short while.
• A 32-bit DSN for vCenter Update Manager
• Compatible operating systems for vCenter Server, which includes:
• Windows XP Pro SP2, 64-bit
• Windows Server 2003 SP1, 64-bit
• Windows Server 2008, 64-bit

vSphere 4.1 Upgrade Process

The basic upgrade process entails the following general steps:

1. Upgrade vCenter
2. Upgrade vSphere Client
3. Upgrade ESX / ESXi servers using VUM (vCenter Upgrade Manager) or vihostupdate
4. Upgrade VMware Tools on each VM

 vSphere 4.1 vCenter Server Data Migration Tool

As mentioned earlier, VMware has a tool for those who don't have vCenter running on a 64-bit server. Since there is no in-place upgrade path for vCenter Server installation on 32-bit systems, you will have to migrate your vCenter from your 32-bit system to a 64-bit system. This is where you'll need the Migration Tool. It will help you migrate the following items from one server to another:

• vCenter Server and its configurations,
• vCenter Update Manager and its configurations,
• VMware Orchestrator and its configurations, and
• the default SQL Express 2005 database that comes with vCenter Server.

How to use the vCenter Agent Pre-Upgrade Check Tool

The vCenter Agent Pre-Upgrade Check Tool, which has just been introduced in vSphere 4.1 and can be found in the vCenter Server installation media, is used for performing diagnostic checks prior to in-place upgrades from vCenter 4.0 to 4.1.
Again, it only performs diagnostic checks, so don't expect it to fix any issues. In fact, it is still possible to encounter issues even if the Pre-Upgrade Check Tool doesn't find any during its diagnostic run. However, this tool can help you a lot in checking whether all the major prerequisites for the upgrade have already been met.
So assuming you're in your version 4.0 vCenter Server and you've already inserted the vCenter 4.1 installation media, double-click the installation media icon (in Start > Computer) to start the autorun program and to launch the splash screen.
On the splash screen, click Agent Pre-Upgrade Check.

When the Agent Pre-check Wizard welcome screen appears, just click the Next button.

In the Select Database window, select the DSN you want to connect to from the drop-down list and enter the appropriate User name and Password. In our case, we selected the Use Virtual Center Credentials option, which prompted the wizard to populate the User name and Password fields automatically.

Click Next.

In the next window, you'll be given the option to scan either all ESX/ESXi servers or specific ESX/ESXi servers. Choose Standard Mode to scan all servers and click Next.

In the following window, click the Run precheck button to start scanning; that should only take a few minutes. If everything goes well, you should see something like this:

Click Next.
You'll then see your ESX servers with a Pass notice right beside each one. If you want to generate a printable report, click the View Report button.

If there are no errors, you'll see a simple summary just like this:

Close that window to go back to the AgentUpdateChecker status + results window and click the Next button. You'll then be informed that you have successfully completed the Upgrade Pre-check Wizard.

Power Consumption Graphing and Control in vSphere 4.1

Hardware Power Management Requirements

For this power saving feature to work, your server must have the capability to support any of these power management technologies: Enhanced Intel SpeedStep or Enhanced AMD PowerNow!
In addition, you may also have to enable this feature in your server's BIOS to allow vSphere to control the CPU frequency, a process sometimes known as Demand Based Switching.

How to Configure Hardware Power Management in vSphere

As mentioned earlier, the main setting of controls takes place in the vSphere Client. Once you're in there, select the server whose power management policy you'd like to configure, click the Configuration tab, then click the link named Power Management found under the Hardware section (not the Power Management link under the Software section).

Notice that it shows the power management technology currently being used. In my case for example, it's Enhanced Intel SpeedStep. If your server does not support any of the technologies mentioned earlier, then it will state this accordingly.
Secondly, that section also shows the power management policy that's currently active, e.g. High Performance. After having said that, this should be the perfect time to take up the possible CPU power management policy values that can be displayed. Here they are in a nutshell.
CPU Power Management Policy Values:

• Not supported - This will be displayed if either the host doesn't support any power management features or if the power management feature is simply not enabled in the BIOS.
• High Performance - This is the default value. It signifies that the VMKernel detected certain power management features, but will not use them unless the BIOS issues a request for power capping or thermal events.
• Balanced Performance -  Signifies that the VMKernel is using available power management features to lower host energy consumption without compromising performance.
• Low Power - Means that the VMKernel is aggressively using available power management features to lower host energy consumption at the possible expense of performance.
• Custom - Displayed if the VMKernel implements specific user-defined power management features based on the values of advanced configuration parameters. These parameters are set in the vSphere Client Advanced Settings dialog box.

Now that you know what the possible power management policies are, you're ready to pick a policy that would best fit your energy/performance needs. To change the current policy, click the Properties link (see encircled text in the last screenshot above)
This will take you to the Edit Power Policy Settings dialog box. Choose your desired power management policy.
For example, if you want to reduce energy consumption without compromising performance, click the Balanced option then click the OK button.

After clicking Refresh, you'll notice that the Active Policy changes to Balanced.

You can repeat this process on all the servers that you'd like to change the power management policy.

vSphere 4.1 Power Consumption Graphs

After enforcing your desired power management policy, you might want to monitor the effects of your settings by viewing the power consumption graphs that come with vSphere 4.1. Select a server you want to monitor and go to its corresponding Performance tab. Next, click the Advanced button and, in the Switch to drop down list, select Power.

If all goes well with your new (and presumably more conservative) power management policy, then that graph will reflect it. You may show additional information on the graph by clicking Chart options (encircled in the last screenshot). That will take you to the Customize Performance Chart window.

Go to the Counters section and check the counters you would like to add. For example, you can check all of them. After that, click OK.

Here's how that would look like on the updated graph.

You can do that for all your ESX and ESXi servers.
That's it. I hope you can take advantage of the power consumption graphing and control features in vSphere 4.1 and add even more to your savings.

VMware vSphere 5 Video Training – Now Available

I am excited to say that with the launch of vSphere 5,


I worked with vExpert and veteran vSphere instructor, Elias Khnaser to create this 16+ hour course. In this course, we both cover the installation, configuration, and management of vSphere as well as all the powerful new features of vSphere 5.

Using vSphere Hot-Add to Dynamically Add CPU and RAM

Overview

There might be instances when an application running on a virtual machine starts consuming a lot of resources and, as a result, the VM starts running out of RAM and CPU. If the application is mission critical and has to be highly available, you can’t just shut it down to add more RAM and CPU. These are the kind of instances when a Hot-Add and Hot-Plug feature can come in handy.

What you need to know about adding, modifying and removing virtual hardware

In most cases, only very few types of hardware can be added “hot”, i.e., while the VM is running. For all other types of virtual hardware, the VM has to be powered off before they can be added or removed. In addition, your operating system and applications have to be configured to recognize the virtual hardware you want to add.
Let me show you a running virtual machine that hasn’t been enabled yet with Hot-Add/Hot-Plug features.
To view a virtual machine’s hardware properties, go to your vSphere client, select a virtual machine (for this example, we selected vMA), click Edit Settings under the Summary tab and when the Virtual Machine Properties window appears, go to the Hardware tab.

Click on Memory and see if the Memory Configuration section is grayed out just like the one below.
Just to remind you, we’re assuming your VM is running while you’re doing this.

If it is grayed, that means we can’t modify the amount of RAM. This is what you’ll see if your VM doesn’t have Hot-Add/Hot-Plug capabilities.
Now, click on CPUs to see if you can modify the number of virtual processors. If the area on the right is grayed out just like the Memory Configuration, again, that’s because you don’t have Hot-Add/Hot-Plug capabilities.

Try clicking the Add button. That’s the button you’re supposed to click if you want to add a device to this virtual machine. Consistent with what we mentioned earlier, you’ll notice that a lot of the devices there are labeled (unavailable). Only Hard Disk and SCSI Device, can be added.

You’ll also notice that when you go back to the previous window and click all the devices found there, most of the devices cannot be removed. That is the Remove button is mostly grayed out. In our case, the only time it wasn’t grayed was when we selected the Hard Disk.
As you can see, you really have very limited options when it comes to adding, modifying, and removing hardware on a running virtual machine without the Hot-Add/Hot-Plug features.

Obstacles you may encounter when using vSphere Hot-Add/Hot-Plug

Before we proceed to show you how to use Hot-Add/Hot Plug, let’s first talk about some of the obstacles/limitations you may encounter when using these features.
Note that, while vSphere supports Hot-Add and thus allows you to add RAM dynamically, it does not support hot-remove RAM. More importantly, support for these “hot” features are largely dependent on your guest OS; not on vSphere.
Here’s a table from Jason Boche of blog.boche.net showing which Windows Server editions, when running as guest operating systems, support specific Hot-Add/Remove and Hot-Plug/Remove features:

In addition to those operating systems, you might like to know that the following OSes support hot plug (CPU) without requiring a reboot:
● Windows 7 Enterprise and Ultimate 64-bit
● Windows Server 2008 64-bit Datacenter
Sad to say, both Hot-Add and Hot-Plug are NOT turned on by default on a virtual machine. Also, to activate these features, the virtual machine has to be powered off. This can be quite a hassle because, normally, you only realize that you need more CPU or RAM when everything is already running.
Therefore, if you think there is the slightest possibility that a VM will be needing more system resources during operation, e.g. if it is a mission critical VM, don’t forget to activate Hot-Add and Hot-Plug beforehand.
Finally, note that Hot-Add/Hot-Plug are not compatible with VMware Fault Tolerance. So if you’re already using VMware FT on a VM, then don’t activate Hot-Add/Hot-Plug there.
Now that you know what Hot-Add and Hot-Plug are, it’s time to show you how to put them into action.

Enabling Hot-Add and Hot-Plug

To enable these features, go to your vSphere Client and power off the VM you want to activate Hot-Plug/Hot-Add on. Make sure the guest OS on that VM supports Hot-Plug/Hot-Add (refer to the table shown earlier) before proceeding. If the guest OS supports it, then select the powered-off VM and, in the Summary tab, click Edit Settings.

In the Virtual Machine Properties window, go to the Options tab, and select Memory/CPU Hotplug. Next, go to the Memory Hot Add section and select the “Enable memory...” option. In the same manner, go to the CPU Hot Plug section and select the “Enable CPU...” option. Then click OK.

At this point, you may now power on the VM. If you want to see how your CPU and memory resources look like from inside the guest OS itself before and after Hot-Add/Hot-Plug are put into action, open the console for that VM and log into the guest OS.
Once you’ve logged in, go to the Windows orb, right-click on Computer, and click Properties.

There you’ll see your currently installed RAM. For example, we currently have only 1 GB of RAM.

To view the current number of CPUs, go to the panel on the left side of that screen and click the Device Manager. Inside the Device Manager, expand the Processors item to see the number of processors. In our example, we have only one processor.

Later, if you want to verify whether Hot-Add/Hot-Plug really changed the amount of RAM and the number of CPUs, you already know where to look.
Let’s now add more RAM and CPU to your system.

How to Hot-Add RAM and Hot-Plug CPU to a Running VM

Go back to your vSphere Client and select the VM in question. Again, go to the Summary tab and click Edit Settings as shown earlier. Once you’re back inside that virtual machine’s Properties window, go to the Hardware tab and select Memory.
You’ll now see that the Memory Configuration section is no longer grayed and that you can already change the memory size. Go ahead and do so. Remember you’re doing this while the VM is running.

After adding more RAM (e.g. we changed ours from 1 GB to 3 GB), you may also add more CPUs. Just click CPUs from the list of Hardware. Add CPUs by selecting a number from the drop-down list at the right (e.g. we changed ours from 1 CPU to 3 CPUs)

Click OK to proceed.
You can easily see the results of the Hot-Add/Hot-Plug operation on the virtual machine’s Summary tab.

If you’re not contented with that and want to see the results of this operation from right inside the guest OS, go back to that virtual machine’s console and into the guest OS. If you navigate back to the places we showed you earlier, you will now see that the changes have in fact taken place.
Here’s are the newly reinforced CPUs:

And here’s the newly beefed up RAM:

Summary

This shows that we’ve succeeded in adding CPUs and RAM on a running virtual machine by using Hot-Plug and Hot-Add.