Wednesday, December 30, 2009

To virtualize Exchange or not to virtualize Exchange?

A very common question I hear when talking to customers is whether or not they should run Exchange in a virtualized environment. The answer I give them is multifold: yes, there are benefits to running Exchange virtually, but you must carefully think first about the prerequisites and make sure it is the best move for your environment.

Virtualization is not magical. Some people have the misunderstanding that because you run virtualized, you don’t need to size your servers. This line of thinking will quickly get you in trouble. You still need to figure out how much RAM, CPU and storage you need, both for volume and for performance reasons. The rules are simple: scale your server in the same way you would if you were using physical hardware, and then apply it to your virtual server.
If you figure out that your server is going to need 16GB of RAM and 1000 IOPS, then make sure that the virtualized server has the same resources available; otherwise, your Exchange servers will have performance issues.

Performance. Think about what virtualization environments look like today when considering your hardware’s performance, you should take into account how virtualized environments are deployed. Most companies deploy one or more physical servers, possibly in a clustered configuration, hosting several virtualized servers on each physical server. This means that each physical server must be able to handle the load for every virtual server instance running on the physical one.
With today’s hardware, this is most likely not a problem if you think about CPU or memory (CPU and RAM are not that expensive and can be added later if needed); but when it comes to storage, that is another story. Every server needs disks, both for booting from and for saving the application data to. If running 5 servers on one physical, then the physical server must have 5 times the disk volume—always keep in mind the importance of performance.
For example, let’s consider the following configuration:
Imagine that you have 5 virtualized servers, each having a 50GB disk for OS and a 100GB disk for storing data, which is 150GB times 5 servers and you end up with 750GB. This is no problem for modern disks since a single disk can easily hold 750GB of data. But if you would have run those 5 servers on physical hardware, then you might have put in 4 spindles and created 2 mirrors with 2 disks each. This would render you a fairly good performance on disk. Then if you also have 5 servers with this configuration, that results in 20 disks.

Now compare that disk performance to the single disk performance. Exchange designers have for a long time been used to and forced to think in number of spindles instead of volume because of the disk performance, but this knowledge isn’t that widespread and there is a chance that the people who maintain the virtualization platforms don’t have this knowledge. With this being said, you most likely end up with your virtualized environment connected to a pretty beefy storage containing a lot of disks to withstand the IO performance need.

Virtual hosts affect each other. With several virtual machines running on a single hardware, one virtual machine that is running very high on CPU can drain the physical server on CPU resources, leaving the other virtual machines with little to no CPU resources, causing them to perform slowly. Virtual platforms have configuration settings to limit this behavior, both from draining resources and for maintaining some amount of resources for virtual machines.
This applies to not just CPU resources, but to all other resources also shared by the physical server.

Virtualization adds complexity. By now you’ve gathered that virtualization adds complexity. The most likely scenario is that you end up with a bunch of virtualized servers with disk files located on a SAN. There is also a good chance that you need some more education to maintain not just the ordinary Exchange server environment, but also to manage the virtualization platform as well as the SAN infrastructure.
There is always going to be a time when something happens to the environment for unknown reasons, which inevitably leads to really complex troubleshooting. In smaller companies there is often one person maintaining everything in IT; but in larger companies there are several departments maintaining one piece of the puzzle, making troubleshooting even more complex! Situations might arise that involve several people blaming each other, saying things like ‘There is nothing wrong with my SAN!’ I’m sure many of you have taken part of such conversations.

Virtualization is not free. The complexity mentioned above is not free, unfortunately. Education costs money and time. If you also add the time spent on maintaining multiple systems and most likely some troubleshooting to get them working together, you can easily see that it will cost more money than a standard windows server with Exchange on it. An ordinary windows technician should be able to maintain a standard windows box with perhaps some local attached disk drives.
On the other hand, what can be free under some circumstances is the software license for the virtualization platform. Keep in mind, however, that this is often a small amount of money compared to all the labor and education costs that will be incurred. Plus, there might be costs associated with putting the environment in a SAN.

Flexibility. Virtualization technology is great for flexibility. It is often very simple to add resources such as disk or memory to a virtualized server, and if done correctly it is easy to add servers as they are needed. With the easy provisioning of servers, virtualization is great for lab environment where you often need to add or restore servers quickly. In a lab you can also test things and don’t need to be afraid of breaking your system since it is easy to restore them.
There is always the element of patching a running system. The process for patching a virtualized server is the same as for a physical one. Some people argue with this and say that they can do a snapshot of the server before patching and if something breaks they will just roll back the snapshot. This, however, is not feasible with Exchange since it relies on the configuration being in Active Directory.

Consider this scenario:
There is a patch for Exchange available that you want to install, so you do a snapshot of your virtualized Exchange server and apply the patch. After the reboot or restart of services you notice that the patch doesn’t work in your environment. ‘No problem,’ you think, ‘I will just go back to my snapshot.’ This is not the best course of action, however. Not only because you will “go back in time”, making people lose mail between the time when your snapshot was taken up to the present time, but also because this patch wrote or changed something in Active Directory. By going back, the installation before the snapshot doesn’t like the information being present in Active Directory, causing Exchange to fail. I don’t think this will be a common problem with Exchange, but it is something to be aware of. My recommendation is to do a snapshot of AD and Exchange at the same time as the rollback, not separate.
Exchange supportability Running virtualized is not limited to use of Microsoft technology with Hyper-V. Microsoft has a program called Microsoft Server Virtualization Validation Program (SVVP) This program allows other vendors to go through tests so that their virtualization technology can be validated and approved by Microsoft. Being a vendor that is a member of SVVP makes it supportable by Microsoft to run Exchange on; therefore you are not limited to Hyper-V.

Microsoft has published a document with policies and recommendations for running Exchange 2007 and Exchange 2003 virtualized, and can be found here:
Information about Exchange 2010 will be published shortly, but in essence it will be very much the same as for Exchange 2007.
This document should be read carefully by the people doing Exchange design work, otherwise you may be out of support from Microsoft. Most important point to remember is that your storage and high availability are designed correctly.

Security is of course something to think about, and the virtual servers we treat the same way as if they were physical. Those servers must be protected and patched just like any other server. There is also the challenge of who can access and manage the virtualized environment? Running Exchange in a virtualization environment adds complexity in terms of security, and is definitely something to consider carefully.

Ask yourself some questions:
What if someone could get access to the physical servers and simply copy the disk file? That person could then sit quietly and try to withdraw data from the file. Thus, it is important to think about permission on files, servers and management tools.
What about if you move a virtualized Exchange server to a physical server that is not located inside a locked computer room, or to a hardware that doesn’t have the needed resources? As you can see, security around virtualization involves many components.

Now that you have all the information, let’s return to the original question about running Exchange in a virtualized environment. Ready for my final answer? Yes, it can be done and many benefits will be incurred as a result, but you must pay close attention to the design outside of Exchange and keep in mind all the prerequisites, pitfalls, and misconceptions that you could face.