VirtualPro

Virtual Appliances have been around for quite some time now and it seems with the recent release of vSphere, VMware are looking to take vApps to the next level. The VMware Virtual Appliance Marketplace (VAM) is the one stop shop providing VMware users with a plethora of pre-configured, ready to download appliances.  There are over a 1000 appliances available in the VAM, all of them capable of being deployed in either the private cloud or in a cloud hosted by a 3rd party.  Now the VAM is also offering on-demand trails of certain vApps through selected vCloud partners.  The vCloud vision is slowly taking shape and and although it may be mostly conceptual now, small steps like this are the real building blocks.

With the release of vSphere 4.0 VMware introduced full support for the OVF 1.0 specification.  What does this mean?  well the Open Virtualisation Format is a platform independent, efficient, extensible, and open packaging and distribution format for virtual machines.  It’s virtual platform independent so if vSphere isn’t your platform of choice the vApp will work with Hyper-V and XenServer. OVF virtual machines are optimised for easy distribution are simple to deploy and support single and multi VM configurations.

Now I spend a lot of my time integrating and supporting applications from 3rd party vendors,  some of them mainstream and some of them more specialist. One of the constant issues I have is around how best to deploy them,  what are the best practices, minimum specifications and correct configurations to ensure successful deployment and reliable operation.  This is something I can see vApps helping all of us with, vApps created using the OVF format will allow vendors to build and maintain pre-packaged systems. Pre-configured to be highly compatible, built to the vendors own best practices and best of all built for rapid deployment to customers regardless of virtualised platform or cloud prefernce. The way I look at it,  it’s not to disimilar to Apple’s approach to controlling the hardware and the software they use. The way Apple operate allows them to guarantee better reliability and compatibility because they know what they’re deploying and what they’re deploying it on. A vendor built vApp has the potential to offer the same benefits and simplify the job of vendors, system integrators and application support teams alike.

So how are VMware assisting vendors in their pursuit of vApp bliss? With the creation of VMware Studio of course,  It was a product I’d never heard of until the beta release of VMware Studio 2.0 was announced earlier this month. The latest features are listed below.

Build vApps and virtual appliances (with in-guest OS and application components) compatible with VMware Infrastructure, VMware vSphere 4.0 and the cloud

- Support for OVF 1.0 and 0.9
- Available as an Eclipse plugin in addition to the standalone version
- Ability to accept existing, Studio-created VM builds as input
- Support for 32 bit and 64 bit versions of Windows 2003 and 2008 Server
- Support  for SLES 10.2, RHEL 5.2 and 5.3, CentOS 5.2 and 5.3 and Ubuntu 8.0.4.1
- Publish patches to update deployed virtual appliances
- Extensible in-guest management framework
- Automatic Dependency resolution
- VMware ESX, ESXi, Vmware Server 1.0.4 - 2.0, VMware Workstation enabled as         provisioning engines.
- Infrastructure enhancements in the GUI and builds

So what does a vApp deployment look like,  well here’s a video that was posted on the VMware vApp developer blog.  In this demo a user deploys a mulit VM, multi-tiered version of SugarCRM in just a few clicks with no need to even start up a VM Console. 

General vApps, vCloud, VMware

I recently had a number of vSphere ESX4i  USB Key installs following my article on putting vSphere ESX4i on a USB key / Pen Drive. I needed to format a couple for general windows usage, only to find that the ESX4i image creates a number of partitions on the USB Key. Unfortunately Windows does not appear to support the removal of partitions on removable devices so when I was trying to format a 2GB USB stick I was able to format a 110MB partition and that was it. I was a bit stuck on the best way to rectify the issue and wasn’t finding much to help out on the web.

That’s when I stumbled upon the HP USB Storage Format Tool,  a great little tool that works with a wide range of USB sticks and not just HP ones.  It allowed me to wipe the USB key as a single entity and didn’t care about the partitioning, returning my USB Key to a useable state within windows.

You can download it from HP’s website by clicking on this link,  sometimes you just don’t know if you can trust other download sites.

General ESX 3.5i, esx4i, USB, VSphere

I’ve recently been evaluating some of the new features in VMware vSphere to see what use they would be to my current employer. One of the areas that I touched upon in my “what’s new in vSphere Storage”  blog post was thin provisioning.  I wanted to come back and cover this particular topic in more detail as it’s a key feature and it’s available throughout all versions of vSphere so I’m sure everyone will be interested in it.

What is Thin Provisioning?

Thin Provisioning in it’s simplest form is only using the disk space you need.  Traditionally with virtual machines if you create a 500GB virtual disk it will use 500GB of your VMFS datastore. With Thin Provisioning you can create a 500GB virtual disk, but if only 100GB is in use only 100GB of your VMFS datastore will be utilised. Credit to Chad Sakac for the diagrams below.

How does it work in vSphere?

Thin Provisioning is being heralded as something new with vSphere,  when in truth it was already available in VI3.  In VI3 creating a thin provisioned disk involved using vmkfstools and was also not a production supported VM configuration.  Now in vSphere the creation of thin provisioned disks can be carried out from the VI Client (see below) and is a supported production configuration for a VM.

It’s as simple as checking a check box, the results are pretty good to.  Below you can see I have created two thin provisioned VM’s on my new ESX4i host and you can see the provisioned space and the used space stats being shown in the VI Client. 

What are the benefits?

The thin provisioning  feature is perfect for my home lab environment where disk space is at a premium, but how does it translate into real world implementations of ESX.  Well I for one have been looking at exactly this to identify what benefits could be achieved within my employers ESX estate. A quick audit found that our development and system test ESX environment was running at 48% disk utilisation,  so straight away thin provisioning would save us 52% on storage capacity used. Paul Manning of VMware mentioned on a recent communities podcast that on average vSphere would save users 50% on storage.   This is possibly not such a big thing when your talking about test environments, but when you move up to production SAN Storage, saving 50% on an expensive SAN array is a very real and tangabile cost saving.  One that people should definately take into account when making a cost benefit case for buying or upgrading to vSphere.

What are the potential downsides?

One of my personal concerns with thin provisioning is the potential overhead on any write activity that would requires the extension of the VMDK file.  To me there is an obvious VMFS operation that needs to take place there which would add to the overall time to complete the disk write.  When there is a requirement to expand a disk, the VMDK files will increase in increments based on the block size of the underlying VMFS partition, 1MB, 2MB, 4MB or 8MB.  So the overhead may be smaller if your VMFS has been formatted with a bigger block size, i.e. for a 16MB write it only has to expand 2 blocks when the VMFS block size is 8MB but would have to expand 16 blocks if it was formatted with the 1MB block size.  I can imagine this percieved overhead could put people off using thin provisioned disks for certain production based environments, especially those where there is a lot of write I/O activity,  SQL Server or Exchange for example.  To counteract that though,  the improvements in the VMware I/O Stack should compensate for this performance overhead.  This could potentially leave you in a situation where you’ve reduced a VM’s storage footprint and still have performance equal to that experienced in VI3,  possibly not a bad trade off.  I’d also expect people running their VMware environment on enterprise SAN technologies from the likes of EMC or NetApp to notice minimal performance impact with thin provisioning as SAN memory caches help take up the strain.

Another downside is if you want to use VMware Fault Tolerance to protect a VM then you cannot use thin provisioned disks.  To be honest this is a small issue as Fault Tolerance protection is most likely going to be on virtual machines that are important to your organisation.  These machines are probably the ones you wouldn’t thin provision in the first place for performance reasons.

Thin provisioning creates it’s own unique problem in that what we’re basically doing here is over provisioning the storage.  You need to keep a very close eye on thin provisioning as it’s quite feasabile that your VMFS datastore could fill up and your virtual machines fall over.  Not what you want to come into on a Monday morning,  or any morning for that matter.  So you need to monitor your storage and ensure that there is enough free space.  One of the simplest ways to do this is through the use of the new alarms in vSphere that allow you to alert on datastore usage and datastore over provisioning.  These should keep you from filling a datastore and killing your VM’s or ESX Servers

One gotcha that you should watch out for is VM swap files, as these are usually stored with your virtual machines vmdk files in the VMFS datastore.  In VI3 the swap file was not deleted when a VM was powered down,  in vSphere the swap file is deleted on power down and recreated when the VM is powered up.  You should be aware of this when over provisioning storage as you could get into a situation whereby you find you can’t power on a VM because there isn’t enough space for the swap file to be created.  This becomes more likely as servers and VM configuration maximum’s increase,  if you have a VM with 20GB of RAM it’s going to need 20GB of disk space for the swap file.  if you have 256GB of RAM in your vSphere host and you allocate it all out to VM’s then you need to think about the 256GB of disk capacity required to service virtual machine swap files.

Storage vMotion

If you’ve already got a VI3 environment then the chances are that your VM’s aren’t thin provisioned,  how on earth are you going to take advantage of this new feature? Well if you have purchased a vSphere edition that supports storage vMotion then you can of course migrate the underlying storage and have it thin provisioned during the move.  This should allow existing VI3 customers to claim back a lot of space,  as I mentioned before I found that our development and test VI environments were only 48% utilised.  If I storage vmotion all those VM’s and thin provision at the same time I will free up about 1.5TB of storage that wasn’t being used in the first place.

I’ve included a video below which demonstrates the Storage vMotion and thin provisoning features in vSphere quite nicely, enjoy!

Storage, VMware, vSphere Storage vMotion, Thin Provisioning, VSphere

Well fresh from my return from the Citrix iForum I decided to fire head long into installing XenServer in my home lab so I could have a look at it.

I already run VMware vSphere 4i on my home lab which consists of an HP Proliant ML115 G5.  Instead of buying another machine to install Xenserver on or rebuilding my current vSphere server I thought I would try and install XenServer inside a virtual machine.  As Eric Gray over at vCritical proved you can install vSphere 4 inside a vSphere 4 virtual machine so surely the same would be possible XenServer 5.5, shouldn’t it?

Well the screenshot below should prove exactly that,  Xenserver 5.5 successfully running on vSphere 4i

So how did I conduct the install,  well first of all I downloaded the ISO from Citrix’s website and then did the following.

1 - Create a Virtual Machine with custom settings.
2 - Select the new Virtual Machine version 7 hardware.
3 - Select Red Hat Enterprise Linux v5 (64-bit).
4 - 1 vCPU and 1GB of RAM will suffice.
5 - I used the LSI Logic Parallel SCSI Controller.
6 - Create a disk based on your requirements.
7 - Make it thin provisioned if you want,  why wouldn’t you?
8 - Connect the ISO image to the VM and start it.
9 - Follow the prompts on screen to complete the install. 

I only had one issue during install and that was when the following message appeared,  I carried on installing XenServer and it completed without issue.

 However when It came to starting up windows based Virtual Machines,  like the message above indicated, I couldn’t.  XenCenter showed the following error.

Basically because Windows requires the hardware virtualised assist features (Intel VT or AMD-V),  hypervisor on top of hypervisor masks this underlying virtualisation assistance and hence Windows can’t operate.  What I did manage to get up and running was virtual machines running Debian Lenny 5.0,  so at least I had something to play about with and test out XenServer features such as live motion. Linux machines on XenServer start up in a para virtualised mode and are therefore supported where hardware virtualisation assist is not available.

check out the Debian Lenny based DreamLinux desktop edition,  this should give you some VM’s to play within your virtualised XenServer environment.

So although I didn’t get XenServer operating like I wanted to in Vmware vSphere, I did get  it working enough to play about with it and it’s features.  To be honest that’s all I was after in the first place!!

Citrix, VMware, XenServer, vSphere Citrix, VMware, VSphere, XenServer

I was lucky enough to attend the Citrix iForum in Edinburgh this week, which was a first for me and a credit crunch alternative to VMWorld. I must admit I was looking forward to it as I’ve been meaning to take a closer look at Citrix XenServer for some time now.  I wasn’t disappointed as I got to spend some time on the demo stand and saw a few technical  and case studies presentation.  I have to admit I was impressed,  I was sitting here thinking Hyper-V was the only other hypervisor worth looking at,  how wrong was I.

So to quickly set the scene, Citrix XenServer 5.5 is completely free, and for that you get the following feature set .

To get hold of the value add features,  known as Citrix Essentials your going to have to spend some money.  I got a quote today for the Citrix Essentials offerings, at the moment it’s £1,360 for Enterprise and £1,680 for Platinum. For this price you get the following, which on first viewing is pretty good,  especially when you consider the Lab and Stage management features which are separate products within the VMware product range.

 Citrix Essentials / VMware price offer

As indicated by the title of this blog post Citrix were advertising a special offer around the purchase of Citrix Essentials 5.5 starting in Q3 this year.  If you are willing to share with Citrix your VMware SNS renewal quotes they will give you a price for Citrix Essentials that matches or betters the price of your VMware renewal.  Technically resulting in you paying out no more than you were going to be spending from your budget anyway. now you’ve just got to find the resource to rip an replace your existing virtual infrastructure (joking!).  I’d suggest following up with your Citrix sales representative for more information as I can’t find anything official on the Citrix web site,  it was however mentioned on 4 different occasions during iforum so it must be true.

In all seriousness though,  I spoke to people at the iforum who were existing VMware customers like me.  One guy had a project to add three additional ESX hosts,  he reckoned he could do 3 XenServer hosts with the same functionality for the cost of one ESX host.  He was off to download it and set it up in their company lab the next day,  I have to say I can see why, especially with company budgets constrained the way they are.

I’m going to get my home lab setup with XenServer so I can do a few more blog posts on some of the interesting features such as Storage Link and the distributed management database.  I’m genuinely intrigued now.

If you want to read a little bit more about the usage of XenServer in the real world,  you might want to read about Tesco’s use of XenServer to virtualise 1,500 servers.  Alternatively you might want to read the following performance review of hyper-v, ESX 3.5 and XenServer to understand a little more about how they compare (vSphere obviously tilts the performance balance back in VMware’s favour).

General Citrix, Citrix Essentials, XenServer

As soon as vSphere was put on General release I downloaded a copy of vSphere 4 ESXi for running on my home lab setup.  I’ve only recently started my home lab and the first machine I purchased was the HP Proliant ML115 G5 server. This was following a recommendation by Kiwi Si over at www.techhead.co.uk  who has extensively blogged about the suitability of the HP ML110 G5 and ML115 G5 for ESX labs.

If your interested in starting a home lab I can thoroughly recommend these HP server.  Simon even has a deal going with ServersPlus.com (UK) for free delivery on either the HP ML110 or ML115 server.  Get over to his hot deals page for further details.

Simon also has some great articles on getting ESXi 3.5 running from a USB pen drive.  This was a perfect place for me to start as I wanted to take advantage of the ML115 G5’s internal USB port and boot my server from a USB pen drive. So where do you start?

What do I need?

A USB Pen Drive that is over 1GB in capacity - nice and cheap at amazon
Download the vSphere ESX4i ISO image from here 
Download Shareware version of Winrar from here
Download Free Trial version of Winimage from here

How do I do it?

Once you have downloaded the ISO image open it up with WinRAR,  make sure you use WinRAR as I had problems with WinZip and UltraISO

Double click on the image.tgz file to open the contents in WinRAR and drill down to the \usr\lib\vmware\installer directory.

Within this folder you will find a file called VMware-VMvisor-big-164009-x86_64.dd.bz2. This is another zip file so double click on it and the contents will be displayed in a seperate WinRAR window.

Once inside extract the file VMware-VMvisor-big-164009-x86_64.dd using WinRAR and copy it somewhere locally on your PC.

Now install and open up the WinImage trial that you downloaded at the start of this process.

Insert your USB key and then select Disk and restore virtual hard disk image on physical drive as per the screenshot below.

 Select the physical USB drive from the list and click OK,  when prompted for the virtual disk file navigate to the dd file you extracted to your local PC.  This will now image your USB Key with the vSphere ESX4i hypervisor.

Once complete stick the USB key in a server / whitebox that supports 64 bit computing and away you go.  The screenshot below shows my own HP Proliant ML115 G5 running vSphere ESX 4i and all this from a simple 2GB USB pen drive.

ESX, VMware, vSphere esx4i, esxi, VSphere

I’m currently working on replacing all our ESX 3.0.2 and 3.0.1 hosts with 3.5 U4 hosts,  unfortunately the leap to vSphere is going to have to wait until next year.

One of my main reasons for doing this was to allow for the introduction of VMware Update Manager for ESX Host patching.  ESX Hosting patching can be a bit of a pain when using the service console and Esxupdate and because of this patching starts to become a nightmare and gets forgotten about.

So I had my first ESX 3.5 host added to virtual center and was poking around the VMware Update Manager tab and noticed that there were absolutely no host patches attached to the default host baselines.  I did a bit of digging around online and plenty of people had problems out there with the actual installation of VUM or their vci-integrity.xml file was missing some of it’s content.

Luckily my issue was resolved far more easily.  My virtual center server didn’t appear to be able to access the websites to get the patches due to the enhanced security pack.  All I had to do was add the following websites into the trusted sites on Internet Explorer on the Virtual Center server.

https://www.vmware.com
https://xml.shavlik.com

When the next scheduled patch download occured I recieved 150 patches for ESX 3.5.0 and was then able to create a custom baseline for all patches after 3.5.0 U4 and apply them.  Worked an absolute treat.

ESX, ESX 3.5i, VMware VMware Update Manager, VUM

Today is the day, VMware is now available for download and some really good news is that VMware are offering a 60 day evaluation trial of vSphere Enterprise Plus and vCenter Server Standard.

Three links for you, the first is the VMware vSPhere Download page and the second is the free 60 day evaluation download link.  For those who want to use the free version of ESX 4i and have a computer / server capable of supporting 64 bit you can get your free copy at this link.

Documentation for the various vSphere components can be found here http://www.vmware.com/support/pubs/vs_pubs.html

Credit to Michael Hany over at www.hypervizor.com for the links to these awesome video’s below, which should help you get to grips with some of the new vSphere 4 features.

ESX Installation and Configuration
ESXi Installation and Configuration
VMware vCenter Server
VMware vSphere Client
Networking configuration
Storage configuration (iSCSI)
Create and manage virtual machines
VMware Host Profiles
VMware Storage VMotion
VMware vCenter Server Linked Mode
VMware vNetwork Distributed Switch (vDS)

ESX, New Products, VMware, vSphere esx, esxi, VSphere

So I had the chance to play about with the latest version of Bluebear Kodiak the other day,  I just hadn’t had the chance to put some of my findings and thoughts down on the blog.

This release has been a long time in coming,  so much so that I’d almost forgotten what was in 0.0.2.  I often found myself looking at 0.0.3 and asking myself, “was that in it already or is that new?”

Most of the changes that have occured appear to be under the hood, one of these changes is the introduction of a Lua scripting engine.  I had never heard of Lua as a scripting language before this release,  however following a bit of reading it appears to be a very lightweight and fast scripting language that is highly customisable to requirements.  To be honest I have no idea what this will mean for Kodiak from an operational perspective,  it doesn’t appear to be any faster than the previous version but I was only testing this against a single ESX server.  The release notes hint that each server gets it’s own independent Lua script which may indicate that the benefits only appear when using Kodiak against multiple hosts.

I notice that support for Citrix XenServer and Microsoft Hyper-V are still missing in this release,  with no real indication as yet of a timeline for support of these mainstream hypervisors.  I’m very interested to see if and how it would handle managing multiple hypervisors within the same application window,  I’m presuming that it’s what they’re working towards and it would be a masterstroke if they can pull it off.  Microsoft have obviously tried something similar by managing VMware hosts within System Center Virtual Machine Manager (SCVMM) with mixed results,  lets hope the guys at BlueBear have more joy in their attempts of building a management tool to bridge all mainstream hypervisors.

So,  lets get down to the nice bits.  First off the map, a great interface and easy to use, move about and navigate.  This release sees the addition of a new map connection node and link display control which allows you to customise what’s shown on the map to good effect.  It can get very busy on screen sometimes and this feature allows you to control exactly what you do and don’t display in a very granular way.  I believe the scripts window is a new edition,  when a machine is selected you are presented with an array of scripts to use including defragment vm disks and migrate reboot to BIOS. From the release notes it appears that users will be able to create their own scripts however a user interface / script builder still needs to be added

I must admit I did have the occasional bit of trouble when navigating round the interface.  I seemed to be able to open the console window without issue by clicking on the appropriate button when a host was selected.  However I was having issues opening the configuration screen in that sometimes it just did not respond at all.  I also saw issues with VM status refresh as well,  I powered down a linux box by issuing a shutdown within the OS. However Kodiak still reported it as switched on within the map section,  a minor annoyance of course but a core feature has to be the accurate reporting of virtual machine status.

It looks like the guys at Bluebear have been working hard on changing some of the core fundamentals in the background,  as such there aren’t maybe as many differences on top as you might expect.  What I’m hoping is that the work done between 0.0.2 and 0.0.3 underpins their master plan and that we’ll start to see development pick up. We hope to see the additional hypervisor support, the continued fine tuning of the interface and of course delivery of features to meet the requirements people will undoubtedly have in their virtualisation management tool of choice.

I don’t have any invites left but Gerald Bunch over at Professionalvmware.com has quite a few left

ESX, ESX 3.5i, New Products bluebear, ESX management

An ex-colleague of mine has been trying to get a Scotland VMUG together for a little while now,  unfortunately without much luck.  So I was very happy to hear John Troyer from the VMware community talk about an online VMUG on a recent community podcast.  Perfect for those of us struggling to get to the only other VMUG I’m aware of in the UK, which is in London.

Alan Renouf, otherwise known as Virtual-Al has posted details of this first online VMUG on his website,  an extract can be seen below

Based on your feedback I would like to announce the first Unofficial Online VMware User Group.

The Agenda at the moment is as follows:

• Introduction
• Presentation and Demo of The Virtualization EcoShell Initiative by Scott Herold
• Virtual Coffee break (bring your own coffee)
• Overview and Demonstration of vSphere new features by a VMware Employee
• Finish and details of next Unofficial Online VMUG

 

This meeting is being hosted over Microsoft Live meeting, if you have not installed the client it is recommended you do this 15 mins before the meeting. 

Video:  Video will be delivered via the Microsoft Live Meeting client.
Audio:  For attendees, audio will be delivered via the Internet using your computer’s speakers. There will not be a call-in number for this meeting.

The meeting will also be recorded for people who are unable to attend.

For those interested the meeting details are as follows

Meeting Details:

Date: 1st June 2009

Time: 08:00 PM GMT / 03:00 PM EDT

Meeting URL:  https://www.livemeeting.com/cc/winserver_usergroup/join?id=QP5K5Q&role=attend&pw=8%5E-%5CzqX

Meeting ID:  QP5K5Q

Attendee Entry Code:  8^-\zqX

Registration Link:
http://www.clicktoattend.com/?id=138358

VMware VMUG, VMware User Group