VirtualPro

I stumbled across a tweet the other day by Carter Shanklin who I hope most of you know. For those that don’t, Carter Shanklin  is a product manager over at VMware specialising in the VMware PowerCLI and other automation tools.

His tweet was about a new project called Project Onyx, which on initial inspection is a tool that allows you to see and capture the powershell code behind actions in vSphere vCenter.  This project is at a very early stage as you will probably see in the video embedded below, however this is a very exciting development for those new to PowerCLI.

Some time back I sat my Hyper-V MCTS exam and one of the things I really liked about Microsoft’s SCVMM product was the ability to see the powershell behind the actions you were carrying out.  It was like a head start on powershell automation, giving you a chance to see what was happening, allowing you to dissect, copy it and re-use it however you wanted.

I’m hoping that VMware are planning something similar here,  perhaps a plug-in for vCenter! I personally struggle sometimes with the PowerCLI and the syntax, etc.  If I could see the code behind an action I was trying to automate a good part of the work would be done already.

This project is currently in the early stages of development.  In order to get this project up and running VMware are looking for people to help them Beta test project Onyx.  They have put the call out for a handful of “dedicated people with a burning need for automation”

Get yourself over to VMworld session VM2241 with your business card and put yourself forward to help this promising looking project gain some traction.

VI Toolkit / Powershell, VMware VI Toolkit / Powershell, VMware

I’m currently revising for an EMC exam and part of the course material involves doing some training on the Navisphere simulator. No problems, except where on earth do you download the simulator software? I searched for ages before finding out that you only get access to it on EMC PowerLink if you are an EMC employee, partner or you have bought the relevant training.  I had done the latter but still couldn’t get it downloaded for some reason,  that’s when a google search turned up a link to the following page which required no credentials, EMC Education Services product simulators.

I managed to download the latest version for EMC Flare release 28 without issue.  I was however asked for my EMC PowerLink credentials when I first opened the simulator so perhaps that’s how it’s authenticated against the training purchased. It’s a really good learning aid and I’ve been able to find out how to do things on it that I wouldn’t have dared to attempt on our actual CLARiiON implementation. Hope this link helps someone who’s searching for this,  took me ages to find it.

Certifications, Storage Certification, EMC, Navisphere

I’ve been meaning to write about the Citrix Branch Repeater product for some time now, so a timely reminder to actually do this was the release of Citrix Branch Repeater V5.5. Earlier this year I attended a branch office infrastructure event run by Microsoft and Citrix in Edinburgh.  This was the first time I had heard about this product, I luckily had the chance to follow up my interest at the recent Citrix iForum in Edinburgh.

Branch Repeater is the rebranding of the old WANScaler product, which, in its simplest form was a WAN acceleration product. The new branch repeater is still a WAN accelerator at heart;  however Citrix have added some clever branch office features as well as some new features for XenApp customers. From a topology perspective, you basically place a larger repeater appliance in your data centre and additional smaller repeater appliances in your branch office.  I was actually surprised to learn that this is not the only option available; there is also a repeater software plug-in for use by remote users.  The diagram below shows the basic topology overview.

Citrix, Gestalt-IT Branch Repeater, Citrix

I was approached some months ago by Stephen Foskett about joining a new site that he was creating called GestalIT.  At the time I was in the middle of a major work project and did not have the time to respond,  however this week I’ve been back in contact with Stephen and I’m now an author on GestaltIT. Over time I will be contributing what I feel are some of my better articles to the GestaltIT site as well as publishing articles on VirtualPro.

If you haven’t been to GestalIT I would highly recommend heading over and checking it out,  especially if storage, cloud computing or virtualisation is your thing.

General

During the recent troubles in Iran a large number of people turned their Twitter avatars green to show their support.  Today I recieved an email about a great new site called called Twibbon which allows you to support your cause by adding an overlay to your Twitter avatar. As you can see from my own Twitter profile pic my particular cause in this case is VMware, so to  join up and support VMware go to  http://twibbon.com/join/VMware

The story behind this site is brilliant,  as I heard it a developer based in Edinburgh, Scotland came up with the idea at some ungodly hour last Wednesday had it live by Thursday and they had over 100,000 hits in the first 30 hours alone.  Great idea,  developed quickly and has great potential for the likes of charity support or product support.

General, VMware Twitter

For those of you that will have heard of Alan Renouf you will undoubtedly know of his talents in the dark art of VMware CLI / Powershell.  For those of you who don’t know him I suggest you check out his web site  to sample some of the many great articles and scripts he’s already produced.

His latest powershell creation has recieved a lot of attention in the last couple of days and with good reason. The Daily Report is a configurable script where you can set thresholds and variables such as snapshot age, datastore space free thresholds or number of days to look at for vCenter warnings and errors.  The script when run goes off and examines your Virtual  Infrastructure based on these variables and then proceeds to email you a nice html report on the following items.

·         VMs created in the  x number of days and who created them.

·         VMs deleted in the  x number of days and who deleted them.

·         Datastores which have less than x% of free space remaining.

·         VMs that have CD-Rom or Floppy drives connected.

·         VMs with no VMware Tools installed.

·         Snapshots that are older than x number of days.

·         Current state of vCenter Services.

·         vCenter events that have been logged in x number of days.

·         Windows events  on the vCenter server that relate to VMware.

·         Hosts in maintenance mode or a disconnected state.

Get yourself over  to Alan’s site and download a copy of the script and give it a try,  I did today and the results were enough for me to go ahead and implement this as a scheduled task.  If you’d like to see more features in Alan’s Daily Report script then give him some feedback,  there are a few good suggestions on the blog post already and I’m sure the next version isn’t far away.  Great work Alan, keep it up!

ESX, VI Toolkit / Powershell, VMware, vCenter esx, vCenter, VI Toolkit / Powershell

I was purusing twitter the other day (as you do) when I came across a link shared by @Stevie_Chambers of VMware.  The link was for a new free tool from EMC called Compliance Checker for VMware ESX, which as you might guess does exactly what  it’s title suggests.  The tool though EMC branded is actually by Configuresoft who EMC bought outright earlier this year following an existing OEM relationship.

Now some of you may be familiar with the Tripwire Config Check tool which allows you to scan your ESX servers security configuration, which in itself is a very handy tool.  This new tool from EMC appears to be a little more comprehensive in that it checks your ESX servers against both the VMware VI3 Security Hardening Guidelines  and the Centre for Internet Security VMware ESX Server 3.x benchmark.  Test results against both are backed with an extensive knowledge base of what the check is and the actions required to rectify the issue.

Included below are some screenshots of the Main interface, the reports returned and the knowledgebase articles you recieve when you click on any item in the report.

I must admit that these compliance checkers are handy because my Linux experience only began with ESX and sometimes the areas of security being checked are ones I wouldn’t have a clue about.  This is where the knowledgebase is great because it explains a little about what the check actually is as well as the actions required to rectify the issue.  Very handy because it would appear that a standard build of ESX 3.5 U4 is only 73% compliant as far as this piece of software is concerned.

To get your free copy of Compliance Checker for VMware ESX, click the link,  you’ll need to register but it’s a small price to pay for this handy tool.

ESX, New Products, VMware EMC, esx

Virtual Appliances have been around for quite some time now and it seems with the recent release of vSphere, VMware are looking to take vApps to the next level. The VMware Virtual Appliance Marketplace (VAM) is the one stop shop providing VMware users with a plethora of pre-configured, ready to download appliances.  There are over a 1000 appliances available in the VAM, all of them capable of being deployed in either the private cloud or in a cloud hosted by a 3rd party.  Now the VAM is also offering on-demand trails of certain vApps through selected vCloud partners.  The vCloud vision is slowly taking shape and and although it may be mostly conceptual now, small steps like this are the real building blocks.

With the release of vSphere 4.0 VMware introduced full support for the OVF 1.0 specification.  What does this mean?  well the Open Virtualisation Format is a platform independent, efficient, extensible, and open packaging and distribution format for virtual machines.  It’s virtual platform independent so if vSphere isn’t your platform of choice the vApp will work with Hyper-V and XenServer. OVF virtual machines are optimised for easy distribution are simple to deploy and support single and multi VM configurations.

Now I spend a lot of my time integrating and supporting applications from 3rd party vendors,  some of them mainstream and some of them more specialist. One of the constant issues I have is around how best to deploy them,  what are the best practices, minimum specifications and correct configurations to ensure successful deployment and reliable operation.  This is something I can see vApps helping all of us with, vApps created using the OVF format will allow vendors to build and maintain pre-packaged systems. Pre-configured to be highly compatible, built to the vendors own best practices and best of all built for rapid deployment to customers regardless of virtualised platform or cloud prefernce. The way I look at it,  it’s not to disimilar to Apple’s approach to controlling the hardware and the software they use. The way Apple operate allows them to guarantee better reliability and compatibility because they know what they’re deploying and what they’re deploying it on. A vendor built vApp has the potential to offer the same benefits and simplify the job of vendors, system integrators and application support teams alike.

So how are VMware assisting vendors in their pursuit of vApp bliss? With the creation of VMware Studio of course,  It was a product I’d never heard of until the beta release of VMware Studio 2.0 was announced earlier this month. The latest features are listed below.

Build vApps and virtual appliances (with in-guest OS and application components) compatible with VMware Infrastructure, VMware vSphere 4.0 and the cloud

- Support for OVF 1.0 and 0.9
- Available as an Eclipse plugin in addition to the standalone version
- Ability to accept existing, Studio-created VM builds as input
- Support for 32 bit and 64 bit versions of Windows 2003 and 2008 Server
- Support  for SLES 10.2, RHEL 5.2 and 5.3, CentOS 5.2 and 5.3 and Ubuntu 8.0.4.1
- Publish patches to update deployed virtual appliances
- Extensible in-guest management framework
- Automatic Dependency resolution
- VMware ESX, ESXi, Vmware Server 1.0.4 - 2.0, VMware Workstation enabled as         provisioning engines.
- Infrastructure enhancements in the GUI and builds

So what does a vApp deployment look like,  well here’s a video that was posted on the VMware vApp developer blog.  In this demo a user deploys a mulit VM, multi-tiered version of SugarCRM in just a few clicks with no need to even start up a VM Console. 

New Products, VMware vApps, vCloud, VMware

I recently had a number of vSphere ESX4i  USB Key installs following my article on putting vSphere ESX4i on a USB key / Pen Drive. I needed to format a couple for general windows usage, only to find that the ESX4i image creates a number of partitions on the USB Key. Unfortunately Windows does not appear to support the removal of partitions on removable devices so when I was trying to format a 2GB USB stick I was able to format a 110MB partition and that was it. I was a bit stuck on the best way to rectify the issue and wasn’t finding much to help out on the web.

That’s when I stumbled upon the HP USB Storage Format Tool,  a great little tool that works with a wide range of USB sticks and not just HP ones.  It allowed me to wipe the USB key as a single entity and didn’t care about the partitioning, returning my USB Key to a useable state within windows.

You can download it from HP’s website by clicking on this link,  sometimes you just don’t know if you can trust other download sites.

General ESX 3.5i, esx4i, USB, VSphere

I’ve recently been evaluating some of the new features in VMware vSphere to see what use they would be to my current employer. One of the areas that I touched upon in my “what’s new in vSphere Storage”  blog post was thin provisioning.  I wanted to come back and cover this particular topic in more detail as it’s a key feature and it’s available throughout all versions of vSphere so I’m sure everyone will be interested in it.

What is Thin Provisioning?

Thin Provisioning in it’s simplest form is only using the disk space you need.  Traditionally with virtual machines if you create a 500GB virtual disk it will use 500GB of your VMFS datastore. With Thin Provisioning you can create a 500GB virtual disk, but if only 100GB is in use only 100GB of your VMFS datastore will be utilised. Credit to Chad Sakac for the diagrams below.

How does it work in vSphere?

Thin Provisioning is being heralded as something new with vSphere,  when in truth it was already available in VI3.  In VI3 creating a thin provisioned disk involved using vmkfstools and was also not a production supported VM configuration.  Now in vSphere the creation of thin provisioned disks can be carried out from the VI Client (see below) and is a supported production configuration for a VM.

It’s as simple as checking a check box, the results are pretty good to.  Below you can see I have created two thin provisioned VM’s on my new ESX4i host and you can see the provisioned space and the used space stats being shown in the VI Client. 

What are the benefits?

The thin provisioning  feature is perfect for my home lab environment where disk space is at a premium, but how does it translate into real world implementations of ESX.  Well I for one have been looking at exactly this to identify what benefits could be achieved within my employers ESX estate. A quick audit found that our development and system test ESX environment was running at 48% disk utilisation,  so straight away thin provisioning would save us 52% on storage capacity used. Paul Manning of VMware mentioned on a recent communities podcast that on average vSphere would save users 50% on storage.   This is possibly not such a big thing when your talking about test environments, but when you move up to production SAN Storage, saving 50% on an expensive SAN array is a very real and tangabile cost saving.  One that people should definately take into account when making a cost benefit case for buying or upgrading to vSphere.

What are the potential downsides?

One of my personal concerns with thin provisioning is the potential overhead on any write activity that would requires the extension of the VMDK file.  To me there is an obvious VMFS operation that needs to take place there which would add to the overall time to complete the disk write.  When there is a requirement to expand a disk, the VMDK files will increase in increments based on the block size of the underlying VMFS partition, 1MB, 2MB, 4MB or 8MB.  So the overhead may be smaller if your VMFS has been formatted with a bigger block size, i.e. for a 16MB write it only has to expand 2 blocks when the VMFS block size is 8MB but would have to expand 16 blocks if it was formatted with the 1MB block size.  I can imagine this percieved overhead could put people off using thin provisioned disks for certain production based environments, especially those where there is a lot of write I/O activity,  SQL Server or Exchange for example.  To counteract that though,  the improvements in the VMware I/O Stack should compensate for this performance overhead.  This could potentially leave you in a situation where you’ve reduced a VM’s storage footprint and still have performance equal to that experienced in VI3,  possibly not a bad trade off.  I’d also expect people running their VMware environment on enterprise SAN technologies from the likes of EMC or NetApp to notice minimal performance impact with thin provisioning as SAN memory caches help take up the strain.

Another downside is if you want to use VMware Fault Tolerance to protect a VM then you cannot use thin provisioned disks.  To be honest this is a small issue as Fault Tolerance protection is most likely going to be on virtual machines that are important to your organisation.  These machines are probably the ones you wouldn’t thin provision in the first place for performance reasons.

Thin provisioning creates it’s own unique problem in that what we’re basically doing here is over provisioning the storage.  You need to keep a very close eye on thin provisioning as it’s quite feasabile that your VMFS datastore could fill up and your virtual machines fall over.  Not what you want to come into on a Monday morning,  or any morning for that matter.  So you need to monitor your storage and ensure that there is enough free space.  One of the simplest ways to do this is through the use of the new alarms in vSphere that allow you to alert on datastore usage and datastore over provisioning.  These should keep you from filling a datastore and killing your VM’s or ESX Servers

One gotcha that you should watch out for is VM swap files, as these are usually stored with your virtual machines vmdk files in the VMFS datastore.  In VI3 the swap file was not deleted when a VM was powered down,  in vSphere the swap file is deleted on power down and recreated when the VM is powered up.  You should be aware of this when over provisioning storage as you could get into a situation whereby you find you can’t power on a VM because there isn’t enough space for the swap file to be created.  This becomes more likely as servers and VM configuration maximum’s increase,  if you have a VM with 20GB of RAM it’s going to need 20GB of disk space for the swap file.  if you have 256GB of RAM in your vSphere host and you allocate it all out to VM’s then you need to think about the 256GB of disk capacity required to service virtual machine swap files.

Storage vMotion

If you’ve already got a VI3 environment then the chances are that your VM’s aren’t thin provisioned,  how on earth are you going to take advantage of this new feature? Well if you have purchased a vSphere edition that supports storage vMotion then you can of course migrate the underlying storage and have it thin provisioned during the move.  This should allow existing VI3 customers to claim back a lot of space,  as I mentioned before I found that our development and test VI environments were only 48% utilised.  If I storage vmotion all those VM’s and thin provision at the same time I will free up about 1.5TB of storage that wasn’t being used in the first place.

I’ve included a video below which demonstrates the Storage vMotion and thin provisoning features in vSphere quite nicely, enjoy!

Gestalt-IT, Storage, VMware, vSphere Storage vMotion, Thin Provisioning, VSphere