The storage industry is undergoing a huge change at the moment, a huge change that is moving at break neck speed and one that I find both exciting and scary all at the same time. Exciting because it’s new technology being used to solve complex business and IT problems and scary because it’s moving so incredibly fast. I think you can only guess what’s going to happen next, which direction it’ll go in and which trends will win out at the end of the day. I guess that’s probably why I’m writing a blog post for the first time in almost 2 years! Yes I know, that is far too long between blog posts and yes I am extremely ashamed of myself!
I’m lucky enough to work for EMC and as part of that I get to see a lot of this change in IT developing momentum from the inside. That said, in an organisation as big as this I also sometimes don’t find out about some of it until it hits the headlines, either at one of our product mega launches, a big industry event or in one of Chad’s now infamous blog posts!
Today’s Modern Storage Architectures
New storage architectures are appearing thick and fast, my boss (Chad Sakac) explains this exceptionally well in his recent VMworld 2013 blog post on all things SDS at VMware and EMC. He calls out 3 separate storage architectures that are fast becoming the norm, they can be seen below and can be explained from left to right simply as.
Type 1 = Traditional storage model.
Type 2 = Scale out / node based build out storage architectures.
Type 3 = Next Gen scale our software defined storage with COTS hardware.
EMC is very well known for the Type 1 storage, VNX is a great example of this model in action. EMC is also heavily into the Type 2 storage, whether that be scale out NAS in Isilon, scale out block storage using the scale out engines of our enterprise VMAX arrays or even scale out block storage from an All Flash Array such as XtremIO.
It is however the type 3 storage I have been viewing with most interest and it is impossible to ignore the rapid increase in the use of the term “Software Defined Storage”. I should really correct myself here though, it’s not being used simply as a term or IT buzzword, there are actually some very tangible products coming to market and this area of storage will gather huge momentum as we enter 2014.
A subject that plays in this field and comes up regularly with one of my larger enterprise customers is Cloud Storage and Open Source Software with Commodity Off the Shelf hardware (COTS). It’s an interesting discussion and one that has been born out of both advances in Cloud storage platforms and open source software stacks (OpenStack, Ceph) and a desire to introduce cost effective, supported “good enough” solutions to support business demand.
This stuff is real, some people (including EMC colleagues) dismiss it as a passing fad, something that will never take off as it’s not supported, won’t work, companies can’t put data in the cloud / on that platform for compliance or security reasons. It’s seriously time to wake up and smell the coffee, there may be challenges be they business or technical ones, but the desire is there and where there is a will, there is always a way!
So where is this blog post going exactly…
"Good Enough" In-Memory storage alternative with software defined storage
…So all of this leads me to an interesting use case that I was discussing both internally and with a customer recently, one that covers off three very interesting areas.
- New Memory Channel Interface Flash (think NAND Flash with a DDR3 Memory interface)
- Software Defined Storage pooling of that MCI based storage using EMC ScaleIO
- A “good enough” performance tier, less than in-memory but more than PCI-E flash cards.
It sits firmly in the type 3 storage architecture discussed earlier, it involves new hardware technology combined with Software Defined Storage and could be used to deliver a cost effective, supported, good enough storage offering to deliver an alternative to in-memory databases. Sound interesting? let’s dig a bit deeper into the 3 areas.
1. Memory Channel Interface (MCI) Storage
MCI storage popped up on my radar about a month ago following some internal distribution list emails. A little digging shows that this particular technology appears to be in the very early stages of adoption. In fact some of the tech companies in this field are still hiring ASIC programmers, so it’s definitely early days.
One of the main companies out there at the moment is Diablo Technologies who are In essence providing non-volatile flash attached to a DDR3 memory interface to deliver up to 400GB of storage with a response time of less than 5μs. The following is an extract from the Diablo press release which says it all really.
Configuring Memory Channel Storage (MCS) as a traditional block storage device enables new performance levels for applications, all the while reducing latencies by more than 85% over PCI-Express based SSDs and 96% over SATA/SAS based SSDs. MCS latencies are not only significantly lower than any other flash based storage, but also provides deterministic latencies with minimal variability, allowing applications to commit data quickly and consistently.
As well as being used for storage they the MCS DIMMs can also be used as an expansion of system memory, allowing TB of addressable system memory instead of GB. That’s another use case for another day though, right now I’m interested in this as super low latency block storage and it sounds like this could be the next big thing in server based flash solutions.
2. Software Defined Storage Pooling with EMC Scale I/O
At present the problem I see with the above technology is the fact that because it’s basically direct attached storage (DAS) there is no protection of the data. So you’re looking at stateless deployments or utilising an overlay to provide some host level clustering of the DAS resources. This is where ScaleIO comes in…
ScaleIO in it’s simplest terms is a software defined storage solution that allows customers to leverage the local disk in applications servers in a pooled manner. Through the installation of the ScaleIO agent known as the ScaleIO Data Sever (SDS), local storage can be combined together to create large scale elastic pools of storage. Some of the key benefits are listed below
ScaleIO technically converges the storage and application layers and can be used in a number of different configurations. Hosts are of course interconnected, this can be achieved using Ethernet or Infiniband (In my example I’m thinking high performance back end here). As ScaleIO deployment is small the SDS (100Mb – 400Mb) and the SDC (20Mb) can be deployed side by side and all the assets in a server can be sweated to deliver a complete solution.
You can however configure things in a traditional 2-tier setup as shown below, where applications utilising the ScaleIO Data Client (SDC) access storage from dedicated storage nodes using the Scale IO Data Server (SDS) agent.
In reality, you can configure your application and ScaleIO in any way you like, depends on your requirements. Add into this mix the flexibility of agnostic support for physical and virtual nodes, full protection zones, quality of service, 2 and 3 way data protection and the upcoming storage pools for performance. It’s quite a neat little product.
So let’s put this back into the context of the use case I was originally discussing. By taking a number of hosts containing the MCI low latency block storage, connecting those hosts with Infiniband and adding ScaleIO over the top I’ve got myself a pretty interesting low latency, highly parallelised IO storage system using commodity off the shelf hardware. What’s not to like?
3. Cost Effective / Good Enough in-memory alternative using MCI and Scale I/O
So the 3rd and final point, the reason that I was discussing this topic with a customer in the first place was cost efficiency. They were interested in how you could do provide a good enough in-memory alternative, both in terms of software and hardware. The following is an interesting extract from a blog, in this instance they’re looking at using the MCI as a memory extension and layering on software to utilise that memory. In my case I’m looking at it as a storage extension. Regardless of the use case though, the point is the potential for cost saving and there is one if you put some thought into it vs the traditional models.
Another important characteristic of MCI storage is the plug-n-play fashion in which it can be used – no custom hardware, no custom software required. Imagine, for example, an array of 100 micro-servers (ARM-based servers in micro form factor), each with 256GB of MCI-based system memory, drawing less than 10 watts of power, costing less than $1000 each.
You now have a cluster with 25TB in-memory storage, 200 cores of processing power, running standard Linux, drawing around 1000 watts for about the same cost as a fully loaded Tesla Model S.
This has been a bit of a ramble but I think what I am trying to say is, the world is changing and there are now so many different ways to achieve your objectives. The commoditisation of hardware and the major up shift in software defined everything is fundamentally changing the approach in IT. As I said at the start, it’s both scary and exciting all at the same time, the only option is to embrace it if you ask me!