Hedvig software-defined storage for large-scale VMware environments

By Rob Whiteley | | Software-defined Storage

“Anything is easy to demo . . . at small scale.”

A customer recently said that to me. This Director of Infrastructure was specifically referring to some challenges he had in selecting a storage solution for his growing VMware environment.

Here’s why.

This is a large enterprise with thousands of VMs across multiple data centers in multiple countries. It’s a large, complex environment. As a way to cut costs they had decided to go down the path of software-defined storage to underpin their VMware vSphere infrastructure.

They evaluated all the usual suspects and, given all the choices on the market, walked away with several valid options. Until they deployed one in production.

That’s when their evaluation, which was extremely rigorous, exposed that their demo environment did not appropriately test scale. After all, anything is easy to demo at small scale. It was only when their original SDS solution was implemented that they learned:

  • Performance did not scale as expected across heterogeneous nodes.

  • Data protection mechanisms (e.g. replication) broke down.

  • Data services (e.g. deduplication) could not be easily configured on a per-VM basis.

That’s where Hedvig comes in. We solved their VMware storage needs at scale. Now they have a single storage cluster that stores and protects their VMs across multiple datacenters.

What is Hedvig’s architecture for VMware?

Hedvig can deploy in both a hyperconverged and hyperscale architecture (or a mix of both). However, when it comes to large-scale environments, hyperscale makes more sense.

A hyperscale deployment means there are two, decoupled parts to the architecture:

  • Hedvig Storage Service on dedicated commodity servers or in a public cloud. The Hedvig Storage Service is deployed on bare metal commodity servers. This where the heavy lifting occurs. The Hedvig Storage Service provides all of the advanced data services for your VMware environment, like in-line deduplication, in-line compression, tiering, caching, snapshots, and cloning. It also provides all the clustering, replication, balancing, and self-healing intelligence needed for scale and resiliency.

  • The Hedvig Storage Proxy deployed as a VM, one per physical vSphere host. This a lightweight, stateless process that provides iSCSI and NFS access to the hypervisor or to the individual OSes inside each VM. The Storage Proxy also provides server-side caching that leverages local PCIe or SSD devices residing at the compute tier. This approach of deploying one Hedvig Storage Proxy per host (note: logically it’s one, but it’s actually a high-availability pair) ensures linear scaling of this caching tier.

Besides the obvious advantage of being able to scale compute and storage independently, this hyperscale architecture enables large organizations to dial in the economics that make the most sense. The hardware profile needed to power storage (the “green” servers above) requires a different core, RAM, SSD, and HDD count than the profile needed for VMs (the “blue” serviers above). Separating the two means you don’t pay a premium for the greatest common denominator. For example, you don’t have to pay a heavy CPU and RAM tax if what you really want is high-capacity 8-TB drives.

Why choose Hedvig for VMware?

The Hedvig Distributed Storage Platform has several capabilities that are unique for large-scale VMware environments:

  • Software-based block, file & object that scales to thousands of nodes.The most common option is to store VMDKs on the Hedvig cluster with NFS access. However, iSCSI LUNs are also available from the same platform. And for companies that are deploying additional workloads, S3, Swift, and soon SMBv3 (Hyper-V anyone?), are all available. All protocols perform as first class citizens in a cluster that can scale to thousands of nodes.

  • Seamless VMware integration with VAAI support and vCenter plugins.The Hedvig platform is easy to configure, whether you’re doing it from our UI, CLI, or APIs, or from within your VMware tools. Our per-volume provisioning granularity ensures you’ll be able to configure advanced data services like in-line deduplication, in-line compressions, tunable replication factors, snapshots, clones, and caching on a per-VM basis. It’s a natural fit for the VVOL architecture, which we’re working to support now.

  • A hybrid architecture with built-in DR across multiple data centers and clouds. A Hedvig cluster can span multiple data centers and clouds. Combine that with our tunable replication and you have distinct DR policies built right in. For example, you could create a volume that is three-way replicated with one replica sent to your San Francisco data center, one replica sent to your Denver data center, and third replica sent to a Rackspace-powered vCloud Air. On the very same cluster you could then have a second volume with totally unique policies, such as four-way replication with replicas destined for San Francisco, Oregon, AWS, and Azure.

The bottom line: Hedvig is the first modern, all-software storage platform designed to provide enterprise-class storage for virtualized environments.

If you’re interested in learning more, click below. Better yet, if you’re attending VMworld in San Francisco, request a meeting with Avinash Lakshman and the rest of the Hedvig team!

Learn More