Are we there yet? What’s up with software-defined storage

By Eric Carter | | Cloud

I recently read this article on about the state of software-defined storage (SDS) and it reveals some important data about the momentum of the technology. 95% of surveyed companies expressed interest in the SDS approach. This is huge. To me it signals that there is now greater awareness of the potential that a modern, web-scale storage solution holds. Beyond solely “interest,” the survey also reveals that there are significant plans underway for actual adoption in the next 12 months. Those who choose wisely will reap the greatest benefit.

According to the survey, nearly two-thirds – 63% – say their organization will begin to adopt SDS in the next 12 months. 82% said organizations in their sector need to rethink their approach to storage given the growth in data volumes to avoid their business being compromised. At Hedvig, we’re excited by the prospect that enterprises are moving in this direction. We know that there are hundreds of thousands of existing storage arrays deployed in data centers across the globe, so the potential for SDS as a replacement means lots of opportunity. One caution to those investigating the technology right now – with so many vendors messaging around software-defined storage it will pay off to do a thorough investigation of the available technology options. Not all SDS is created equal!

  • Some solutions package traditional array controller software as a virtual appliance and call it SDS.
  • Some focus on the control plane only, meaning software that sits above your existing arrays to virtualize capacity – what I think of as storage virtualization – and call it SDS.
  • Some package scale-out capabilities into appliances (hyperconverged or otherwise) and call it SDS.
  • Some are object storage platforms that are SDS.
  • Some are global namespace solutions – again all SDS.
  • Some, like Hedvig, use distributed systems principles and provide multi-protocol support – you guessed it, we call it SDS.

At Hedvig, we begin most our discussions and presentations by providing our definition and vision of SDS. We do this to cut through the noise as we are keenly aware of the dilemma posed by the wide range of solutions being sold under that SDS moniker.

We believe SDS is first and foremost available as software – something that installs and runs on industry-standard commodity server and disk hardware and/or instances in the public cloud. It abstracts underlying resources, whether flash or HDD, and allows the addition of resources over time without disruption (scale-out). SDS is built on modern, distributed systems principles meaning it harnesses a group of nodes and distributes control functions and data to eliminate single points of failure. It is designed with automation and programmability to provide more flexibility and enable organizations to define and allocate capacity dynamically and efficiently to meet business and application needs. And for us, knowing that organizations have come to rely on a wide range of enterprise storage features, we believe that to be successful, SDS must deliver these as well. This includes block, file, and object protocols, snaps and clones, deduplication and compression, encryption, multi-site replication, and support to name a few.


If your IT shop or organization is considering SDS, spend time researching the available options and understand which SDS platforms provide the most interoperability, flexibility, performance and scalability to meet your existing and future requirements. In some cases, it may be a walk-before-you-run scenario – where you first audit your data center to determine how best to benefit from SDS. Is there a project or application that is ripe for a modern approach to storage? A cloud project? VMware modernization or perhaps Docker container apps? Or what about an economical and more efficient backup storage or archiving target? These are instances where SDS can improve existing infrastructure or even replace obsolete, slow hardware.

The biggest benefits associated with SDS, according to the survey, were improved system performance, facilitation of web, mobile and big data apps and services, and reduced support and maintenance costs. Our take on this is that while performance from a sheer IOPS delivery perspective can be a consideration, the main factors that compel customers to move to SDS are manageability, automation and scalability. SDS provisions additional storage quickly, even programmatically, avoids human latency, and brings products or services to market faster.

At the same time, new apps as well as modern, web-based apps are driving SDS adoption. Many of these systems are built on the principles of leveraging a software-defined approach and commodity infrastructure throughout the organization, and storage naturally follows suit. And more than just support and maintenance cost reduction, our customers also expect capital expenditure reductions by moving to off-the-shelf x86 server hardware as a way to get out from under the “bezel tax” of proprietary arrays.


The survey also found that more than half – 54% – of respondents cited the need to simplify their storage as a data management priority during the next 12 months. There’s little doubt that organizations want storage simplicity; they also increasingly want simplicity in provisioning. That is, can it be more “AWS-like,” where they can click a few buttons to get what they need, or have the application framework programmatically ask for what it needs, and get it?

This is a shift away from the historical approach of opening trouble tickets to request storage and then waiting hours, days, or weeks to have the storage provided and the ticket closed. Customers also want simplicity in scaling, which arises from the ability to add nodes: additional servers with CPU, RAM, SSD/HDD join the storage cluster and the software immediately starts to take advantage of the added capacity.

More than 90% of respondents said their businesses are considering or would consider an open source approach to SDS. But in reality open source SDS has proven quite complicated because achieving the desired results involves far more than just pulling some code off GitHub. While our founder, Avinash Lakshman, has a pedigree of open-sourcing Cassandra to the Apache Software Foundation, we’ve found that enterprises are still somewhat leery of putting all their data eggs in the open source basket, especially for something as critical as long term retention of core business data. It is done – Ceph is an example – but the complaints we hear focus on complexity, performance, code stability and support, to name a few. Ceph and other open source storage often takes a larger set of IT staff with very specialized skills.

Hedvig is not open source – by choice. We’ve built a business around our unique approach to storage and customers have let us know they believe in our vision of software-defined data centers. They want a solid product and around-the-clock reliable support, and are willing to pay for this value. Having said this, our customers already know that open source isn’t free. In fact, we’ve compared the cost of support for open source solutions with the total cost of a Hedvig deployment, and the two figures are in the same ballpark.

To those who say that proprietary solutions are designed to lock in customers, allow me to counter that the point, rather, is to find a solution that delivers value commensurate with the dollars you’re prepared to spend to solve a problem. The solution, whether proprietary or open source, must be reliable, easy-to-use, provide broad interoperability, robust APIs, and a rich feature set. In short, the solution must provide an advantage to the business.


Of those surveyed, 78% noted they need a flexible storage solution that keeps pace with the always-changing demands of their data centers. We agree. It’s why Avinash saw enterprise storage as such a strong opportunity to which he could bring to bear his distributed-systems expertise gained from building systems that run key functionality at Amazon and Facebook. There are two areas in particular that apply:

  • Digital transformation. Storage plays a central role in digital transformation. If data is the second most critical asset an organization has after its people (which has long been Hedvig’s view), then in today’s hyper-connected world, storing that data is by definition central to digital transformation. The Hedvig Distributed Storage Platform is designed to put an organization’s data front and center in its digital transformation process. How? Because Hedvig supports all storage protocols, multiple APIs, has hardware flexibility, works in multi-datacenter environments as well as in public cloud vendors, and allows full customization per storage volume to fit applications.
  • The rise of multi-cloud architectures. The limits of hyperconvergence and the growing importance of multi-cloud were not addressed in the survey. Hyperconverged storage solutions have found a niche in many data centers, bringing appliance simplicity and scale-out capabilities. But they’re not the panacea many paint them to be. As organizations look to public cloud and a multi-cloud strategy, the hyperconverged approach is seen as a limiting construct. Instead, the flexibility to distribute data and scale storage across sites independent from application compute resources that leverage software-defined solutions is the model that holds more promise to meet the diverse needs of modern business. VMware’s partnership with AWS will soon make a hybrid approach that enables applications from existing data centers running vSphere to migrate seamlessly to and from the public cloud a reality. This capability requires data to be synchronized and distributed across geographies to support business needs. Only Hedvig today is able to deliver this flexibility with our Universal Data Plane.

Overall, the survey’s findings are compelling and, in our view, support the argument that SDS is turning the corner from experimental and “early days” to being put in place by the world’s largest enterprises. Adopting this approach will be crucial to the ability for any organization to adapt quickly to the opportunities they have in front of them. Anyone making the decision to deploy a software-defined approach should carefully define their use cases, research their options, and separate the wheat from the chaff to ensure their journey into SDS is successful.

Want to learn more about the value of SDS? Check out our handy infographic that shows how our approach to SDS delivers a lower TCO.