Docker containers and software-defined storage

By Rob Whiteley | | Cloud

Last month was the second annual DockerCon, where 2,100 people gathered to discuss the architectural shift to containers. Missed the event? No worries. You can read my favorite recap in TechCrunch or Eric’s blog here. If you’re familiar with Docker and DockerCon, then you know how fast containerization is growing. Consider these stats from 745 respondents of a recent StackEngine survey:

  • 70% are using or evaluating Docker

  • As always, adoption starts in QA/test (63%) and Dev (53%)

  • 31% are already using it or plan to use in Production

  • 49% mentioned security and lack of operational tools as the #1 challenge

  • 50% are open to using new tools, which is consistent with our experience

So all’s well, right? Not so much. Docker has taken hold so quickly that many in-the-know IT professionals and storage admins don’t yet understand the value of this open-source project. But they will very soon.

Due to all the next-generation storage developments we are working on at Hedvig, we’ve been on the forefront of containerization and are happy to share what we know so far. Docker containers provide a lot of opportunity for storage, especially as the container technology matures. Here is our primer on the essential info to know when it comes to the intersection of software-defined storage and Docker.

Why should I deploy containers?

Containerization emerged out of a need to isolate workloads on a host, but has gained momentum as a distributed architecture. It has two benefits: 1) apps can be broken down into microservices, providing a more scalable alternative to monolithic apps; and 2) developers and IT teams can streamline application development and distribution. We’ll cover the microservices aspect in a future blog. For now, let’s focus on the fact that software like Docker and CoreOS provides a virtual container where developers can build and package an application or component of an application for deployment to hundreds or even thousands of machines.

Containers are ephemeral – so at their very nature, temporary. Container storage capabilities are basic and minimal. This is evident in the rudimentary Docker data volumes, which store and provide access to data as long as they are attached to a container. However, as Docker makes its way into the enterprise and production environments there is an exponentially greater need for well-managed, persistent storage. Solutions like ClusterHQ’s Flocker solution are helping to create APIs and orchestration services for persistent data. It aims to solve the kludgy problem of creating “data-only containers” for persistent storage by effectively connecting a container to underlying storage. While traditional storage solutions can be adapted to this strategy, software-defined storage is increasingly a more attractive choice because of its flexibility and programmability. As you spin up more containers you can spin up more corresponding storage, a task much harder to accomplish with monolithic arrays. As Docker takes hold in the enterprise, IT simply cannot invest in multi-million dollar hardware infrastructure that cannot be scaled or replaced quickly enough to keep up with change.

How does Hedvig work with containers?

At their core, Docker and Hedvig share a common bond – they were both created to function as distributed systems. Docker to solve server virtualization woes at scale; Hedvig to provide flexible storage for cloud and on-premise environments. Both also share many similar features, for example access to REST and RPC APIs as well as elasticity — you can provision and deprovision capacity dynamically. However, the biggest benefits of using Hedvig in conjunction with Docker lie in the simple fact that the Hedvig Distributed Storage Platform, like Docker, is dedicated to making development easier by:

  • Scaling both storage performance and capacity. Since Docker containers run a variety of application services, developers need storage that is responsive to a variety of datasets. Hedvig supports block, file, and object storage protocols and can scale from just a few nodes to thousands, ensuring fast, flexible access to persistent data storage.

  • Extending container data volumes. Managing storage for containerized apps in production environments requires performance, capacity optimization, data protection, and business continuity capabilities that don’t exist in the basic Docker data volume functionality. Hedvig augments Docker with capabilities like replication, compression, deduplication, snapshots, and clones.

  • Focusing on automation and programmability. The app development lifecycle is shrinking, so data access and storage can’t be a bottleneck for developers. The Hedvig Distributed Storage Platform provides a full suite of ready-to-use APIs for configuration, provisioning, maintenance, reporting, and analytics. It also provides a simple, intuitive interface for self-service developer portals.

  • Docker has only existed for a few years and is still very much in its infancy. Like VMware has advanced the storage functionality and integration capabilities of vSphere with VAAI and now VVOLs, we expect that Docker and the open-source community will take containers down a similar path. Already, evidence of rapid improvements are forthcoming, including last month’s engine-extending plugins. Hedvig is the right choice for organizations looking for a fool-proof storage solution to deploy with Docker. Our goal is to support the growing appetite of developers for Docker with a storage platform that takes versatility to the next level, and to provide the tools needed by IT organizations to be successful managing storage in the evolving world of containers.

  • Want to learn more? Click below for more info on Hedvig and Docker storage.

    Learn More