Of cattle and pets – the modern world of distributed systems

By Eric Carter | | Cloud

Do you love your storage array like you love your dog? There’s a very-useful-but-some-hate-it metaphor commonly used when discussing distributed systems – Cattle vs. Pets. In our IT shops sometimes we adopt gear that is like our pet cat Fluffy – we name it, we care for it, we feed it. When it gets sick, we do everything we can to nurse it back to health. But is this the best approach to IT infrastructure?

On our webinar yesterday, Luke Marsden, the Co-founder and CTO of ClusterHQ (the folks behind Flocker) expressed this concept:

“This is how we used to deal with servers. We would lovingly go down to the data center at 3am and try to nurse a sick machine back to health. With the advent of things like the cloud, that’s no longer the case. We ought to be able to just shoot a machine in the head if it is unwell and have the systems that we’re building automatically recover.”

Numbering and herding our storage infrastructure

Modern systems are geared to enable you to use commodity servers (cattle) in an aggregate cluster of compute/storage power, letting software automatically handle the movement of apps and data when any server or any of its components get sick. Here I choose WHEN on purpose, and not IF. We should expect component failure. Hedvig’s Distributed Storage Platform software is designed for this. It’s built on distributed systems principles. Think about managing a storage system comprised of hundreds of nodes – numbered, not named. When your software detects failed nodes or components, recreating data from the down node on good, healthy nodes – call it repair, self-healing – is done automatically. You can swing through your data center and shoot your broken server in the head at your leisure – no emergency here.

Here’s additional, helpful reading on this same theme:

My goal with this blog is simply to put in perspective the transformation going on with modern IT infrastructure. I sincerely hope I do not offend anyone. In particular I hope my wife doesn’t read it because she would have 99 reasons why cattle are mistreated. I get it. To redeem myself for raising this off-putting metaphor, I leave you with this classic pet-focused video: