I thought that was interesting. It was the first time this skills grouping appeared on the top-25 list—bumping 2014’s #1, Statistical Analysis and Data Mining, to #2. Pretty cool! It further validates that the tech job market and how workers are defining and developing their skills are catching up to the reality and popularity of distributed systems.
Why distributed computing is top of mind for companies of all sizes
We all know that businesses are changing at such a fast rate that more traditional, large enterprises are finding it hard to keep up. IT has been unable to keep pace, requiring IT pros to architect entirely new systems that can match the speed at which businesses are evolving. Hedvig CEO, Avinash Lakshman, often says that the order of magnitude of this problem can only be solved in a distributed manner. Parallelizing efforts enables IT to change rapidly and survive failure. This trend is often referred to as webscale or hyperscale IT.
This dilemma now extends to more traditional non-webscale enterprises. Companies of every size face mushrooming amounts of data. However, harnessing that data unlocks new business models and revenue opportunities. So much so, that it’s no longer about the big winning over the small, it’s about the fast beating the slow. Companies that quickly monetize their data will win. This used to be the domain of a relative handful of webscale companies, but now the demand for distributed computing technology and skills has spread to nearly every enterprise. The problems that frustrated the “webscalers” in their early days have now come to roost at everything from banks and oil and gas companies to retailers and hospitals alike.
The shifts that brought distributed computing to the forefront in 2015
But what shifts—tectonic or otherwise—took place to bring cloud and distributed computing to the top spot in LinkedIn’s list for 2015? One explanation may be the recognition that, while big data has been very much top of mind in the last couple of years and is now generating serious IT investment, all of these big data tools and software put a tremendous strain on traditional IT architectures. They simply cannot provide enough horsepower. A distributed computing approach is necessary for the timely analysis of data. Distributed systems are the supply following big data demand.
Another reason for the shift may be the popularity of Docker and containers. In the past, a lack of skills has held back popular platforms like OpenStack. But what makes Docker so compelling is that it was the first time infrastructure has been a distributed system modeled to be very developer oriented, creating a vast and free open-source playground where people could develop this skillset. We haven’t seen this kind of adoption since AWS, another infrastructure trend that took root with developers.
Now, we see distributed computing populating the entire IT landscape. Docker is a distributed system that uses containers to change how applications get built, shipped, and run. Cassandra is a distributed system that’s applied to the database tier. In fact, Hedvig is a distributed system. It just so happens it’s one Avinash created to solve storage problems.
Finding opportunities to learn distributed computing skills
While it’s exciting to see cloud and distributed computing at #1 on LinkedIn’s top job skills list for 2015, unfortunately we still see a lack of job candidates steeped in distributed computing models. The good news, though, is that more schools, including University of Illinois at Urbana–Champaign (UIUC), University of California at Santa Cruz (UCSC), and Cornell University, now have dedicated distributed systems curricula. These universities are teaching students not only how to think about solving problems with distributed computing, but to actually code those solutions as well.
We’re fortunate to be leading the charge in distributed systems in data storage at Hedvig. But as this LinkedIn list shows, cloud and distributed computing (and distributed systems, more broadly) are poised for even more rapid growth ahead.
What we’ve found is that combining distributed computing skills with more traditional IT skill like storage (#8 on LinkedIn’s 2015 skills list, and #3 on the 2014 list) makes for a productive developer at Hedvig. This blended skillset ensures distributed computing has a practical application while at the same time ensuring the “old” way of thinking is weeded out. Otherwise you end up with the incremental innovation that’s plagued storage for the past 20 years.
Bottom line, if you’re looking to expand your career then we think LinkedIn got it right. Learn distributed and cloud computing. It’s the future. Or better yet, join Hedvig! We’re a fantastic place to learn from distributed systems pioneers. Get inspired and make an impact—to learn more click below for our current openings.