Despite Hadoop’s and OpenStack’s adoption, our recent discussions with enterprises and technology providers revealed two prominent trends:
-
Big Data will need more than a Hadoop: Along with NoSQL technologies, Hadoop has really taken the Big Data bull by the horns. Indications of a healthy ecosystem are apparent when you see that leading vendors such as MapR is witnessing a 100% booking growth, Cloudera is expecting to double itself, and Hortonworks is almost doubling itself. However, the large vendors that really drive the enterprise market/mindset and sell multiple BI products – such as IBM, Microsoft, and Teradata – acknowledge that Hadoop’s quantifiable impact is as of yet limited. Hadoop’s adoption continues on a project basis, rather than as a commitment toward improved business analytics. Broader enterprise class adoption remains muted, despite meaningful investments and technology vendors’ focus.
-
OpenStack is difficult, and enterprises still don’t get it: OpenStack’s vision of making every datacenter a cloud is facing some hurdles. Most enterprises find it hard to develop OpenStack-based cloud themselves. While this helps cloud providers pitch their OpenStack offerings, adoption is far from enterprise class. The OpenStack foundation’s survey indicates that approximately 15 percent of organizations utilizing OpenStack are outside the typical ICT industry or academia. Moreover, even cloud service providers, unless really dedicated to the OpenStack cause, are reluctant to meaningfully invest in it. Although most have an OpenStack offering or are planning to launch one, their willingness to push it to clients is subdued.
Why is this happening?
It’s easy to blame these challenges on open source and contributors’ lack of coherent strategy or vision. However, that just simplifies the problem. Both Hadoop and OpenStack suffer from lack of needed skills and applicability. For example, a few enterprises and vendors believe that Hadoop needs to become more “consumerized” to enable people with limited knowledge of coding, querying, or data manipulation to work with it. The current esoteric adoption is driving these users away. The fundamental promise of new-age technologies making consumption easier is being defeated. Despite Hortonworks’ noble (and questioned) attempt to create an “OpenStack type” alliance in Open Data Platform, things have not moved smoothly. While Apache Spark promises to improve Hadoop consumerization with fast processing and simple programming, only time will tell.
OpenStack continues to struggle with a “too tough to deploy” perception within enterprises. Beyond this, there are commercial reasons for the challenges OpenStack is witnessing. Though there are OpenStack-only cloud providers (e.g., Blue Box and Mirantis), most other cloud service providers we have spoken with are half-heartedly willing to develop and sell OpenStack-based cloud services. Cloud providers that have offerings across technologies (such as BMC, CloudStack, OpenStack, and VMware) believe they have to create sales incentives and possibly hire different engineering talent to create cloud services for OpenStack. Many of them believe this is not worth the risk, as they can acquire an “OpenStack-only” cloud provider if real demand arises (as I write the news has arrived that IBM is acquiring Blue Box and Cisco is acquiring Piston Cloud).
Now what?
The success of both Hadoop and OpenStack will depend on simplification in development, implementation, and usage. Hadoop’s challenges lie both in the way enterprises adopt it and in the technology itself. Targeting a complex problem is a de facto approach for most enterprises, without realizing that it takes time to get the data clearances from business. This impacts business’ perception about the value Hadoop can bring in. Hadoop’s success will depend not on point solutions developed to store and crunch data, but on the entire value chain of data creation and consumption. The entire process needs to be simplified for more enterprises to adopt it. Hadoop and the key vendors need to move beyond Web 2.0 obsession to focus on other enterprises. With the increasing focus on real-time technologies, Hadoop should get a further leg up. However, it needs to provide more integration with existing enterprise investments, rather than becoming a silo. While in its infancy, the concept of “Enterprise Data Hub” is something to note, wherein the entire value chain of Big Data-related technologies integrate together to deliver the needed service.
As for OpenStack, enterprises do not like that they currently require too much external support to adopt it in their internal clouds. If the drop in investments is any indication, this will not take OpenStack very far. Cloud providers want the enterprises to consume OpenStack-based cloud services. However, enterprises really want to understand the technology to which they are making a long-term commitment, and are cautious of anything that requires significant reskill or has the potential to become a bottleneck in their standardization initiatives. OpenStack must address these challenges. Though most enterprise technologies are tough to consume, the market is definitely moving toward easier deployments and upgrades. Therefore, to really make OpenStack an enterprise-grade offering, its deployment, professional support, knowledge management, and requisite skills must be simplified.
What do you think about Hadoop and OpenStack? Feel free to reach out to me on [email protected].
Photo credit: Flickr
