What Pivotal’s big open source move really means

Yesterday, Pivotal revealed it will open-source its entire Big Data Suite, including HAWQ, Greenplum, and GemFire. This was on the heels of an announcement that Pivotal would partner with Hortonworks on the Open Data Platform, a new consortium intended to prevent Hadoop fragmentation and create a “tested reference core” of Hadoop technologies.

According to Pivotal, the company is rocking and everything it does results in business success. I’m no MBA, but in boxing you don’t totally change your strategy unless it isn’t working or you see trouble ahead. Imagine: “I have my opponent on the ropes, time to switch to a stick-and-move strategy instead.” (Disclaimer: My company has ties to many of the companies mentioned in this post as well as some of their competitors. Those who know me realize this has no effect on my opinion.)

At the beginning of the year, I called out Pivotal in my annual “10 things you don’t need to worry about” post, noting that Pivotal is pushing its Hadoop offering, Pivotal HD, and you can probably ignore the rest. Within a week or so, a press release appeared claiming Pivotal had booked $40 million in nine months. Then James Waters, the head of Pivotal’s Cloud Foundry group, slammed me on Twitter, saying I was exactly wrong and implied everything was exactly the opposite.

I’m almost ready to eat half of a crow. I was right that Pivotal is a double-headed monster and you can’t have a wide-platform startup (or even a pseudo-startup in Pivotal’s case). So far PaaS has been mostly a nonstarter; I’ve never seen an implementation of a “hybrid cloud” PaaS. I’ve seen lots of field sales activity from Pivotal in big data, but not much for Cloud Foundry, so I supposed that Pivotal HD was the favorite son.

But with the announcement of Pivotal open-sourcing Big Data Suite and joining with Hortonworks to add HAWQ (to oversimplify, Pivotal’s answer to Impala or Hive) to the Hortonworks Data Platform, Pivotal seems to be punting its big data business to Hortonworks. Supposedly, despite cooperating with Hortonworks (until now a competitor) Pivotal will continue to sell Pivotal HD, but move to more of a subscription model.

I couldn’t help but feel like today’s announcement was bit like a Vietnam-era ceremony where Johnson pinned a medal on General Westmoreland and fired him.

Here is the question about Pivotal’s future: Like VMware, it’s largely a division of EMC, which is in the business of selling expensive storage solutions. While open source and the subscription model are great for startups, EMC’s margins are usually much higher. Pivotal does not help EMC sell storage solutions, so what, ultimately, will EMC do with Pivotal?

Here’s what I’ll be looking for in the coming months:

• How many committers does Pivotal have on the Open Data Platform or the aggregate of its Apache projects? Will those committers do more than “Cloud Foundry integration” in six months?
• What sort of field sales activity will we see from Pivotal?
• Most of the marketing is now about Cloud Foundry. Will we see anything more about Pivotal HD or its Open Data Platform as a commercial product?

This is where we stand now: Pivotal says Pivotal HD was a huge success, but it’s open-sourcing the entire Big Data Suite and integrating into a competitor’s platform … although Pivotal will continue to sell Pivotal HD, and everything will eventually be part of Cloud Foundry, which will have its own app store.

What does it all mean? Well, an open source Greenplum isn’t great news for Teradata, and folding into Hortonworks isn’t great news for Cloudera or MapR (though I haven’t seen HAWQ set the world on fire). Clearly, I was wrong about Pivotal choosing HD over the Cloud Foundry PaaS, but I get half a point for asserting it had to choose one.

The question remains: Can you build a business on local or hybrid PaaS? In the coming months and years, we shall see.