Qubole rains down analytic insights from Hadoop cloud

Comment A whole new generation of storage software application startups focused on Big Data is growing furiously in front of our eyes.

Engineers involved with Google, Facebook and other big data-using, web-scale services pioneers have seen that where Facebook, Google and others lead enterprises are going to have to follow.

Relational databases, SANs and filers are simply crap at storing unstructured data. Legacy data warehouse and business intelligence tools are crap at analysing it.

Yet, the world is awash with it and rather than stumbling around in the dark business should put on Big Data glasses and explore it using new-style analytics so that it can offer better, more efficient services.

A whole Big Data storage and analytics infrastructure and eco-system is in a phase of rampaging growth, with millions and millions of dollars of funding encompassing many and diverse customers. Some we know about, like Cloudera and Elastic Search. Here’s another one - Qubole, offering analytics on Hadoop data as a service (HaaS).

Qubole was started up by Ashish Thusoo and Joydeep Sen Sarma in 2011. Both are ex-Facebook engineers who have parlayed their Facebook big data experience into Qubole; taking that experience and building a self-service offering for enterprises.

They claim they created Facebook’s big data infrastructure and Apache Hive, which is data warehouse-type software using a SQL-style language to ask questions of data stored in Hadoop. The two also managed an analytics team inside Facebook and were project leads for Apache Hive for a year.

Their startup received $6m A-round funding in 2013, and that, plus their seed money totalled $7m which they spent on developing the Qubole Data Service (QDS) - a Hadoop platform running in the cloud.

Qubole Data Service concept

The idea is to have on-demand elastic clusters in the cloud as an alternative to on-premises Hadoop clusters. You scale up the HaaS cluster as you need and scale it back down as you need, avoiding having a relatively static infrastructure in your own data centre.

QDS does the scaling with no need for users to have Hadoop cluster skills.

It claims it is much faster than Amazon at this, bringing up a 200-node cluster in four minutes where Amazon EMR (Elastic Map Reduce) takes 40 minutes. Qubole estimates its HaaS has 50 per cent fewer failures than Amazon EMR to.

The service has been successful enough - web and mobile application company Pinterest is a customer, and there are 70-60 paying customers in US, India, and Europe. Some have enterprise deals to support their business.