Organizations that want to run Apache Hadoop to analyze their big data without setting up a computer cluster can now procure the data processing framework as a service from a startup co-founded by the former CTO of Yahoo.
Altiscale aims to reduce much of the administrative overhead usually required to run the open source software.
"In all other Hadoop offerings, you have to worry about nodes. 'Do I have enough nodes?' 'Do I have too many nodes?' 'How do I get nodes need to get?'," said Raymie Stata, Altiscale CEO, and former CTO of Yahoo.
With the Altiscale Data Cloud, "There's no worrying about nodes. You submit jobs and they just run," Stata said.
Stata was the CTO at Yahoo who initially sponsored the use of Hadoop, which was first developed at Yahoo. After he left the company, he co-founded Altiscale in 2012 and has received US$12 million in investment from Sequoia Capital, General Catalyst and Accel Partners.
Altiscale employs other Yahoo Hadoop veterans as well. Altiscale CTO David Chaiken deployed Hadoop to drive all of Yahoo's advertising systems. Altiscale head of operations Charles Wimmer ran a 40,000 node multitenant Hadoop cluster at Yahoo.
The company has been running in stealth mode since last year, though now has opened its services for general availability.
The pricing structure is similar to most cell phone plans, meaning the customer pays a set monthly fee for a certain amount of usage and then is billed for any overages. The basic plan is 10TBs and 10,000 task hours for US$2,500 per month. Additional storage and compute can be purchased as well.
The company can handle jobs as small as a few gigabytes to as large as multiple terabytes. The service can scale to whatever size the customer needs. Clients submit jobs through Internet-accessible APIs (application programming interfaces), using the commands for Hadoop's YARN (Yet Another Resource Scheduler) and the HDFS (Hadoop File System).
Using Hadoop as a hosted service could offer a number of benefits over other approaches, Stata said.
Running Hadoop on premises would require provisioning a lot of equipment and requires a fair amount of expertise as well, which the average enterprise may not have, he said.
Many companies, such as IBM and Computer Sciences Corp., offered managed Hadoop deployments, but those can be expensive and may not offer help desk support.