| Story Views | |
| Now: | |
| Last Hour: | |
| Last 24 Hours: | |
| Total: | |
The Future of large-scale, high-performance data processing infrastructure
From
Wired provided coverage of Google spanner
“If you want to know what the large-scale, high-performance data processing infrastructure of the future looks like, my advice would be to read the Google research papers that are coming out right now,” Mike Olson, the CEO of Hadoop specialist Cloudera, said at recent event in Silicon Valley. According to Charles Zedlewski, vice president of products at Cloudera, the company was already aware of Spanner — after recruiting some ex-Google engineers — and it may eventually incorporate ideas from the paper into its software.
Facebook is already building a system that’s somewhat similar to Spanner, in that it aims to juggle information across multiple data centers. Judging from our discussions with Facebook about this system — known as Prism — it’s quite different from Google’s creation. But it shows that other outfits are now staring down many of the same data problems Google first faced in years past.
The Spanner paper lists many authors, but two stand out: Jeff Dean and Sanjay Ghemawat. After joining Google from the research operation at DEC — the bygone computer giant — Dean and Ghemawat helped design three massive software platforms that would have a major impact on the rest of the internet. MapReduce and the Google File System gave rise to Hadoop, while BigTable helped spawn an army of “NoSQL” databases suited to storing and retrieving vast amounts of information.
Spanner draws on BigTable, but it goes much further. Whereas BigTable is best used to store information across thousands of servers in a single data center, Spanner expands this idea to millions of servers and multiple data centers.
See more and subscribe to NextBigFuture at 2012-11-12 05:01:54 Source: http://nextbigfuture.com/2012/11/the-future-of-large-scale-high.html
Source:
