通信工程文献翻译——摘要-当谷歌在2004年首次提出Map / Reduce框架时，没有与之可比较的系统提供给公众。从那时起情况发生了变化。Map / Reduce框架已经变得越来越受欢迎。并且在当今的计算世界中也不缺少Map / Reduce的实现。主要的解决方案是目前Apache Hadoop，由雅虎开始。除了采用定制的Map / Reduce安装，云服务的客户现在可以利用现成的安装（例如弹性地图/恢复系统）。

与此同时，其他第二代框架也开始出现，他们可以为特定的场景微调Map / Reduce模型，或者完全改变框架，例如谷歌的Pregel。在本文中，我们介绍这些第二代框架和当前事实上的标准Hadoop之间的比较，通过集中在一个特定的场景：大规模图分析。我们通过利用它们的独特特征来分析微调这些系统的不同方法。我们的分析基于k-核心分解问题，其目标是计算给定图形中每个节点的中心性;我们在Amazon EC2节点集群中测试了我们的实现，并使用SNAP项目公开提供的现实数据集。

第一章 引言

美国民俗有一个神话是说车库作为创业的发源地。 Google就是一个著名的例子：公司的规模是从Menlo Park的一个车库扩展到一个索引超过450亿页的搜索提供商，1。 为了最小化其初始硬件成本，他们利用商品硬件集群，而不是投资大型超级计算机。 为了开发新的并行算法，Google的工程师很快意识到，他们需要一种方便的编程方法和一个协调其执行的框架。 这些要求导致了Map / Reduce平台的设计[7]。 Map / Reduce从2004年首次发布以来，在Google内使用的所有地方，受到了科学和商业计算机界的很多关注。之后，Map / Reduce模型在开源框架Apache Hadoop中重新实现[25]。

Map / Reduce框架的可用性和云提供的资源（例如亚马逊的EC2 [1]）的丰富性使得对大量数据的分析可用于每个人，并且开始了现在称之为“大数据”的运动。今天，每个人都可以基于大数据的处理开展业务，而不会因为初始投资过度压倒。通过使用这些技术，它现在可以分析如此大的数据集，但如果没有超级计算机，这些数据集可能不会被处理。最初，Map / Reduce被提出用于非常简单，尴尬的并行任务，如应用于非常大的分布式的数据集的日志分析。后来，越来越多的论文试图将Map / Reduce应用于更大的问题，包括机器学......

Abstract—When Google first introduced the Map/Reduce paradigm in 2004, no comparable system had been available to the general public. The situation has changed since then.

The Map/Reduce paradigm has become increasingly popular and there is no shortage of Map/Reduce implementations in today’s computing world. The predominant solution is currently Apache Hadoop, started by Yahoo. Besides employing custom Map/Reduce installations, customers of cloud services can now exploit ready-made made installations (e. g. the Elastic Map/Reduce System).

In the mean time, other, second generation frameworks have started to appear. They either fine tune the Map/Reduce model for specific scenarios, or change the paradigm altogether, such as Google’s Pregel. In this paper, we present a comparison between these second generation frameworks and the current de-facto standard Hadoop, by focusing on a specific scenario: large-scale graph analysis. We analyze the different means of fine-tuning those systems by exploiting their unique features. We base our analysis on the k-core decomposition problem, whose goal is to compute the centrality of each node in a given graph; we tested our implementation in a cluster of Amazon EC2 nodes with realistic datasets made publicly available by the SNAP project.

I. INTRODUCTION

A key myth of American folklore is the garage as birthplace of start-ups. Google is a famous example: the company scaled from a garage in Menlo Park to a search provider indexing more than 45 billion pages1. To minimize their initial hardware costs, they exploited clusters of commodity hardware instead of investing into large supercomputers. To develop novel parallel algorithms, engineers at Google soon realized that they needed both a convenient way of programming them and a framework to orchestrate their execution. These requirements resulted in the design of the Map/Reduce platform [7]. Used everywhere inside Google [6], Map/Reduce received a lot of attention from scientific and business computing since its first publication in 2004. The Map/Reduce model was later reimplemented in the open-source framework Apache......