GET HADOOP TECHNOLOGY STACK TO FIT IN YOUR ARCHITECTURE

Hadoop Core / Common Project

  • Distributed Storage : HDFS
  • Distributed Processing : MapReduce (MR1)
  • Distributed Scheduling : YARN (MR2) (its started in Hadoop v2)

How data can be accessed and processed  from Hadoop FrameWork without writing Map Reduce Job

  • PIG : http://pig.apache.org/
  • Hive : http://hive.apache.org/

How to Process Data Storage or DB in Hadoop

  • HBase : http://hbase.apache.org/
  • Cassandra : http://cassandra.apache.org/

Storage Management Services

  • HCatalog : http://incubator.apache.org/projects/hcatalog.html

RegEx and Search Tool

  • Lucene : http://lucene.apache.org/

Bulk Synchronous Parallel computing engine

  • Hama : http://hama.apache.org/

Managing MapReduce Pipelining

  • Crunch : http://crunch.apache.org/

Data Serialization to send data to another application in some format like JSON, XML

  • Avro : http://avro.apache.org/
  • Thrift : http://thrift.apache.org/

Data Intelligence

  • Drill : https://incubator.apache.org/drill/drill_overview.html
  • Mahout : http://mahout.apache.org/

Real Time Log Processing Tool

  • Flume : http://flume.apache.org/
  • Chukwa : http://chukwa.apache.org/

Data Integration to connect RDBMS to HDFS

  • Sqoop : http://sqoop.apache.org/

Distributed Service Coordinator

  • Zookeeper : http://zookeeper.apache.org/

Work Flow or Job Scheduler

  • Oozie : http://oozie.apache.org/

Centralized Service Management, monitoring and Orchestration

  • Ambari : http://ambari.apache.org/

 Centralized Security of Hadoop Project

  • Knox : http://knox.apache.org/

Eclipse IDE plugin for Development

  • HDT : http://hdt.incubator.apache.org/

Project that is 100x Times faster than MapReduce

  • Spark : http://spark.apache.org/

 To get the list of ALL apache Incubator project  : http://incubator.apache.org/projects/

Hadoop Technology
  • November 4th, 2014

You May Also Like

How To Install phpmyadmin in Ubuntu 16.04 LTS
  • April 25th, 2016

From What so ever source you upgrade your system to ubuntu 16.04 it willl remove the mcrypt from the present system if you have phpmyadmin previously installed in the system. This will create a error of “mbstring”. To rectify this and install phpmyadmin in your 16.04 ubuntu please run th following commands one by one. […]

WordPress Plugins – Email, Print, Fonts Size Plugins (All in One)
  • February 5th, 2016

Our Plugin is a remarkable Plugin which works to facile the users with various exceptional options. It provides the users with the options to Send an Email, Print a Post and Increasing and Decreasing the Font Size of a post along with the addition of appropriate buttons to the content. A user can perform a […]

Installation Guide For Django (Python)
  • January 1st, 2016

Introduction Django is one of the most popular python web framework. Although it is considered as an MVC framework but basically it is an MTC (model, template, controller) architecture . You can install Django in your Ubuntu Operating System by following the below mentioned steps – Before we start, I hope you are familiar with […]

Hadoop Technology
  •  November 4th, 2014
keyboard_arrow_up