AkbarAhmed.com

Learn more

Convert CDH4 from YARN (MRv2) to MRv1

Introduction

I had configured only YARN in my original post on how to Install Cloudera Hadoop (CDH4) with YARN (MRv2) in Pseudo mode on Ubuntu 12.04 LTS.

Importantly, YARN is not ready for production yet, so we’ll go ahead and install MRv1 to get some production development done.

Stop the YARN Daemons

We first have to stop all daemons associated with YARN only packages. 

sudo service hadoop-yarn-resourcemanager stop
sudo service hadoop-yarn-nodemanager stop
sudo service hadoop-mapreduce-historyserver stop

Install the Missing MRv1 Packages

Next, we’ll install 2 packages that are required for Map Reduce v1, but were not also part of the MRv2/YARN installation.

sudo apt-get install hadoop-0.20-mapreduce-jobtracker
sudo apt-get install hadoop-0.20-mapreduce-tasktracker

Start the MapReduce v1 Daemons

sudo service hadoop-0.20-mapreduce-jobtracker start
sudo service hadoop-0.20-mapreduce-tasktracker start

Responses

  1. shoshy Avatar
    shoshy

    great tutorial. One problem: i’ve followed your java installation tutorial
    and when i echo $JAVA_HOME or when i hit “java” it works.
    But when i try to execute the commands to run , i get:

    +======================================================================+
    | Error: JAVA_HOME is not set and Java could not be found |
    +———————————————————————-+
    | Please download the latest Sun JDK from the Sun Java web site |
    | > http://java.sun.com/javase/downloads/ < |
    | |
    | Hadoop requires Java 1.6 or later. |
    | NOTE: This script will find Sun Java whether you install using the |
    | binary or the RPM based installer.

    I've done this twice by now on a brand new ubuntu 64 bit os. Something is missing in the java tutorial (https://akbarahmed.com/2012/06/22/install-java-jdk-6-0-update-26-on-ubuntu-12-04-lts/) which doesnt make it to work.

    Reply
    1. Pranesh Avatar
      Pranesh

      Still looking for a solution? Anyway, I’ll leave a reply for anyone else coming to this excellent website.
      Go to /usr/lib/hadoop/etc/hadoop/hadoop-env.sh and export JAVA_HOME to the adress in the Java tutorial. (https://akbarahmed.com/2012/06/22/install-java-jdk-6-0-update-26-on-ubuntu-12-04-lts/). Should fix the problem

      Reply

Leave a Reply