AkbarAhmed.com

Engineering Leadership

The following is a repost of my answer to a question on LinkedIn, but I thought it may prove useful to people evaluating Hadoop distributions. The following is a substantially over simplified set of choices (in alphabetical order): Amazon: Apache Hadoop provided as a web service. Good solution if your data is collected on Amazon…saves …

Continue reading

Introduction Start the HBase Shell All subsequent commands in this post assume that you are in the HBase shell, which is started via the command listed below. hbase shell You should see output similar to: 12/08/12 12:30:52 WARN conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter ‘help<RETURN>’ for list of supported commands. Type …

Continue reading

Introduction I ran into an annoying error in HBase due to the localhost loopback. The solution was simple, but took some trial and error. Error I was following the HBase logs with the following command: tail -1000f /var/log/hbase/hbase-hbase-master-freshstart.log The following error kept poping up in the log file. org.apache.hadoop.hbase.master.AssignmentManager: Unable to find a viable location …

Continue reading

Introduction Installing Pig is drop dead simple. Installation sudo apt-get install pig Check the Pig version. pig –version Setup the Environment We’re going to set the environment variables system-wide for Pig programming. sudo vi /etc/environment Paste the following environment variables into the environment file. HADOOP_MAPRED_HOME=”/usr/lib/hadoop-mapreduce” PIG_CONF_DIR=”/etc/pig/conf” source /etc/environment That’s it. You can now start to …

Continue reading