AkbarAhmed.com

Engineering Leadership

Introduction Start the HBase Shell All subsequent commands in this post assume that you are in the HBase shell, which is started via the command listed below. hbase shell You should see output similar to: 12/08/12 12:30:52 WARN conf.Configuration: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter ‘help<RETURN>’ for list of supported commands. Type …

Continue reading

Introduction I ran into an annoying error in HBase due to the localhost loopback. The solution was simple, but took some trial and error. Error I was following the HBase logs with the following command: tail -1000f /var/log/hbase/hbase-hbase-master-freshstart.log The following error kept poping up in the log file. org.apache.hadoop.hbase.master.AssignmentManager: Unable to find a viable location …

Continue reading

Introduction Installing Pig is drop dead simple. Installation sudo apt-get install pig Check the Pig version. pig –version Setup the Environment We’re going to set the environment variables system-wide for Pig programming. sudo vi /etc/environment Paste the following environment variables into the environment file. HADOOP_MAPRED_HOME=”/usr/lib/hadoop-mapreduce” PIG_CONF_DIR=”/etc/pig/conf” source /etc/environment That’s it. You can now start to …

Continue reading

Introduction hdfs dfsadmin -metasave provides additional information compared to hdfs dfsadmin -report. With hdfs dfsadmin -metasave provides information about blocks, including> blocks waiting for replication blocks currently being replication total number of blocks hdfs dfsadmin -metasave filename.txt Run the command with sudo -u hdfs prefixed to ensure you don’t get a permission denied error. CDH4 …

Continue reading

Introduction HBase is a tabular-oriented database that runs on top of HDFS. It is modeled on Google’s BigTable. In this post, I’m going to install HBase in Pseudo mode, so please use these instructions for setting up a developer’s workstation, not for a production cluster. When should you use HBase HBase should be used when …

Continue reading

Introduction Zookeeper provides cluster management for Hadoop. In this post, I’m going to install Zookeeper in Pseudo mode, so please use these instructions for setting up a developer’s workstation, not for a production cluster. Installation The zookeeper package should already be installed, but we’ll double check. sudo apt-get install zookeeper Next, we’ll install the Zookeeper …

Continue reading