Posted by Marta on October 6, 2021 Viewed 379 times
Learn how to fix the error Hadoop command not found, so you can fix your Hadoop installation and start working on big data projects.
In this article you will learn how to fix the Hadoop command not found error, so you can fix your Hadoop installation and start working on challenging big data projects.
Hadoop is an open source software optimised for reliable and scalable distributed computing written in Java, and therefore to use Hadoop you will need java installed in your machine, and the JAVA_HOME environment variable. If you haven’t installed Java or download Hadoop, you can find some help here.
Let’s dive into your problem. You just downloaded Hadoop and unzip the file. Then you check if you installed Hadoop correctly by running the following command:
And instead of a version number, you got the following message:
-bash: hadoop: command not found
Let’s start with some background about how commands work. When you run a command in your terminal, you are actually executing a binary file. For example, when you execute command
cp, this command corresponds with binary files locate in the
How can your machine find those command files? Using the
PATH environment variable. Every time you type a command in your terminal, the kernel will look for the corresponding binary file in all folders included in the
Therefore if you like to use the Hadoop command, after downloading it , you will need to update the
PATH environment variable to include the Hadoop binaries, so the terminal knows where to find Hadoop.
To fix this problem you will need to create the
HADOOP_HOME environment variable, which will point to the directory where you uncompressed the Hadoop file. You should also include this folder in your PATH variable. See below the command you should execute:
>> export HADOOP_HOME=~/sw/hadoop-x.y.z >> export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
If you run these commands from your terminal, Hadoop will be available through the terminal. Unfortunately this change will last only during the terminal session.
As a result, next time you restart the laptop, or close the terminal and reopen it, you will need to run these commands again. How can you avoid this? Adding the above commands to the
.bash_profile file, located in your user home folder. The content of this file get executed every time you open the terminal.
Once you update the
.bash_profile file, make sure you close and reopen the terminal so your change get applied. Or run the following command to reload the
>> source ~/.bash_profile
Once you have corrected the
PATH environment variable, and run the command:
hadoop version from your terminal, you should use something similar to the message below:
Hadoop 2.10.0 Subversion ssh://git.corp.linkedin.com:29418/hadoop/hadoop.git -r e2f1f118e465e787d8567dfa6e2f3b72a0eb9194 Compiled by jhung on 2019-10-22T19:10Z Compiled with protoc 2.5.0 From source with checksum 7b2d8877c5ce8c9a2cca5c7e81aa4026 This command was run using /Users/martarey/dev/hadoop-2.10.0/share/hadoop/common/hadoop-common-2.10.0.jar
Here is the command you need to run from your terminal to visualise the value of the PATH environment variable:
>> echo $PATH
And the output should be similar to the following message, where the different folder paths are separated by a colon
To summarise, we have seen what the error command not found means, and how you can repair your Hadoop installation and fix this error
Hadoop command not found.
In case you would like to learn more about Hadoop and the Hadoop ecosystem and how to run it locally, please feel free to check this tutorial. You will find useful information about the several components that make up Hadoop like HDFS and MapReduce which makes the platform reliable and scalable.
Steady pace book with lots of worked examples. Starting with the basics, and moving to projects, data visualisation, and web applications
Unique lay-out and teaching programming style helping new concepts stick in your memory
Great guide for those who want to improve their skills when writing python code. Easy to understand. Many practical examples
Perfect Boook for anyone who has an alright knowledge of Java and wants to take it to the next level.
Excellent read for anyone who already know how to program and want to learn Best Practices
Perfect book for anyone transitioning into the mid/mid-senior developer level
Great book and probably the best way to practice for interview. Some really good information on how to perform an interview. Code Example in Java