27400

Could not find or load main class org.apache.hadoop.fs.FsShell

I understand this question might have been answered already, well, my issue is still here:

I have a vm created for hadoop on vmware using CentOS7, I can start namenode and datanode, however, when I tried to view hdfs file using the following command:

hdfs dfs -ls

it throws out an error below:

Could not find or load main class org.apache.hadoop.fs.FsShell

My google searchings suggest this might relate to hadoop variables setting in bash, here is my settings:

# .bashrc # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi export HADOOP_HOME=/opt/hadoop/hadoop-2.7.2 export HADOOP_MAPRED_HOME=$HADOOP_HOME export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export HADOOP_YARN_HOME=$HADOOP_HOME export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop export HADOOP_PREFIX=$HADOOP_HOME export HIVE_HOME=/opt/hadoop/hive export PATH=$HIVE_HOME/bin:$PATH export ANT_HOME=/usr/local/apache-ant-1.9.7 export PATH=${PATH}:${JAVA_HOME}/bin export PIG_HOME=/opt/hadoop/pig-0.15.0 export PIG_HADOOP_VERSION=0.15.0 export PIG_CLASSPATH=$HADOOP_HOME/etc/hadoop export PATH=$PATH:$PIG_HOME/bin export PATH=$PATH:$HADOOP_HOME/bin export HADOOP_USER_CLASSPATH_FIRST=true export SQOOP_HOME=/usr/lib/sqoop export PATH=$PATH:$SQOOP_HOME/bin export HADOOP_CLASSPATH=$HADOOP_HOME/share/hadoop/common/ export PATH=$PATH:$HADOOP_CLASSPATH # Uncomment the following line if you don't like systemctl's auto-paging feature : # export SYSTEMD_PAGER= # User specific aliases and functions

I checked my hadoop folder: /opt/hadoop/hadoop-2.7.2/share/hadoop/common, here is the list: <img src=https://www.e-learn.cn/content/wangluowenzhang/"https://i.stack.imgur.com/bJObX.jpg" alt="enter image description here">

I am doing this practice using root account, can anyone help to find out where is the cause of this issue and fix it? Thank you very much.

Answer1:

this typically happens when you have multiple instances of hadoop, check which hadoop and see if its pointing out to the version that you have installed.

say if it points to /usr/bin/hadoop and not /your-path/hadoop, then you can point /usr/bin/hadoop to that (with symlink)

Answer2:

As Filmon Gebreyesus pointed out this can happen when you have multiple hadoop instances. First, check what you have in $PATH. There should be paths to the hadoop/bin .

If it still not working run whereis hdfs . Check the output. If there is an hdfs which should not be there, remove/move it.

Answer3:

Try to unset $HADOOP_COMMON_HOME

unset HADOOP_COMMON_HOME

Recommend

  • hadoop in windows with cygwin noclassdefinition found error
  • Moving HDFS data into MongoDB
  • Trouble running pig in both local or mapreduce mode
  • hadoop namenode is not formatting
  • nodeJS util.format passing an array
  • parallelize process in missForest package
  • Python object matching using string
  • Deploying pre-encrypted configuration files to a production environment
  • setSelected() with JRadioButton r[]=new JRadioButton[3] not working [duplicate]
  • Instanciate service on startup in Angular2
  • javascript variables, What does var x = a = {} do?
  • get passwords from chrome
  • Using an enum contained in a Cloud Endpoint model on a Android client
  • Java Garbage collection, setting reference to null
  • rapply over a nested list in R
  • Python adding lots of things to PATH. How do I stop?
  • Powershell Hash Table to HTML
  • Ionic storage “get” returns null only on the second call within method
  • Cannot find “Grammar.txt” in python-sphinx
  • Randomizing -and remembering that randomisation- multiple choice questions in php
  • Open Existing DB in MySQL WorkBench
  • Updating Dojo provide
  • How do I retrieve the user information of a user authenticated with Apache's mod_ldap?
  • pip in virtualenv gets ConnectTimeoutError
  • Defined variables not working in javascript files when I use getScript
  • How to specify input and output paths from cmd.exe for a PowerShell script?
  • ThreadStatic in asynchronous ASP.NET Web API
  • During installation of Django, why do I keep getting ImportError: No module named django?
  • How can I display the parent menu item's description using Wordpress walkers?
  • gspread or such: help me get cell coordinates (not value)
  • Azure Cloud Service Web Role web pages do not load
  • Counter field in MS Access, how to generate?
  • How to extract text from Word files using C#?
  • Javascript + PHP Encryption with pidCrypt
  • Websockets service method fails during R startup
  • Why can't I rebase on to an ancestor of source changesets if on a different branch?
  • Are Kotlin's Float, Int etc optimised to built-in types in the JVM? [duplicate]
  • Does armcc optimizes non-volatile variables with -O0?
  • Observable and ngFor in Angular 2
  • Conditional In-Line CSS for IE and Others?