63251

Hadoop jar execution failing on class not found

Question:

I am running my hadoop job and it is failing on class not found. 4 java files in total.

logProcessor.java logMapper.java logReducer.java logParser.java

Everything is in a com folder on unix and I have "package com;" in the first line in all classes

that means if you do below command head -5 *java You will see package com; in all 4 files.

logProcessor is the Driver class. All files are in "com" folder on unix.

ls -ltr com/ logProcessor.java logMapper.java logReducer.java logParser.java I compiled the java program and made a jar out of it. hadoop jar /var/lib/hadoop-hdfs/xxxx/jarFiles/LogParser.jar com.LogProcessor /user/hdfs/flume/2015-03-30/03 /user/xxxx/output_xxx

It gives me below error:

Exception in thread "main" java.lang.ClassNotFoundException: com.RFCLogProcessor at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:270) at org.apache.hadoop.util.RunJar.main(RunJar.java:201)

Answer1:

First, you need to write your jar path in classpath. write directory path of your jar file into .bashrc file.

Then you can using below command:

hadoop jar directorypath/yourjarname.jar packagename.mainclassname outputpath

Answer2:

Everything that I was doing was right. Except that the classes which got created didnt have execute permission on them

So I did

chmod -R 777 com/

and jar'd it again and ran it with same above command and it executed.

Recommend

  • Unreadable var_dump in Firebug when xdebug is enabled
  • What are valid values for CSS Property “padding-right-ltr-source”?
  • Hebrew and normal text not displaying right
  • Respond.js not working in Internet Explorer 7, but H5BP example works
  • Find literals in aspx page
  • Text direction with print or println in Swift for mixed text
  • Nested file input doesn't work in Firefox
  • load css file conditionally
  • Forcing RTL order in a JTextArea
  • Google Analytics API example JAVA
  • How to validate form field in PHP using Object Oriented Technique
  • How do i make Python count how many letters are in a word? [closed]
  • RTL language behavior in iOS 9
  • Hadoop tasktracker issue
  • 'include' of functions in groovy scripts
  • integrate POCO library in android ndk
  • Spark (Scala) Writing (and reading) to local file system from driver
  • “mvn clean generate-source” could not resolve dependencies
  • In Java, how can I construct a File from a resource?
  • Using $compile in a directive triggers AngularJS infinite digest error
  • pip in virtualenv gets ConnectTimeoutError
  • OSX - always hide certain files
  • How to get latest version of a artifact on Bintray using JSONP
  • Android Studio Can't Find tools.jar
  • Memory error in python- how to use more memory
  • During installation of Django, why do I keep getting ImportError: No module named django?
  • Differences in dis-assembled C code of GCC and Borland?
  • pyodbc doesn't report sql server error
  • Stop Bash Script if Hive Fails
  • error importing numpy
  • x64 applications using gdi+: what are the consequences on performance?
  • Very simple C++ DLL that can be called from .net
  • Algorithm for a smudge tool?
  • Why is the size of this struct 32?
  • 'TypeError' while using NSGA2 to solve Multi-objective prob. from pyopt-sparse in OpenMDAO
  • How to delete a row from a dynamic generate table using jquery?
  • using HTMLImports.whenReady not working in chrome
  • Authorize attributes not working in MVC 4
  • EntityFramework adding new object to nested object collection
  • Checking variable from a different class in C#