Strange error in mapreduce class - java

this error seems trivial, but it won't go away. I have the following class defined:
import java.io.IOException;
import java.util.Iterator;
import java.util.StringTokenizer;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reducer;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapred.TextInputFormat;
import org.apache.hadoop.mapred.TextOutputFormat;
import org.apache.hadoop.mapreduce.Mapper;
public class Anagram_Mapper extends Mapper<LongWritable, Text, Text, Text> {
in the 'main' function i am trying to use JobConf to launch a simple mapreduce:
public static void main(String args[]){
JobConf conf = new JobConf(Anagram_Mapper.class);
conf.setJobName("anagram_mapper");
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(Anagram_Mapper.class);
conf.setCombinerClass(Reduce.class);
conf.setReducerClass(Reduce.class);
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
FileInputFormat.setInputPaths(conf, new Path(args[0]));
FileOutputFormat.setOutputPath(conf, new Path(args[1]));
try {
JobClient.runJob(conf);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Eclipse is throwing an error on this line:
conf.setMapperClass(Anagram_Mapper.class);
the error is:
The method setMapperClass(Class<? extends Mapper>) in the type JobConf
is not applicable for the arguments (Class<Anagram_Mapper>)
but, as you can see above, my Anagram_Mapper class extends Mapper, right? so, i don't understand why this error....
EDIT: someone posted here, then retracted their post, but it helped steer me in the right direction. apparently i am using:
org.apache.hadoop.mapreduce.Mapper
but JobConf.setMapperClass accepts only:
org.apache.hadoop.mapred.Mapper
now i'm a little confused about the difference, they seem to be fundamentally the same, and the API tells me they are both valid in Hadoop 2.2.0, the version i'm using....

Indeed you are mixing the old mapred API with the new mapreduce API.
Basically Hadoop mapreduce supports two incompatibles APIs and you have to decide which one to use. I can be confusing because they share classes with same or similar names. You should carefully look at your import statements.
Both API can achieve almost the same thing. mapred has not been deprecated nor removed to no break legacy applications. mapreduce is the same API with a slightly better design.
If you are starting a new project, I would advise to use the new one. But it is not a big deal. The easy fix is to change your org.apache.hadoop.mapreduce.Mapper import statement.

Faced same error after writing Driver class, below is the error
The method setReducerClass(Class) in the type Job is not applicable for the arguments (Class)
reason of getting this error : after creation of reducer class i have immediately passed the class name in setReducerClass(); without defining the reducer class.
The function is expecting a class name that actually extends Reducer, it will throw same error until the passed argument is as per the method expected argument type.

Related

Class imported in the same package is not being recognized

I have these two java files: Xls_Reader.java and WriteXMLFile.java in the same directory called filegeneration. I am trying to use Xls_Reader class in WriteXMLFile.java so I did this
package fileGeneration;
import fileGeneration.Xls_Reader;
However I keep getting this error when trying to compile
WriteXMLFile.java:27: error: cannot find symbol
import fileGeneration.Xls_Reader;
symbol: class Xls_Reader
location: class WriteXMLFile
WriteXMLFile.java:65: error: cannot find symbol
Xls_Reader xlsDataSheet = new Xls_Reader(dataSheetPath);
Why is Xls_Reader not being found?
Update: the following is in Xls_Reader
package fileGeneration;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Calendar;
import org.apache.poi.ss.usermodel.DateUtil;
import org.apache.poi.hssf.usermodel.HSSFDateUtil;
//import org.apache.poi.openxml4j.util.ZipSecureFile;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.xssf.usermodel.XSSFCell;
import org.apache.poi.xssf.usermodel.XSSFRow;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
public class Xls_Reader {
In order to let the compiler knows about the classes to be compiled, you need to specify the classes explicitly. The compiler won't use the same folder to know that. You can read more about how ClassPath works here (https://docs.oracle.com/javase/tutorial/essential/environment/paths.html)
An example of compiling multiple files from the command line here:
https://www.baeldung.com/java-compile-multiple-files
I see that you are using another libraries like Apache POI, therefore it would be more convenient to use Maven or Gradle to handle both dependencies and deployments.

How to resolve class conflict in import statements

Im trying to write a program that will read an xml file and print to itext.
However, I getting a class conflict in import statements. I dont know how to resolve this issue.
import org.dom4j.Document;
import org.dom4j.DocumentException;
import org.dom4j.Element;
import org.dom4j.Node;
import org.dom4j.io.SAXReader;
import org.w3c.dom.NodeList;
import org.w3c.dom.*;
import com.itextpdf.text.Annotation;
import com.itextpdf.text.BaseColor;
import com.itextpdf.text.Chunk;
import com.itextpdf.text.Document;
import com.itextpdf.text.DocumentException;
import com.itextpdf.text.Element;
import com.itextpdf.text.Font;
import com.itextpdf.text.FontFactory;
import com.itextpdf.text.PageSize;
import com.itextpdf.text.Paragraph;
import com.itextpdf.text.Phrase;
import com.itextpdf.text.pdf.PdfAction;
import com.itextpdf.text.pdf.PdfPCell;
import com.itextpdf.text.pdf.PdfPTable;
import com.itextpdf.text.pdf.PdfWriter;
The "Document" and "Element" classes between dom4j and itext are conflicting.
Does anyone know a workaround? Is there any eclipse magic I can do?
The workaround here is to only import one of the two (or many) conflicting classes. Then use the fully qualified class name for the class not imported. For example:
import org.dom4j.Document;
// import com.itextpdf.text.Document; <-- don't import this
Then when using the itext class, refer to it as:
com.itextpdf.text.Document doc;
I would recommed importing the class/package of classes which you use the most in your Java file. This would let you avoid having to type a fully qualified class name as much as possible.
In such cases , you should use fully qualified class name for one of the conflicting classes while using it.
You need to specify the package when using them in your code.
Otherwise there is an ambiguity as Document could refer to either org.dom4j.Document or com.itextpdf.text.Document and Element could refer to org.dom4j.Element or com.itextpdf.text.Element.
Import statements in Java act like aliases for fully qualified type names. Trying to use the same alias for two or more types won't work because of ambiguity.
Therefore do either not use import statements for any of those conflicting types and use their fully qualified names (package plus type name) or import at most one of those types only.

How to define a NotesXspDocument Object in JAVA

In a JAVA bean that I am working on I want to pass a NotesXspDocument (could use a NotesDocument) to a method which looks like this:
public List<String> getReaders(NotesXspDocument thisXspDoc){
// do some stuff
}
But JAVA does not recognize the NotesXspDocument definition. I have imported the following packages:
import lotus.domino.NotesException;
import lotus.domino.Session;
import lotus.domino.Database;
import lotus.domino.View;
import lotus.domino.Document;
Is there a further package to import to make use the NotesXspDocument?
To elaborate on Jesses answer: in your case you need to do this to work with the XPages version of Document:
import com.ibm.xsp.model.domino.wrapped.DominoDocument;
public List<String> getReaders(DominoDocument thisXspDoc){
// do some stuff
}
NotesXspDocument is an SSJS-only alias; the real class is com.ibm.xsp.model.domino.wrapped.DominoDocument: http://public.dhe.ibm.com/software/dw/lotus/Domino-Designer/JavaDocs/DesignerAPIs/com/ibm/xsp/model/domino/wrapped/DominoDocument.html

LWJGL/opengl in eclipse

I cannot seem to import the opengl properly.
I am following this simple tutorial:
https://www.youtube.com/watch?v=ZKJC2cloIqc
As far as I understand, I have the correct jar and native files.
// I can import these
import org.lwjgl.opengl.Display;
import org.lwjgl.opengl.DisplayMode;
//I cannot however import this:
import org.lwjgl.LWJGLEXCEPTION;
//and gl methods such as this are not recognized:
glClear(GL_COLOR_BUFFER_BIT);
I am using eclipse, I also have netbeans and am debating getting the intelij IDE being used in this tutorial if it will make this work.
You didn't Import GL11 to use gl methods (or other versions for certain methods )
You can use a static import so you don't have to put GL11 in front of every method or anything else

Simple Script With Interactive Brokers Java API

I am new to java though I have some experience with R.
I have taken part of a java course and have read a book or two as well as the API guides published by interactive brokers. This API is higher level than anything I have worked with before, obviously.
The very first thing I want to do is simply connect to the software. I have been able to do this with the test GUI that Interactive Brokers provides. However, when writing my own script, I am getting an error: Uncompilable source code - Erroneous sym type. I have imported the javaclient/com directory into my new project.
The line that is causing the error is econnect(port=7496, clientid=0);
Reading the documentation, this should work, but obviously does not.
Below is the full code. All of the import calls were copied from a sample file that IB provided. onHowToDetermineStock() is copied from another part of the documentation. Before I can do anything, I obviously need to to connect.
Any ideas?
Thank you.
package ibapp;
import java.awt.BorderLayout;
import java.awt.Dimension;
import java.awt.Rectangle;
import java.util.ArrayList;
import javax.swing.Box;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextArea;
import javax.swing.JTextField;
import javax.swing.SwingUtilities;
import javax.swing.border.EmptyBorder;
import com.ib.controller.ApiConnection.ILogger;
import com.ib.controller.ApiController;
import com.ib.controller.ApiController.IBulletinHandler;
import com.ib.controller.ApiController.IConnectionHandler;
import com.ib.controller.ApiController.ITimeHandler;
import com.ib.controller.Formats;
import com.ib.controller.Types.NewsType;
import com.ib.client.EClientSocket;
/**
*
* #author
*/
void onHowToDetermineStock(){
Contract contract = new Contract();
Order order = new Order();
contract.m_symbol = "IBKR";
contract.m_secType = "STK";
contract.m_exchange = "SMART";
contract.m_currency = "USD";
order.m_action = "BUY";
order.m_totalQuantity = 100;
order.m_orderType = "LMT";
order.m_lmtPrice = enteredLmtPrice;
m_client.placeOrder(GlobalOrderId, contract, order);
}
public class IBApp {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
econnect(port=7496, clientid=0);
onHowToDetermineStock();
}
}
There are a number of problems with your code that make it an invalid Java program.
In Java, all methods must be contained within a class, unlike your onHowToDetermineStock method. Also, unlike R, Java doesn't use named parameters (i.e. port=7496 is not valid except to assign a variable named port). There are other problems.
Java is an object-oriented language and is very different from R. I would suggest forgetting the IB API for the time being, and spending some time learning how to code a basic Java application. There are many free tutorials on the web.
E.g.: https://docs.oracle.com/javase/tutorial/

Categories

Resources