Why I am getting nullpointerexception in ts.reset() line in InputFile class? If I use any inbuilt analyser like whitespaceanalyser, I don't get any exception. What is the problem here?
public class CourtesyTitleFilter extends TokenFilter
{
TokenStream input;
Map<String,String> courtesyTitleMap = new HashMap<String,String>();
private CharTermAttribute termAttr;
public CourtesyTitleFilter(TokenStream input) throws IOException
{
super(input);
termAttr = input.addAttribute(CharTermAttribute.class);
courtesyTitleMap.put("Dr", "doctor");
courtesyTitleMap.put("Mr", "mister");
courtesyTitleMap.put("Mrs", "miss");
}
#Override
public boolean incrementToken() throws IOException
{
if (!input.incrementToken())
return false;
String small = termAttr.toString();
if(courtesyTitleMap.containsKey(small)) {
termAttr.setEmpty().append(courtesyTitleMap.get(small));
System.out.print(courtesyTitleMap.get(small));
}
return true;
}
}
public class CourtesyTitleAnalyzer extends Analyzer
{
#Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader)
{
TokenStream filter = null;
Tokenizer whitespaceTokenizer = new WhitespaceTokenizer(reader);
try
{
filter = new CourtesyTitleFilter (whitespaceTokenizer);
}
catch(IOException e)
{
e.printStackTrace();
}
return new TokenStreamComponents(whitespaceTokenizer,filter);
}
}
public class InputFile
{
public static void main(String[] args) throws IOException, ParseException
{
TokenStream ts=null;
CourtesyTitleAnalyzer cta=new CourtesyTitleAnalyzer();
try
{
StringReader sb=new StringReader("Hello Mr Hari. Meet Dr Kalam and Mrs xyz");
ts = cta.tokenStream("field",sb);
OffsetAttribute offsetAtt = ts.addAttribute(OffsetAttribute.class);
CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class);
ts.reset();
while (ts.incrementToken())
{
String token = termAtt.toString();
System.out.println("[" + token + "]");
System.out.println("Token starting offset: " + offsetAtt.startOffset());
System.out.println(" Token ending offset: " + offsetAtt.endOffset());
System.out.println("");
}
ts.end();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
ts.close();
cta.close();
}
}
}
input is already defined in the TokenFilter abstract class. You are hiding it by declaring it in your implementation.
So, just delete the line TokenStream input; in your CourtesyTitleFilter.
Related
I have 3 Classes: Regulate, Luminosity, Test
From the class Regulate, I which to setting an attribute in the class Luminosity by invoking the method setAttribute
Then in class Test, I calling the method getAttribute.
The problem is, When I calling the method getAttribute, I find a different value that I set it.
This is the Class Luminosity
public class Luminosity{
public static int attribute;
public static int getAttribute(){
return attribute;
}
public static void setAttribute(int v) {
attribute=v;
try {
File fichier = new File("../../WorkspaceSCA/Lamp/value.txt");
PrintWriter pw = new PrintWriter(new FileWriter(fichier)) ;
String ch=Integer.toString(attribute);
pw.append(ch);
pw.println();
pw.close();
}catch (Exception e) {
e.printStackTrace();
}
}
}
the Regulate Code:
public class Regulate {
public static void main(String[] args) throws InterruptedException {
Luminosity.setSensedValue(50));
System.out.println("Value of Luminosity= "+ Luminosity.getSensedValue());
}
}
this shows me: Value of Luminosity= 50
Now, I want to recover this value from a different class(Test), like this:
public class Test {
public static void main(String[] args) throws InterruptedException {
System.out.println("Value = "+ Luminosity.getSensedValue());
this shows me: Value= 0
I want to recover the same value.
Thank's in advance
You are start two different classes in two different threads.
Of course Luminosity doesn't have previous value, it was setting in different JVM.
If you want to setup an attribute and transfer it between two threads you can place it in a text file.
public class Luminosity {
private static final String FILE_NAME = "attribute.txt";
private int attribute;
public void writeAttribute(int val) throws IOException {
try (FileWriter fileWriter = new FileWriter(FILE_NAME)) {
fileWriter.append("" + val);
fileWriter.flush();
}
attribute = val;
}
public int readAttribute() throws IOException {
StringBuilder sb = new StringBuilder();
try (FileReader fileReader = new FileReader(FILE_NAME)) {
int r;
while (true) {
char[] buffer = new char[100];
r = fileReader.read(buffer);
if (r == -1) break;
sb.append(new String(Arrays.copyOf(buffer, r)));
}
} catch (FileNotFoundException e) {
return 0;
}
if (sb.length() == 0) return 0;
return Integer.parseInt(sb.toString());
}
public static void main(String[] args) throws IOException {
Luminosity luminosity = new Luminosity();
System.out.println("attribute after start: " + luminosity.readAttribute());
luminosity.writeAttribute(50);
System.out.println("new attribute: " + luminosity.readAttribute());
}
}
What should be the value of or initialize InputStreamSupplier?
I was trying to zip all the files in a directory and that should be fast.
So multi threading is the option i'm going for.
public class ScatterSample {
ParallelScatterZipCreator scatterZipCreator = new ParallelScatterZipCreator();
ScatterZipOutputStream dirs = ScatterZipOutputStream.fileBased(File.createTempFile("scatter-dirs", "tmp"));
public ScatterSample() throws IOException {
}
public void addEntry(ZipArchiveEntry zipArchiveEntry, InputStreamSupplier streamSupplier) throws IOException {
if (zipArchiveEntry.isDirectory() && !zipArchiveEntry.isUnixSymlink())
dirs.addArchiveEntry(ZipArchiveEntryRequest.createZipArchiveEntryRequest(zipArchiveEntry, streamSupplier));
else
scatterZipCreator.addArchiveEntry( zipArchiveEntry, streamSupplier);
}
public void writeTo(ZipArchiveOutputStream zipArchiveOutputStream)
throws IOException, ExecutionException, InterruptedException {
dirs.writeTo(zipArchiveOutputStream);
dirs.close();
scatterZipCreator.writeTo(zipArchiveOutputStream);
}
}
FirstMain Class:
public class FirstMain {
public FirstMain() {
// TODO Auto-generated constructor stub
}
public static void compressFolder(String sourceFolder, String absoluteZipfilepath)
{
try
{
ScatterSample scatterSample=new ScatterSample();
File srcFolder = new File(sourceFolder);
if(srcFolder != null && srcFolder.isDirectory())
{
Iterator<File> i = FileUtils.iterateFiles(srcFolder, new String []{"pdf"}, true);
File zipFile = new File(absoluteZipfilepath);
OutputStream outputStream = new FileOutputStream(zipFile);
ZipArchiveOutputStream zipArchiveOutputStream= new ZipArchiveOutputStream(outputStream);
int srcFolderLength = srcFolder.getAbsolutePath().length() + 1; // +1 to remove the last file separator
while(i.hasNext())
{
File file = i.next();
String relativePath = file.getAbsolutePath().substring(srcFolderLength);
InputStreamSupplier streamSupplier=new InputStreamSupplier(){
#Override
public InputStream get() {
// TODO Auto-generated method stub
return null;
}
};
ZipArchiveEntry zipArchiveEntry = new ZipArchiveEntry(relativePath);
scatterSample.addEntry(zipArchiveEntry, streamSupplier);
}
scatterSample.writeTo(zipArchiveOutputStream);
}
}catch (Exception e) {
e.printStackTrace();
}
}
public static void main( String[] args )
{
compressFolder("C:\\Users\\akatm\\Desktop\\Stuff\\zipdata\\Newtry\\","C:/Users/akatm/Desktop/Stuff/Newtry.zip");
}
}
The get() method must return an InputStream to the file.
You could define an internal class as the following:
static class FileInputStreamSupplier implements InputStreamSupplier {
private Path sourceFile;
FileInputStreamSupplier(Path sourceFile) {
this.sourceFile = sourceFile;
}
#Override
public InputStream get() {
InputStream is = null;
try {
is = Files.newInputStream(sourceFile);
} catch (IOException e) {
e.printStackTrace();
}
return is;
}
}
That you could then invoke as:
scatterSample.addEntry(zipArchiveEntry, new FileInputStreamSupplier(file.toPath());
You need to set the compress method in the ZipEntry
ZipArchiveEntry zipArchiveEntry = new ZipArchiveEntry(relativePath);
zipArchiveEntry.setMethod(ZipArchiveEntry.STORED);
scatterSample.addEntry(zipArchiveEntry, streamSupplier);
if you don't set the compress method, the method throws an exception.
I want to use design pattern for this switch - case code.
I tried to use the command pattern, but I could not understand how(I was programming only 2 for months)
I wrote this program to learn how to better program.
My code:
public class Server {
private static final String READ_NEW_MESSAGES = "read new mes";
private static final String SEND_PRIVATE_MESSAGES = "send mes";
private static final String JOIN_CHAT = "find chat";
private static final String CREATE_CHAT = "chating";
private static final String QUIT = "quit";
private static final String EXIT = "exit";
private static final String REGISTRATION = "reg";
private static final String CREATE_PRIVATE_CHAT = "priv chat";
private static final String CONNECT_TO_PRIVATE_CHAT = "connect pm";
private static final String START_CHAT = "Start";
private Populator<PrivateMessage> privateMessagePopulator;
private Populator<Registration> registrationPopulator;
private Populator<Message> messagePopulator;
private Populator<PrivateChat> privateChatPopulator;
private Populator<Chat> publicChatPopulator;
private List<PrivateMessage> privateMessages;
private BufferedReader reader;
private String currentUser;
private Set<String> users;
private static Logger log = Logger.getLogger(Server.class.getName());
private List<Chat> chats;
private String password;
private Set<Registration> registration;
private List<PrivateChat> pmChat;
private String chatName;
public Server() {
server();
}
public void server() {
reader = new BufferedReader(new InputStreamReader(System.in));
privateMessages = new ArrayList<PrivateMessage>();
users = new HashSet<String>();
chats = new ArrayList<Chat>();
privateMessagePopulator = new PrivateMessagePopulator();
privateChatPopulator = new PrivateChatPopulator();
publicChatPopulator = new PublicChatPopulator();
messagePopulator = new MessagePopulator();
registrationPopulator = new RegistratorPopulator();
registration = new HashSet<Registration>();
pmChat = new ArrayList<PrivateChat>();
}
public void start() {
String decition = "";
while (true) {
try {
registrationOrLogin();
} catch (IOException e1) {
e1.printStackTrace();
}
while (decition != QUIT) {
System.out.println("Create a chat - chating");
System.out.println("Join the chat - find chat");
System.out.println("Send private message - send mes");
System.out.println("Read new messages - new mes");
System.out.println("Quit - quit");
System.out.println("Create private chat - priv chat");
System.out.println("Connect to private chat - connect pm");
try {
decition = reader.readLine();
switch (decition) {
case CREATE_PRIVATE_CHAT:
createPrivateChat();
break;
case CREATE_CHAT:
createChat();
break;
case JOIN_CHAT:
joinChat();
break;
case SEND_PRIVATE_MESSAGES:
sendPrivateMessage();
break;
case READ_NEW_MESSAGES:
showNewMessages();
break;
case QUIT:
logout();
break;
case REGISTRATION:
registration();
break;
case CONNECT_TO_PRIVATE_CHAT:
joinToPrivateChat();
break;
default:
break;
}
} catch (IOException e) {
log.warning("Error while reading decition from keyboard. "
+ e.getMessage());
}
}
}
}
private void sendPrivateMessage() throws IOException {
PrivateMessage privateMessage = privateMessagePopulator.populate();
privateMessage.setSenderName(currentUser);
privateMessages.add(privateMessage);
}
private void joinChat() throws IOException {
System.out.println("Exist public chat");
for (Chat chat : chats) {
System.out.println(chat.getChatName());
}
System.out.println("Enter the name of chat you wish to join");
chatName = reader.readLine();
for (Chat chat : chats) {
if (chatName.equals(chat.getChatName())) {
for (Message mes : chat.getMessages()) {
System.out.println(mes.getSenderName() + ": "
+ mes.getContent());
}
publicComunication(chat);
}
}
}
private boolean hasNewMessages() {
boolean result = false;
for (PrivateMessage privateMessage : privateMessages) {
if (currentUser.equals(privateMessage.getReceiverName())) {
result = true;
}
}
for (PrivateChat pm : pmChat) {
if (pm.getAddUserName().equals(currentUser)) {
result = true;
}
}
return result;
}
private void showNewMessages() {
if (hasNewMessages()) {
for (PrivateMessage privateMessage : privateMessages) {
if (currentUser.equals(privateMessage.getReceiverName())
&& MessageStatus.DIDNT_READ.equals(privateMessage
.getStatus())) {
System.out.println(privateMessage.getSenderName() + ": "
+ privateMessage.getContent());
}
privateMessage.setStatus(MessageStatus.ALREADY_READ);
}
}
if (hasNewMessages()) {
for (PrivateChat pm : pmChat) {
for (Message message : pm.getMessages()) {
if (pm.getAddUserName().equals(currentUser)) {
System.out.println(message.getSenderName() + ": "
+ message.getContent());
}
}
}
} else {
System.out.println("you don't have new message ");
}
}
private void registrationOrLogin() throws IOException {
String logOrReg;
System.out
.println("Hi,if you already have account - 1,\nIf you would like to register - 2");
logOrReg = reader.readLine();
if (logOrReg.equals("1")) {
login();
} else if (logOrReg.equals("2")) {
registration();
} else {
registrationOrLogin();
}
}
private boolean hasUser() {
boolean result = false;
for (Registration reg : registration) {
if (currentUser.equals(reg.getUserName())
&& password.equals(reg.getUserPassword())) {
result = true;
}
}
return result;
}
private void login() throws IOException {
System.out.println("Please,enter user name and password ");
currentUser = reader.readLine();
password = reader.readLine();
if (hasUser()) {
System.out.println("You already logged in system");
} else {
System.out.println("Wrong user name or password");
registrationOrLogin();
}
}
private void logout() throws IOException {
currentUser = null;
password = null;
registrationOrLogin();
}
private void createChat() throws IOException {
Chat chat = new Chat();
chat = publicChatPopulator.populate();
publicComunication(chat);
chats.add(chat);
}
private void joinToPrivateChat() throws IOException {
for (PrivateChat pm : pmChat) {
for (String user : pm.getUsers()) {
if (user.equals(currentUser)) {
System.out.println(pm.getChatName());
}
}
}
System.out.println("Enter the name of the chat you wish to join");
chatName = reader.readLine();
for (PrivateChat pm : pmChat) {
if (chatName.equals(pm.getChatName())) {
for (Message message : pm.getMessages()) {
System.out.println(message.getSenderName() + " "
+ message.getContent());
}
privateComunication(pm);
}
}
}
private void createPrivateChat() throws IOException {
PrivateChat privateChat = new PrivateChat();
Set<String> chatUsers = new HashSet<String>();
privateChat = privateChatPopulator.populate();
while (true) {
privateChat.setAddUserName(reader.readLine());
chatUsers.add(privateChat.getAddUserName());
privateChat.setUsers(chatUsers);
for (String user : users) {
if (user.equals(privateChat.getAddUserName())) {
System.out.println("you add too chat user - "
+ privateChat.getAddUserName());
}
}
if (privateChat.getAddUserName().equals(START_CHAT)) {
break;
}
}
privateComunication(privateChat);
pmChat.add(privateChat);
}
private void registration() throws IOException {
Registration reg = registrationPopulator.populate();
registration.add(reg);
currentUser = reg.getUserName();
users.add(reg.getUserName());
}
private void privateComunication(PrivateChat privateChat) {
while (true) {
Message message = messagePopulator.populate();
message.setSenderName(currentUser);
System.out.println(message.getSenderName());
System.out.println("\t" + message.getContent());
if (EXIT.equals(message.getContent())) {
break;
}
privateChat.setStatus(MessageStatus.DIDNT_READ);
privateChat.addMessage(message);
}
}
private void publicComunication(Chat chat) {
while (true) {
Message message = messagePopulator.populate();
message.setSenderName(currentUser);
System.out.println(message.getSenderName());
System.out.println("\t" + message.getContent());
if (EXIT.equals(message.getContent())) {
break;
}
chat.addMessage(message);
}
}
}
I never thought about improving the ugly process of creating shells; never used a switch, but a lot of if-elses, which is the same essentially.
package command.example;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.util.HashMap;
import java.util.Map;
import java.util.Scanner;
public class CommandExample implements ApplicationContext {
private final Writer writer = new BufferedWriter(new OutputStreamWriter(
System.out));
private boolean quit = false;
private Map<String, Command> commands = new HashMap<>();
{
commands.put("create", new CreateChat(this));
commands.put("join", new JoinChat(this));
commands.put("exit", new ExitCommand(this));
}
public static void main(String[] args) throws Exception {
CommandExample example = new CommandExample();
example.run();
}
public void run() throws IOException {
try (Scanner s = new Scanner(System.in)) {
writer.write("> ");
writer.flush();
while (!quit && s.hasNextLine()) {
String input = s.nextLine().trim();
// get or default is java8, alternatively you could check for null
Command command = commands.getOrDefault(input, new UnknownCommand(this, input));
command.execute();
if (!quit)
writer.write("> ");
writer.flush();
}
}
}
#Override
public void shutdown() {
quit = true;
}
#Override
public Writer getWriter() {
return writer;
}
static interface Command {
public void execute() throws IOException;
}
static abstract class AbstractCommand implements Command {
protected final ApplicationContext context;
protected final Writer writer;
public AbstractCommand(ApplicationContext context) {
this.context = context;
this.writer = context.getWriter();
}
}
static class CreateChat extends AbstractCommand {
public CreateChat(ApplicationContext context) {
super(context);
}
#Override
public void execute() throws IOException {
writer.write(String.format("Successfully created a chat!%n"));
writer.flush();
}
}
static class JoinChat extends AbstractCommand {
public JoinChat(ApplicationContext context) {
super(context);
}
#Override
public void execute() throws IOException {
writer.write(String.format("Successfully joined chat!%n"));
writer.flush();
}
}
static class UnknownCommand extends AbstractCommand {
private final String command;
public UnknownCommand(ApplicationContext context, String command) {
super(context);
this.command = command;
}
#Override
public void execute() throws IOException {
writer.write(String.format("'%s' is not a supported command!%n",
command));
writer.flush();
}
}
static class ExitCommand extends AbstractCommand {
public ExitCommand(ApplicationContext context) {
super(context);
}
#Override
public void execute() throws IOException {
writer.write(String.format("Application is shutting down!%n"));
writer.flush();
context.shutdown();
}
}
};
interface ApplicationContext {
public void shutdown();
public Writer getWriter();
}
Here you have a little start. The Command implementations should not read their input (due to separation of concern), they should specify what they want and some kind of reader should provide it to them (cf. the approach with Writer).
Furthermore I'm asking myself how one would design the QuitProgram command - without using System.exit(0).
I have started working with Apache Pig for one of our projects. I have to create a custom input format to load our data files. For this, I followed this example Hadoop:Custom Input format. I also created my custom RecordReader implementation to read the data (we get our data in binary format from some other application) and parse that to proper JSON format.
The problem occurs when I use my custom loader in Pig script. As soon as my loader's getNext() method is invoked, it calls my custom RecordReader's nextKeyValue() method, which works fine. It reads the data properly, passes it back to my loader which parses the data and returns a Tuple. So far so good.
The problem arises when my loader's getNext() method is called again and again. It gets called, works fine, and returns the proper output (I debugged it till return statement). But then, instead of letting the execution go further, my loader gets called again. I tried to see the number of times my loader is called, and I could see the number go till 20K!
Can somebody please help me understand the problem in my code?
Loader
public class SimpleTextLoaderCustomFormat extends LoadFunc {
protected RecordReader in = null;
private byte fieldDel = '\t';
private ArrayList<Object> mProtoTuple = null;
private TupleFactory mTupleFactory = TupleFactory.getInstance();
#Override
public Tuple getNext() throws IOException {
Tuple t = null;
try {
boolean notDone = in.nextKeyValue();
if (!notDone) {
return null;
}
String value = (String) in.getCurrentValue();
byte[] buf = value.getBytes();
int len = value.length();
int start = 0;
for (int i = 0; i < len; i++) {
if (buf[i] == fieldDel) {
readField(buf, start, i);
start = i + 1;
}
}
// pick up the last field
readField(buf, start, len);
t = mTupleFactory.newTupleNoCopy(mProtoTuple);
mProtoTuple = null;
} catch (InterruptedException e) {
int errCode = 6018;
String errMsg = "Error while reading input";
e.printStackTrace();
throw new ExecException(errMsg, errCode,
PigException.REMOTE_ENVIRONMENT, e);
}
return t;
}
private void readField(byte[] buf, int start, int end) {
if (mProtoTuple == null) {
mProtoTuple = new ArrayList<Object>();
}
if (start == end) {
// NULL value
mProtoTuple.add(null);
} else {
mProtoTuple.add(new DataByteArray(buf, start, end));
}
}
#Override
public InputFormat getInputFormat() throws IOException {
//return new TextInputFormat();
return new CustomStringInputFormat();
}
#Override
public void setLocation(String location, Job job) throws IOException {
FileInputFormat.setInputPaths(job, location);
}
#Override
public void prepareToRead(RecordReader reader, PigSplit split)
throws IOException {
in = reader;
}
Custom InputFormat
public class CustomStringInputFormat extends FileInputFormat<String, String> {
#Override
public RecordReader<String, String> createRecordReader(InputSplit arg0,
TaskAttemptContext arg1) throws IOException, InterruptedException {
return new CustomStringInputRecordReader();
}
}
Custom RecordReader
public class CustomStringInputRecordReader extends RecordReader<String, String> {
private String fileName = null;
private String data = null;
private Path file = null;
private Configuration jc = null;
private static int count = 0;
#Override
public void close() throws IOException {
// jc = null;
// file = null;
}
#Override
public String getCurrentKey() throws IOException, InterruptedException {
return fileName;
}
#Override
public String getCurrentValue() throws IOException, InterruptedException {
return data;
}
#Override
public float getProgress() throws IOException, InterruptedException {
return 0;
}
#Override
public void initialize(InputSplit genericSplit, TaskAttemptContext context)
throws IOException, InterruptedException {
FileSplit split = (FileSplit) genericSplit;
file = split.getPath();
jc = context.getConfiguration();
}
#Override
public boolean nextKeyValue() throws IOException, InterruptedException {
InputStream is = FileSystem.get(jc).open(file);
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
data = writer.toString();
fileName = file.getName();
writer.close();
is.close();
System.out.println("Count : " + ++count);
return true;
}
}
Try this in Loader
//....
boolean notDone = ((CustomStringInputFormat)in).nextKeyValue();
//...
Text value = new Text(((CustomStringInputFormat))in.getCurrentValue().toString())
Does anyone know where to find a little how to on using dbpedia spotlight in java or scala? Or could anyone explain how it's done? I can't find any information on this...
The DBpedia Spotlight wiki pages would be a good place to start.
And I believe the installation page has listed the most popular ways (using a jar, or set up a web service) to use the application.
It includes instructions on using the Java/Scala API with your own installation, or calling the Web Service.
There are some additional data needed to be downloaded to run your own server for full service, good time to make a coffee for yourself.
you need download dbpedia spotlight (jar file) after that u can use next two classes ( author pablomendes ) i only make some change .
public class db extends AnnotationClient {
//private final static String API_URL = "http://jodaiber.dyndns.org:2222/";
private static String API_URL = "http://spotlight.dbpedia.org:80/";
private static double CONFIDENCE = 0.0;
private static int SUPPORT = 0;
private static String powered_by ="non";
private static String spotter ="CoOccurrenceBasedSelector";//"LingPipeSpotter"=Annotate all spots
//AtLeastOneNounSelector"=No verbs and adjs.
//"CoOccurrenceBasedSelector" =No 'common words'
//"NESpotter"=Only Per.,Org.,Loc.
private static String disambiguator ="Default";//Default ;Occurrences=Occurrence-centric;Document=Document-centric
private static String showScores ="yes";
#SuppressWarnings("static-access")
public void configiration(double CONFIDENCE,int SUPPORT,
String powered_by,String spotter,String disambiguator,String showScores){
this.CONFIDENCE=CONFIDENCE;
this.SUPPORT=SUPPORT;
this.powered_by=powered_by;
this.spotter=spotter;
this.disambiguator=disambiguator;
this.showScores=showScores;
}
public List<DBpediaResource> extract(Text text) throws AnnotationException {
LOG.info("Querying API.");
String spotlightResponse;
try {
String Query=API_URL + "rest/annotate/?" +
"confidence=" + CONFIDENCE
+ "&support=" + SUPPORT
+ "&spotter=" + spotter
+ "&disambiguator=" + disambiguator
+ "&showScores=" + showScores
+ "&powered_by=" + powered_by
+ "&text=" + URLEncoder.encode(text.text(), "utf-8");
LOG.info(Query);
GetMethod getMethod = new GetMethod(Query);
getMethod.addRequestHeader(new Header("Accept", "application/json"));
spotlightResponse = request(getMethod);
} catch (UnsupportedEncodingException e) {
throw new AnnotationException("Could not encode text.", e);
}
assert spotlightResponse != null;
JSONObject resultJSON = null;
JSONArray entities = null;
try {
resultJSON = new JSONObject(spotlightResponse);
entities = resultJSON.getJSONArray("Resources");
} catch (JSONException e) {
//throw new AnnotationException("Received invalid response from DBpedia Spotlight API.");
}
LinkedList<DBpediaResource> resources = new LinkedList<DBpediaResource>();
if(entities!=null)
for(int i = 0; i < entities.length(); i++) {
try {
JSONObject entity = entities.getJSONObject(i);
resources.add(
new DBpediaResource(entity.getString("#URI"),
Integer.parseInt(entity.getString("#support"))));
} catch (JSONException e) {
LOG.error("JSON exception "+e);
}
}
return resources;
}
}
second class
/**
* #author pablomendes
*/
public abstract class AnnotationClient {
public Logger LOG = Logger.getLogger(this.getClass());
private List<String> RES = new ArrayList<String>();
// Create an instance of HttpClient.
private static HttpClient client = new HttpClient();
public List<String> getResu(){
return RES;
}
public String request(HttpMethod method) throws AnnotationException {
String response = null;
// Provide custom retry handler is necessary
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
try {
// Execute the method.
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
LOG.error("Method failed: " + method.getStatusLine());
}
// Read the response body.
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
} catch (HttpException e) {
LOG.error("Fatal protocol violation: " + e.getMessage());
throw new AnnotationException("Protocol error executing HTTP request.",e);
} catch (IOException e) {
LOG.error("Fatal transport error: " + e.getMessage());
LOG.error(method.getQueryString());
throw new AnnotationException("Transport error executing HTTP request.",e);
} finally {
// Release the connection.
method.releaseConnection();
}
return response;
}
protected static String readFileAsString(String filePath) throws java.io.IOException{
return readFileAsString(new File(filePath));
}
protected static String readFileAsString(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
#SuppressWarnings("resource")
BufferedInputStream f = new BufferedInputStream(new FileInputStream(file));
f.read(buffer);
return new String(buffer);
}
static abstract class LineParser {
public abstract String parse(String s) throws ParseException;
static class ManualDatasetLineParser extends LineParser {
public String parse(String s) throws ParseException {
return s.trim();
}
}
static class OccTSVLineParser extends LineParser {
public String parse(String s) throws ParseException {
String result = s;
try {
result = s.trim().split("\t")[3];
} catch (ArrayIndexOutOfBoundsException e) {
throw new ParseException(e.getMessage(), 3);
}
return result;
}
}
}
public void saveExtractedEntitiesSet(String Question, LineParser parser, int restartFrom) throws Exception {
String text = Question;
int i=0;
//int correct =0 ; int error = 0;int sum = 0;
for (String snippet: text.split("\n")) {
String s = parser.parse(snippet);
if (s!= null && !s.equals("")) {
i++;
if (i<restartFrom) continue;
List<DBpediaResource> entities = new ArrayList<DBpediaResource>();
try {
entities = extract(new Text(snippet.replaceAll("\\s+"," ")));
System.out.println(entities.get(0).getFullUri());
} catch (AnnotationException e) {
// error++;
LOG.error(e);
e.printStackTrace();
}
for (DBpediaResource e: entities) {
RES.add(e.uri());
}
}
}
}
public abstract List<DBpediaResource> extract(Text text) throws AnnotationException;
public void evaluate(String Question) throws Exception {
evaluateManual(Question,0);
}
public void evaluateManual(String Question, int restartFrom) throws Exception {
saveExtractedEntitiesSet(Question,new LineParser.ManualDatasetLineParser(), restartFrom);
}
}
main()
public static void main(String[] args) throws Exception {
String Question ="Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
System.out.println("resource : "+c.getResu());
}
I just add one little fix for your answer.
Your code is running, if you add the evaluate method call:
public static void main(String[] args) throws Exception {
String question = "Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
c.evaluate(question);
System.out.println("resource : "+c.getResu());
}
Lamine
In the request method of the second class (AnnotationClient) in Adel's answer, the author Pablo Mendes hasn't finished
TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
which is an annoying warning that needs to be removed by replacing
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
with
Reader in = new InputStreamReader(method.getResponseBodyAsStream(), "UTF-8");
StringWriter writer = new StringWriter();
org.apache.commons.io.IOUtils.copy(in, writer);
response = writer.toString();