I'm struggling to get my simple Tomcat app to work.
I have csv file placed in src/main/resources/cities.csv
and in my test main method everything goes well. However when I refer to this using a method in servlet I get:
java.nio.file.NoSuchFileException
FileOperations:
public class FileOparations {
static private final Path path = Paths.get("src/main/resources/cities.csv");
public static List<City> getCitiesList() {
return getCitiesStream().collect(Collectors.toList());
}
public static Stream<City> getCitiesStream() {
try {
return Files.readAllLines(path)
.stream()
.skip(1)
.map((String line) -> {
return line.split("\",\"");
})
.map((String[] line) -> {
String[] newData = new String[line.length];
for (int i = 0; i < line.length; i++) {
newData[i] = line[i].replaceAll("\"", "");
}
return newData;
})
.map((String[] data) -> {
String name = data[0];
String nameAscii = data[1];
String gps = data[2] + ":" + data[3];
String country = data[4];
String adminName = data[7];
String capitol = data[8];
long population;
try {
population = Long.parseLong(data[9]);
} catch (NumberFormatException e) {
population = Integer.MIN_VALUE;
}
int id = Integer.parseInt(data[10]);
return new City(name, nameAscii, id, country, gps, capitol, population, adminName);
});
} catch (IOException e) {
e.printStackTrace();
}
return Stream.empty();
}
My code in servlet looks like this:
protected void doGet(HttpServletRequest request, HttpServletResponse response){
List<City> cities = FileOparations.getCitiesList();
request.setAttribute("cities", cities);
request.getRequestDispatcher("result.jsp").forward(request, response);
}
I'm surprised, because I'm not passing any URL through servlet, I want to call static method from Java. Method getCitiesList calls stream, mapping and returns ready to use list.
Try using getServletContext():
String relativePath = "/resources/cities.csv";
InputStream input = getServletContext().getResourceAsStream(relativeWebPath);
Relative path will be the path from the directory, expanded from the WAR file in your tomcat. So before building, it should be in src/main/webapp/
Related
#Test
public void testBatchFailClientBatchSyncCallIllegalArgumentExceptions() throws Exception {
Map<String, String> singletonMap = Collections.singletonMap(ACCEPT_STRING_ID, defaultLocalizationMap.get(ACCEPT_STRING_ID));
StringRequest[] requests = stringRequestFactory.createRequests(singletonMap);
when(lmsClient.batchSyncCall(requests)).thenThrow(new IllegalArgumentException());
List<Backend.Response> responses = callLms(new StringRequest[] {requests[0]});
Assert.assertNotNull(responses);
assertEquals(EntityDescriptors.ERROR_V1, responses.get(0).entityDescriptor());
assertEquals(Http.Status.SERVICE_UNAVAILABLE, responses.get(0).status());
}
#Test
public void testBatchFailClientBatchSyncCallIOException() throws Exception {
Map<String, String> singletonMap = Collections.singletonMap(ACCEPT_STRING_ID, defaultLocalizationMap.get(ACCEPT_STRING_ID));
StringRequest[] requests = stringRequestFactory.createRequests(singletonMap);
when(lmsClient.batchSyncCall(requests)).thenThrow(new IOException());
List<Backend.Response> responses = callLms(new StringRequest[] {requests[0]});
Assert.assertNotNull(responses);
assertEquals(EntityDescriptors.ERROR_V1, responses.get(0).entityDescriptor());
assertEquals(Http.Status.SERVICE_UNAVAILABLE, responses.get(0).status());
}
Source Code -
#Override
public List<Backend.Response> handleRequests(BackendRequestContext context, List<Backend.Request> requests, Metrics metrics) {
StringRequest[] stringRequests = new StringRequest[requests.size()];
final String language = context.locale().toLanguageTag().replace("-", "_");
for (int i = 0; i < requests.size(); i++) {
final Backend.Request request = requests.get(i);
final String id = request.requiredPathParam(STRING_ID_PATH_PARAM);
final Optional<String> marketplaceDisplayName = request.queryParam(MARKETPLACE_NAME_QUERY_PARAM);
final Optional<String> stage = request.queryParam(STAGE_QUERY_PARAM);
final StringRequest stringRequest = new StringRequest(id);
stringRequest.setLanguage(language);
marketplaceDisplayName.ifPresent(stringRequest::setMarketplaceName);
stage.map(Stage::getStage).ifPresent(stringRequest::setStage);
stringRequests[i] = stringRequest;
}
StringResultBatch batchResult = invokeBatchSync(stringRequests);
return IntStream.of(requests.size()).mapToObj(i -> {
final Backend.Request request = requests.get(i);
try {
return transform(request, batchResult.get(i), language);
} catch (IOException e) {
LOGGER.error("", e);
return Backend.Response.builder()
.withRequest(request)
.withEntityDescriptor(EntityDescriptors.ERROR_V1)
.withStatus(Http.Status.SERVICE_UNAVAILABLE)
.withBody(ErrorResponses.ServerError.serviceUnavailable(ErrorResponse.InternalInfo.builder()
.withMessage("Error retrieving ["
+ request.requiredPathParam(STRING_ID_PATH_PARAM)
+ "]")
.build())
.tokens())
.build();
}
}
).collect(Collectors.toList());
}
private StringResultBatch invokeBatchSync(StringRequest[] stringRequests) {
try {
// LMS Client has an async batch call,
// but it returns a proprietary class (StringResultBatchFuture) which eventually wraps a BSFFutureReply.
// Neither of which provide access to anything like a Java-standard Future.
return client.batchSyncCall(stringRequests);
} catch (IllegalArgumentException | IOException e) {
//
return null;
}
}
I have two test cases here for the source file. I'm getting the Error N/A. It says null pointer exception. Can someone please review this and help me with this. It will be really appreciated. Thank you in advance
P.S - The source file takes input request as string and performs string translation and returns us that string.
I'm trying to read CSV files from GTFS.zip with help of uniVocity-parsers and run into an issue that I can't figure out. For some reason it seems the first column of some CSV files won't be parsed correctly. For example in the "stops.txt" file that looks like this:
stop_id,stop_name,stop_lat,stop_lon,location_type,parent_station
"de:3811:30215:0:6","Freiburg Stübeweg","48.0248455941735","7.85563688037231","","Parent30215"
"de:8311:30054:0:1","Freiburg Schutternstraße","48.0236251356332","7.72434519425597","","Parent30054"
"de:8311:30054:0:2","Freiburg Schutternstraße","48.0235446600679","7.72438739944883","","Parent30054"
The "stop_id" field won't be parsed correctly will have the value "null"
This is the method I'm using to read the file:
public <T> List<T> readCSV(String path, String file, BeanListProcessor<T> processor) {
List<T> content = null;
try {
// Get zip file
ZipFile zip = new ZipFile(path);
// Get CSV file
ZipEntry entry = zip.getEntry(file);
InputStream in = zip.getInputStream(entry);
CsvParserSettings parserSettings = new CsvParserSettings();
parserSettings.setProcessor(processor);
parserSettings.setHeaderExtractionEnabled(true);
CsvParser parser = new CsvParser(parserSettings);
parser.parse(new InputStreamReader(in));
content = processor.getBeans();
zip.close();
return content;
} catch (Exception e) {
e.printStackTrace();
}
return content;
}
And this is how my Stop Class looks like:
public class Stop {
#Parsed
private String stop_id;
#Parsed
private String stop_name;
#Parsed
private String stop_lat;
#Parsed
private String stop_lon;
#Parsed
private String location_type;
#Parsed
private String parent_station;
public Stop() {
}
public Stop(String stop_id, String stop_name, String stop_lat, String stop_lon, String location_type,
String parent_station) {
this.stop_id = stop_id;
this.stop_name = stop_name;
this.stop_lat = stop_lat;
this.stop_lon = stop_lon;
this.location_type = location_type;
this.parent_station = parent_station;
}
// --------------------- Getter --------------------------------
public String getStop_id() {
return stop_id;
}
public String getStop_name() {
return stop_name;
}
public String getStop_lat() {
return stop_lat;
}
public String getStop_lon() {
return stop_lon;
}
public String getLocation_type() {
return location_type;
}
public String getParent_station() {
return parent_station;
}
// --------------------- Setter --------------------------------
public void setStop_id(String stop_id) {
this.stop_id = stop_id;
}
public void setStop_name(String stop_name) {
this.stop_name = stop_name;
}
public void setStop_lat(String stop_lat) {
this.stop_lat = stop_lat;
}
public void setStop_lon(String stop_lon) {
this.stop_lon = stop_lon;
}
public void setLocation_type(String location_type) {
this.location_type = location_type;
}
public void setParent_station(String parent_station) {
this.parent_station = parent_station;
}
#Override
public String toString() {
return "Stop [stop_id=" + stop_id + ", stop_name=" + stop_name + ", stop_lat=" + stop_lat + ", stop_lon="
+ stop_lon + ", location_type=" + location_type + ", parent_station=" + parent_station + "]";
}
}
If I call the method i get this output which is not correct:
PartialReading pr = new PartialReading();
List<Stop> stops = pr.readCSV("VAGFR.zip", "stops.txt", new BeanListProcessor<Stop>(Stop.class));
for (int i = 0; i < 4; i++) {
System.out.println(stops.get(i).toString());
}
Output:
Stop [stop_id=null, stop_name=Freiburg Stübeweg, stop_lat=48.0248455941735, stop_lon=7.85563688037231, location_type=null, parent_station=Parent30215]
Stop [stop_id=null, stop_name=Freiburg Schutternstraße, stop_lat=48.0236251356332, stop_lon=7.72434519425597, location_type=null, parent_station=Parent30054]
Stop [stop_id=null, stop_name=Freiburg Schutternstraße, stop_lat=48.0235446600679, stop_lon=7.72438739944883, location_type=null, parent_station=Parent30054]
Stop [stop_id=null, stop_name=Freiburg Waltershofen Ochsen, stop_lat=48.0220902613143, stop_lon=7.7205756507492, location_type=null, parent_station=Parent30055]
Does anyone know why this happens and how I can fix it? This also happens in the "routes.txt" and "trips.txt" files that I tested.
This is the GTFS file : http://stadtplan.freiburg.de/sld/VAGFR.zip
If you print the headers you will notice that the first column doesn't look right. That's because you are parsing a file encoded using UTF-8 with a BOM marker.
Basically the file starts with a few bytes indicating what is the encoding. Until version 2.5.*, the parser didn't handle that internally, and you had to skip these bytes to get the correct output:
//... your code here
ZipEntry entry = zip.getEntry(file);
InputStream in = zip.getInputStream(entry);
if(in.read() == 239 & in.read() == 187 & in.read() == 191){
System.out.println("UTF-8 with BOM, bytes discarded");
}
CsvParserSettings parserSettings = new CsvParserSettings();
//...rest of your code here
The above hack will work on any version before 2.5.*, but you could also use Commons-IO provides a BOMInputStream for convenience and a more clean handling of this sort of thing - it's just VERY slow.
Updating to a recent version should take care of it automatically.
Hope it helps.
Good day!
I have a method which returns me an array of report names
System.out.println(bc[i].getDefaultName().getValue()
i want to use array output in other class, how i need to linked method outpud in my array in other class?
Method is:
public class ReoprtSearch {
public void executeTasks() {
PropEnum props[] = new PropEnum[] { PropEnum.searchPath, PropEnum.defaultName};
BaseClass bc[] = null;
String searchPath = "//report";
//searchPath for folder - //folder, report - //report, folder and report - //folder | //report
try {
SearchPathMultipleObject spMulti = new SearchPathMultipleObject(searchPath);
bc = cmService.query(spMulti, props, new Sort[] {}, new QueryOptions());
} catch (Exception e) {
e.printStackTrace();
return;
}
if (bc != null) {
for (int i = 0; i < bc.length; i++) {
System.out.println(bc[i].getDefaultName().getValue();
}
}
}
}
array in what i want put the array looks like:
String [] folders =
my trying like:
ReoprtSearch search = new ReoprtSearch();
String [] folders = {search.executeTasks()};
Returns me an error: cannot convert from void to string
Give me an explanations to understand how i can related to method output from other class.
Thanks
The problem is that your executeTasks method doesn't actually return anything (which is why it's void), and just prints to stdout. Instead of printing, add the names to an array and then return it. Something like this:
public class ReoprtSearch {
public String[] executeTasks() {
PropEnum props[] = new PropEnum[] { PropEnum.searchPath, PropEnum.defaultName};
BaseClass bc[] = null;
String searchPath = "//report";
//searchPath for folder - //folder, report - //report, folder and report - //folder | //report
try {
SearchPathMultipleObject spMulti = new SearchPathMultipleObject(searchPath);
bc = cmService.query(spMulti, props, new Sort[] {}, new QueryOptions());
} catch (Exception e) {
e.printStackTrace();
return null;
}
if (bc != null) {
String results[] = new String[bc.length];
for (int i = 0; i < bc.length; i++) {
results[i] = bc[i].getDefaultName().getValue();
}
return results;
}
return null;
}
}
I'm reading from a file a list of paths. I want to save them in a built-in java structure that can erase the duplicates automatically. By duplicates I mean if I have /usr/bin and then I add /usr the bin folder has to be erased because is "contained" inside the usr folder. I read the file sequentially so I'd prefer to not have to check all data twice, if possible.
Example code:
UnknownType<Path> database;
BufferedReader reader = new BufferedReader(new FileReader(new File("db.txt")));
String line;
while ((line = reader.readLine()) != null) {
Path path = Paths.get(line).toRealPath();
database.add(path);
}
Example file:
/usr/bin
/usr
/dev
/dev/sda1
/dev/sda2
/home/user/Desktop/file.txt
/home/user/Documents/file2.txt
/home/user/Documents/file3.txt
Expected output:
data structure containing paths:
/usr
/dev
/home/user/Desktop/file.txt
/home/user/Documents/file2.txt
/home/user/Documents/file3.txt
A tree-based solution (probably more efficient):
class Database {
public void add(String p) {
root.add(Arrays.asList(p.split("\\\\|/")), 0);
}
public void addAll(Collection<? extends String> list) {
for (String p : list)
add(p);
}
public List<String> getPathsList() {
ArrayList<String> list = new ArrayList<>();
root.listPaths(list, "");
return list;
}
PathNode root = new PathNode("");
static class PathNode {
public final String name;
public Map<String, PathNode> children = new HashMap<>();
public PathNode(String name) {
this.name = name;
}
public boolean isLeaf() {
return children.size()==0;
}
public boolean isRoot() {
return name.isEmpty();
}
public void add(List<String> path, int i) {
String childName = path.get(i);
PathNode child = children.get(childName);
if (child != null) {
if (path.size()-i <= 1) child.children.clear();
else child.add(path, i+1);
} else if (!isLeaf() || isRoot()) {
PathNode node = this;
for (; i < path.size(); i++) {
String key = path.get(i);
node.children.put(key, node = new PathNode(key));
}
}
}
public void listPaths(ArrayList<String> list, String prefix) {
for (PathNode child : children.values()) {
if (child.isLeaf()) list.add(prefix+child.name);
else child.listPaths(list, prefix+child.name+File.separator);
}
}
}
}
Test to verify correctness: http://ideone.com/cvqEVT
This implementation will accept both Windows and Unix paths when running on any platform. The paths returned by Database.getPathsList() will still use the OS's file separator; you could change that by changing File.separator in Database.PathNode.listPaths (the last line of real code).
A simple solution:
class Database {
public void add(Path p) {
for (int i = 0; i < paths.size(); i++) {
Path p2 = paths.get(i);
if (p2.startsWith(p)) {
// replace with new path
paths.set(i, p);
return;
}
if (p.startsWith(p2)) {
// don't add this new one
return;
}
}
// else, add the new one
paths.add(p);
}
ArrayList<Path> paths = new ArrayList<>();
}
LinkedList implementation:
class Database {
public void add(Path p) {
for (ListIterator<Path> it = paths.listIterator(0); it.hasNext();) {
Path p2 = it.next();
if (p2.startsWith(p)) {
// replace with new path
it.set(p);
return;
}
if (p.startsWith(p2)) {
// don't add this new one
return;
}
}
// else, add the new one
paths.add(p);
}
LinkedList<Path> paths = new LinkedList<>();
}
static ArrayList<Path> paths = new ArrayList<Path>();
public static void main (String[]args) {
add(Paths.get("/usr/bin"));
add(Paths.get("/usr"));
add(Paths.get("/dev"));
add(Paths.get("/dev/sda"));
add(Paths.get("/home/user/Desktop/file.txt"));
System.out.println(paths.toString());
}
public static void add(Path path){
// get root
String firstDir = path.subpath(0, 1).toString();
// check all known paths
for (int q = 0; q < paths.size(); q++){
Path p = paths.get(q);
// get root of saved path
String pFirstDir = p.subpath(0, 1).toString();
// do they have the same root path
if (pFirstDir.equals(firstDir)){
// the new path needs to have less folders otherwise return
if (path.getNameCount()>p.getNameCount()){
return;
}
// set the new path and return
paths.set(q, path);
return;
}
}
// no paths found taht match so add
paths.add(path);
}
will print:
[\usr, \dev, \home\user\Desktop\file.txt]
Does anyone know where to find a little how to on using dbpedia spotlight in java or scala? Or could anyone explain how it's done? I can't find any information on this...
The DBpedia Spotlight wiki pages would be a good place to start.
And I believe the installation page has listed the most popular ways (using a jar, or set up a web service) to use the application.
It includes instructions on using the Java/Scala API with your own installation, or calling the Web Service.
There are some additional data needed to be downloaded to run your own server for full service, good time to make a coffee for yourself.
you need download dbpedia spotlight (jar file) after that u can use next two classes ( author pablomendes ) i only make some change .
public class db extends AnnotationClient {
//private final static String API_URL = "http://jodaiber.dyndns.org:2222/";
private static String API_URL = "http://spotlight.dbpedia.org:80/";
private static double CONFIDENCE = 0.0;
private static int SUPPORT = 0;
private static String powered_by ="non";
private static String spotter ="CoOccurrenceBasedSelector";//"LingPipeSpotter"=Annotate all spots
//AtLeastOneNounSelector"=No verbs and adjs.
//"CoOccurrenceBasedSelector" =No 'common words'
//"NESpotter"=Only Per.,Org.,Loc.
private static String disambiguator ="Default";//Default ;Occurrences=Occurrence-centric;Document=Document-centric
private static String showScores ="yes";
#SuppressWarnings("static-access")
public void configiration(double CONFIDENCE,int SUPPORT,
String powered_by,String spotter,String disambiguator,String showScores){
this.CONFIDENCE=CONFIDENCE;
this.SUPPORT=SUPPORT;
this.powered_by=powered_by;
this.spotter=spotter;
this.disambiguator=disambiguator;
this.showScores=showScores;
}
public List<DBpediaResource> extract(Text text) throws AnnotationException {
LOG.info("Querying API.");
String spotlightResponse;
try {
String Query=API_URL + "rest/annotate/?" +
"confidence=" + CONFIDENCE
+ "&support=" + SUPPORT
+ "&spotter=" + spotter
+ "&disambiguator=" + disambiguator
+ "&showScores=" + showScores
+ "&powered_by=" + powered_by
+ "&text=" + URLEncoder.encode(text.text(), "utf-8");
LOG.info(Query);
GetMethod getMethod = new GetMethod(Query);
getMethod.addRequestHeader(new Header("Accept", "application/json"));
spotlightResponse = request(getMethod);
} catch (UnsupportedEncodingException e) {
throw new AnnotationException("Could not encode text.", e);
}
assert spotlightResponse != null;
JSONObject resultJSON = null;
JSONArray entities = null;
try {
resultJSON = new JSONObject(spotlightResponse);
entities = resultJSON.getJSONArray("Resources");
} catch (JSONException e) {
//throw new AnnotationException("Received invalid response from DBpedia Spotlight API.");
}
LinkedList<DBpediaResource> resources = new LinkedList<DBpediaResource>();
if(entities!=null)
for(int i = 0; i < entities.length(); i++) {
try {
JSONObject entity = entities.getJSONObject(i);
resources.add(
new DBpediaResource(entity.getString("#URI"),
Integer.parseInt(entity.getString("#support"))));
} catch (JSONException e) {
LOG.error("JSON exception "+e);
}
}
return resources;
}
}
second class
/**
* #author pablomendes
*/
public abstract class AnnotationClient {
public Logger LOG = Logger.getLogger(this.getClass());
private List<String> RES = new ArrayList<String>();
// Create an instance of HttpClient.
private static HttpClient client = new HttpClient();
public List<String> getResu(){
return RES;
}
public String request(HttpMethod method) throws AnnotationException {
String response = null;
// Provide custom retry handler is necessary
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
try {
// Execute the method.
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
LOG.error("Method failed: " + method.getStatusLine());
}
// Read the response body.
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
} catch (HttpException e) {
LOG.error("Fatal protocol violation: " + e.getMessage());
throw new AnnotationException("Protocol error executing HTTP request.",e);
} catch (IOException e) {
LOG.error("Fatal transport error: " + e.getMessage());
LOG.error(method.getQueryString());
throw new AnnotationException("Transport error executing HTTP request.",e);
} finally {
// Release the connection.
method.releaseConnection();
}
return response;
}
protected static String readFileAsString(String filePath) throws java.io.IOException{
return readFileAsString(new File(filePath));
}
protected static String readFileAsString(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
#SuppressWarnings("resource")
BufferedInputStream f = new BufferedInputStream(new FileInputStream(file));
f.read(buffer);
return new String(buffer);
}
static abstract class LineParser {
public abstract String parse(String s) throws ParseException;
static class ManualDatasetLineParser extends LineParser {
public String parse(String s) throws ParseException {
return s.trim();
}
}
static class OccTSVLineParser extends LineParser {
public String parse(String s) throws ParseException {
String result = s;
try {
result = s.trim().split("\t")[3];
} catch (ArrayIndexOutOfBoundsException e) {
throw new ParseException(e.getMessage(), 3);
}
return result;
}
}
}
public void saveExtractedEntitiesSet(String Question, LineParser parser, int restartFrom) throws Exception {
String text = Question;
int i=0;
//int correct =0 ; int error = 0;int sum = 0;
for (String snippet: text.split("\n")) {
String s = parser.parse(snippet);
if (s!= null && !s.equals("")) {
i++;
if (i<restartFrom) continue;
List<DBpediaResource> entities = new ArrayList<DBpediaResource>();
try {
entities = extract(new Text(snippet.replaceAll("\\s+"," ")));
System.out.println(entities.get(0).getFullUri());
} catch (AnnotationException e) {
// error++;
LOG.error(e);
e.printStackTrace();
}
for (DBpediaResource e: entities) {
RES.add(e.uri());
}
}
}
}
public abstract List<DBpediaResource> extract(Text text) throws AnnotationException;
public void evaluate(String Question) throws Exception {
evaluateManual(Question,0);
}
public void evaluateManual(String Question, int restartFrom) throws Exception {
saveExtractedEntitiesSet(Question,new LineParser.ManualDatasetLineParser(), restartFrom);
}
}
main()
public static void main(String[] args) throws Exception {
String Question ="Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
System.out.println("resource : "+c.getResu());
}
I just add one little fix for your answer.
Your code is running, if you add the evaluate method call:
public static void main(String[] args) throws Exception {
String question = "Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
c.evaluate(question);
System.out.println("resource : "+c.getResu());
}
Lamine
In the request method of the second class (AnnotationClient) in Adel's answer, the author Pablo Mendes hasn't finished
TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
which is an annoying warning that needs to be removed by replacing
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
with
Reader in = new InputStreamReader(method.getResponseBodyAsStream(), "UTF-8");
StringWriter writer = new StringWriter();
org.apache.commons.io.IOUtils.copy(in, writer);
response = writer.toString();