Develop Reference

How to read JSON file in reactive way using spring webflux?

I am trying to read file from classpath in a reactive way using spring webflux. I am able to read the file. But I am not able to parse into an Foo object.
I am trying the following way, but not sure how to convert to an FOO class.
public Flux<Object> readFile() {
Flux<DataBuffer> readFile1 = DataBufferUtils.read("classpath:test.json", new DefaultDataBufferFactory(), 4096);
return new Jackson2JsonDecoder().decode(readFile1,
ResolvableType.forType(List.class,Foo.class), null, Collections.emptyMap());
}
Help appreciated.

I think you are doing it correctly but you unfortunately must cast the Object back into the correct type. This is safe to do because the JSON decoding will fail if it was unable to construct a list of Foo:
public Flux<Foo> readFile() {
ResolvableType type = ResolvableType.forType(List.class,Foo.class);
Flux<DataBuffer> data = DataBufferUtils.read("classpath:test.json", new DefaultDataBufferFactory(), 4096);
return new Jackson2JsonDecoder().decode(data, type, null, null)
.map(Foo.class::cast);
}

You can use jackson ObjectMapper:
ObjectMapper mapper = new ObjectMapper();
Student student = mapper.readValue(jsonString, Student.class);
And before that, you should read the file and use FileReader and readLines() to parse line by line.
[UPDATE]
Ok, for reading file, reactive means, reading file in a stream, and whenever a line is read, process this line. From this point, the BufferReader.readLines will be fine. But if you really want to use reactive way, you can use:
package com.test;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.stream.Stream;
public class TestReadFile {
public static void main(String args[]) {
String fileName = "c://lines.txt";
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
stream.forEach(parseLine);
} catch (IOException e) {
e.printStackTrace();
}
}
}

Map GeoJSON to Java with Jackson

I am trying to map a JSON file into Java objects using Jackson library. This json file is a multi-level file that can be found here:
https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.geojson
It is the list of earthquakes that happened in the last 30 days in the US.
Here is the structure of this son file: https://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php
Now, I wrote a Java program that is reading fields from this file, specifically I am trying to access the field which is under features -> properties -> place (e.g. from the original file "place":"17km NW of Pinnacles, CA")). When I get to the properties field I can read it as a LinkedHashMap, but the next level, so the keys and values of this LinkedHashMap are being read as Strings:
for example this is one of the values : {type=Point, coordinates=[-121.2743333, 36.6375, 8.61]}
I WANT TO READ THESE VALUES AS ANOTHER OBJECT (NOT STRING, MAP MAYBE?) SO I COULD EXTRACT FURTHER DATA FROM IT.
Here is my class:
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.Map;
import org.codehaus.jackson.annotate.JsonIgnoreProperties;
import org.codehaus.jackson.map.ObjectMapper;
#JsonIgnoreProperties(ignoreUnknown = true)
public class ReadJSONFile {
private StringBuffer stringBuffer = new StringBuffer();
public void convert_json_to_java() throws Exception {
String url = "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.geojson";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
while ((inputLine = in .readLine()) != null) {
stringBuffer.append(inputLine);
stringBuffer.append("\n");
} in.close();
}
#SuppressWarnings("unchecked")
public void map_to_object() throws Exception {
ObjectMapper om = new ObjectMapper();
//ignore fields that are not formatted properly
om.configure(org.codehaus.jackson.map.DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES, false);
Map<Object, Object> resultMap = om.readValue(stringBuffer.toString(), Map.class);
ArrayList<Object> featuresArrayList = (ArrayList<Object>) resultMap.get("features");
for(Object o : featuresArrayList) {
LinkedHashMap<Object, Object> propertiesMap = (LinkedHashMap<Object, Object>) o;
for(Map.Entry<Object, Object> entry : propertiesMap.entrySet()) {
//HERE IS THE PROBLEM, THE VALUES OF THIS MAP (SECOND OBJECT) IS BEING READ AS A STRING
//WHILE SOME VALUES ARE NOT A STRING:
//e.g. {type=Point, coordinates=[-121.2743333, 36.6375, 8.61]}
//AND I WANT TO READ IT AS A MAP OR ANY OTHER OBJECT THAT WOULD ALLOW ME TO ACCESS THE DATA
String propertiesMapValues = entry.getValue().toString();
}
}
}
}
Main.java
public class Main {
public static void main(String[] args) throws Exception {
ReadJSONFile rjf = new ReadJSONFile();
rjf.convert_json_to_java();
rjf.map_to_object();
}
}
Maven dependency: https://mvnrepository.com/artifact/org.codehaus.jackson/jackson-mapper-asl
When I try casting this last object to anything else than String, the program gives me exception (can't cast String to another object). Did I do something wrong? Can someone tell me what can I do to access those fields without modifying Strings (e.g. splitting them into arrays etc.)?

Actually your code works but it could be a bit simplified. The method convert_json_to_java is unnecessary, you can pass the URL directly to the ObjectMapper.
The values in the map are not read as Strings, but you are converting them to Strings by calling toString(), which is defined for all objects. Acctual types can be Map, List, String, Integer etc., depending on the JSON content. Working with a generic map is indeed a bit annoying, so I would suggest you converting values to structured objects. GeoJSON is an open standard, so there are open-source libraries facilitating using it, e.g. geojson-jackson.
You would need to add a maven dependency:
<dependency>
<groupId>de.grundid.opendatalab</groupId>
<artifactId>geojson-jackson</artifactId>
<version>1.8.1</version>
</dependency>
Then the program could look something like:
import org.geojson.*
// ...
public class ReadJSONFile {
ObjectMapper om = new ObjectMapper();
public void mapToObject(String url) throws Exception {
Map<String, Object> resultMap = om.readValue(new URL(url), new TypeReference<Map<String, Object>>() {});
List<Feature> features = om.convertValue(resultMap.get("features"), new TypeReference<List<Feature>>() {});
for(Feature f : features) {
// Write the feature to the console to see how it looks like
System.out.println(om.writeValueAsString(f));
// Extract properties
Map<String,Object> properties = f.getProperties();
// ....
// Extract geometry
GeoJsonObject geometry = f.getGeometry();
if(geometry instanceof Point) {
Point p = (Point) geometry;
// do something with the point
} else if(geometry instanceof LineString) {
LineString mls = (LineString) geometry;
// ...
} else if(geometry instanceof MultiLineString) {
MultiLineString mls = (MultiLineString) geometry;
// ...
} else if(geometry instanceof MultiPoint) {
MultiPoint mp = (MultiPoint) geometry;
// ...
} else if(geometry instanceof Polygon) {
Polygon pl = (Polygon) geometry;
// ...
} else if(geometry != null) {
throw new RuntimeException("Unhandled geometry type: " + geometry.getClass().getName());
}
}
}
public static void main(String[] args) throws Exception {
ReadJSONFile rjf = new ReadJSONFile();
rjf.mapToObject("https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.geojson");
}
}

How to create multiple files based on one Freemarker Template

I'm having a little bit trouble with freemarker right now. What I want to do basically in my template: iterate over a list of elements and create for each element a new file.
<#assign x=3>
<#list 1..x as i>
${i}
...create a new file with the output of this loop iteration...
</#list>
I did not find anything about this in the freemarker manual or google. Is there a way to do this?

You can implement this with a custom directive. See freemarker.template.TemplateDirectiveModel, and particularly TemplateDirectiveBody. Custom directives can specify the Writer used in their nested content. So you can do something like <#output file="...">...</#output>, where the nested content will be written into the Writer you have provided in your TemplateDirectiveModel implementation, which in this case should write into the file specified. (FMPP does this too: http://fmpp.sourceforge.net/qtour.html#sect4)

You cannot do this using only FreeMarker. Its idea is to produce the single output stream from your template. It doesn't even care whether you will save the result to file, pass directly to TCP socket, store in the memory as string or do anything else.
If you really want to achieve this, you have to handle file separation by yourself. For example, you can insert special line like:
<#assign x=3>
<#list 1..x as i>
${i}
%%%%File=output${i}.html
...
</#list>
After that you should post-process FreeMarker output by yourself looking for the lines started with %%%%File= and create a new file at this point.

As ddekany said, you can do that implementing a directive. I have coded a little example:
package spikes;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.util.HashMap;
import java.util.Map;
import freemarker.core.Environment;
import freemarker.template.Configuration;
import freemarker.template.SimpleScalar;
import freemarker.template.Template;
import freemarker.template.TemplateDirectiveBody;
import freemarker.template.TemplateDirectiveModel;
import freemarker.template.TemplateException;
import freemarker.template.TemplateModel;
import io.vertx.core.json.JsonArray;
import io.vertx.core.json.JsonObject;
class OutputDirective implements TemplateDirectiveModel {
#Override
public void execute(
Environment env,
#SuppressWarnings("rawtypes") Map params,
TemplateModel[] loopVars,
TemplateDirectiveBody body)
throws TemplateException, IOException {
SimpleScalar file = (SimpleScalar) params.get("file");
FileWriter fw = new FileWriter(new File(file.getAsString()));
body.render(fw);
fw.flush();
}
}
public class FreemarkerTest {
public static void main(String[] args) throws Exception {
Configuration cfg = new Configuration(Configuration.VERSION_2_3_0);
cfg.setDefaultEncoding("UTF-8");
JsonObject model = new JsonObject()
.put("entities", new JsonArray()
.add(new JsonObject()
.put("name", "Entity1"))
.add(new JsonObject()
.put("name", "Entity2")));
Template template = new Template("Test", "<#assign model = model?eval_json><#list model.entities as entity><#output file=entity.name + \".txt\">This is ${entity.name} entity\n</#output></#list>", cfg);
Map<String, Object> root = new HashMap<String, Object>();
root.put("output", new OutputDirective());
root.put("model", model.encode());
Writer out = new OutputStreamWriter(System.out);
template.process(root, out);
}
}
This will generate two files:
"Entity1.txt": This is Entity1 entity
"Entity2.txt": This is Entity2 entity
:-)

Java: CSV to XML using Jackson

I'm trying to update the following code to output a CSV file into XML. The code below does a brilliant job in converting the CSV to JSON using the headers in the CSV to make the required JSON.
But I want to output as XML instead, and I can't seem to find any good advice to complete this.
Would appreciate some pointers.
package reader;
import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;
public class JacksonPackage
{
public static void main(String[] args) throws Exception
{
File input = new File("c:\\temp\\data.csv");
File output = new File("c:\\temp\\data.json");
List<Map<?, ?>> data = readObjectsFromCsv(input);
writeAsJson(data, output);
}
public static List<Map<?, ?>> readObjectsFromCsv(File file)
throws IOException
{
CsvSchema bootstrap = CsvSchema.emptySchema().withHeader();
CsvMapper csvMapper = new CsvMapper();
MappingIterator<Map<?, ?>> mappingIterator = csvMapper
.reader(Map.class).with(bootstrap).readValues(file);
return mappingIterator.readAll();
}
public static void writeAsJson(List<Map<?, ?>> data, File file)
throws IOException
{
ObjectMapper mapper = new ObjectMapper();
mapper.writeValue(file, data);
}
}

Isn't it just a matter of replacing ObjectMapper with XmlMapper?

JSON decode in Java

I get some JSON code like this:
{"1":{"id":"1","Vorname":"x","Nachname":"y","MaleFemale":0,"interests":[]},
"2":{"id":"2","Vorname":"x","Nachname":"y","MaleFemale":1,"interests":[]},
...
from my PHP script. Could you tell me how to decode this format in Java?
I only get examples where you have to have to have a format like this:
{"contacts": [{"user.id":"1","Vorname":"x","Nachname":"y","MaleFemale":1},
{"user.id":"2","Vorname":"x1","Nachname":"y2","MaleFemale":0}]}
So the difference is that in the first given code there is no "main node". In the second given code there is one ("contacts"). Do I need this node? I try so much but i do not get how to work this out.
Thank you very much.

I thinks you should use jackson mapper. here is a link:
How to convert Java object to / from JSON (Jackson)

You can do this easily with Jackson java library. Here is an example code snippet.
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import java.io.IOException;
public class Test {
public static void main(String[] args) throws IOException {
ObjectMapper mapper = new ObjectMapper();
// Reading the string to a JSON object
JsonNode jsonObject = mapper.readTree("{\"contacts\": [{\"user.id\":\"1\",\"Vorname\":\"x\",\"Nachname\":\"y\",\"MaleFemale\":1},\n" +
" {\"user.id\":\"2\",\"Vorname\":\"x1\",\"Nachname\":\"y2\",\"MaleFemale\":0}]}");
//Some basic querying
JsonNode contacts = jsonObject.get("contacts");
if (contacts.isArray()){
ArrayNode contactsArray = (ArrayNode) contacts;
for (JsonNode contact : contactsArray) {
System.out.println(contact.get("user.id"));
}
}
}
}
You can download the Jackson library from here.