I am reading JSON data from a file.
Sample Data
{"name":"user1","myparam0":false,"myparam1":"44750004-23df-4960-88be-ba0884291597","myparam2":"36A3BF29-23df-EE2A-76B9-19BC1C854BA7","myparam3":"http://www.seloger.com/","myparam4":"http://www.seloger.com/erreur-temporaire/husk-pie","ver":"4.0.0"}
{"name":"user1","myparam0":true,"myparam1":"44750004-8bff-4960-88be-ba0884291597","myparam2":"36A3BF29-88be-EE2A-76B9-19BC1C854BA7","myparam3":"","myparam4":"http://www.seloger.com/erreur-temporaire/binde","ver":"4.0.0"}
I have written a sample code to read from the file and converted data to JSON like this
DataStream<Object> input = env.readTextFile("file:///home/ravisankar/workspace/temporary/input.file")
.map((line) -> {
return JSON.parseFull(line);
});
Now I need to calculate how many myparam3 are empty in 15 seconds based on the name. and group by myparam4
Ex: {
"user1": {
"myparams3": 1,
"myparam4": {
"http://www.seloger.com/erreur-temporaire/binde": 1,
"http://www.seloger.com/erreur-temporaire/husk-pie": 1
}
}
}
Is it possible to extract such data like this from Flink?? I don't see any examples working on JSON using Java. Thanks for your time
You could parse your json string to object i.e. via jackson library and operate a stream of java objects as usual
you can use jackson to parse the json to an object then make a loop to count your elements
private ObjectMapper objectMapper = new ObjectMapper() ;
...
Object element = objectMapper.readValue( jsonString , Object.class );
or you can use a regex that matches "myparam3":"" and calculate tha matches
public static void main( String[] args ) throws IOException
{
String str = "{\"name\":\"user1\",\"myparam0\":false,\"myparam1\":\"44750004-23df-4960-88be-ba0884291597\",\"myparam2\":\"36A3BF29-23df-EE2A-76B9-19BC1C854BA7\",\"myparam3\":\"http://www.seloger.com/\",\"myparam4\":\"http://www.seloger.com/erreur-temporaire/husk-pie\",\"ver\":\"4.0.0\"}\r\n" +
"{\"name\":\"user1\",\"myparam0\":true,\"myparam1\":\"44750004-8bff-4960-88be-ba0884291597\",\"myparam2\":\"36A3BF29-88be-EE2A-76B9-19BC1C854BA7\",\"myparam3\":\"\",\"myparam3\":\"\",\"myparam3\":\"\"\"myparam4\":\"http://www.seloger.com/erreur-temporaire/binde\",\"ver\":\"4.0.0\"}";
Pattern pattern = Pattern.compile("\"myparam3\":\"\"");
Matcher matcher = pattern.matcher(str);
int count = 0;
while (matcher.find()) {
count++;
}
System.out.println("Matches found : " + count );
}
Related
I'm not sure what format this object is in but can I parse the following invalid JSON object to Java class Pojo? I tried doing it using Jackson but since it's invalid, I was wondering if pojo class would work?
{
name: (sindey, crosby)
game: "Hockey"
type: athlete
}
The file would have multiple objects of this format
Geesh, don't recognise this format! If you want to use Jackson you could pre-process you data to wrap the values... perhaps a regex to catpure the groups and wrap the values in quotes something like (name|type):\s(.+) => $1: "$2"
I was wondering if pojo class would work?
Sure, you could make that work with a simple parser; plenty of room for improvement, but something like this would do it:
Record record = null;
var records = new LinkedList<>();
// I'm using a reader as example, but just provide lines any way you like
while ((line = reader.readLine()) != null) {
line = line.trim();
// may need to skip empty lines if you have them in your file...
if (line.equals("{")) {
record = new Record();
records.add(record);
} else {
// may need substrings if your data contains ":"
var tokens = line.split(":");
var field = tokens[0];
var value = tokens[1];
if (field.equals("name")) {
// perhaps shuffle the format to something nicer here..
record.setName(value);
}
/// same for game and type fields...
}
}
This question already has an answer here:
json object convert to string java
(1 answer)
Closed 2 years ago.
Hi Guys I'm new to programming.
In my java code i have string like this.
String json ="{"name":"yashav"}";
Please help me out to print the values using pre-build java functions.
Expected output should be like below
name=yashav
First of all its not JSON.
If you want to work for actual JSON. There are many libraries which help you to transfer string to object.
GSON is one of those libraries. Use this to covert object then you can use keys to get values. Or you can iterate whole HashMap as per your requirements.
https://github.com/google/gson
{name:yashav} this is not a valid JSON format.
If you have {"name": "yashav"} you can use Jackson to parse JSON to java object.
class Person {
String name;
...
}
ObjectMapper mapper = new ObjectMapper();
Person person = mapper.readValue("{\"name\" : \"mkyong\"}", Person.class);
Forst of all, given String is NOT a json. It should be "{\"name\":\"yashav\"}". If you have a correct json string, you can use JacksonUtils.
Define a model:
class Url {
private String name;
public String getName() {
return name;
}
}
And parse the json string:
String json = "{\"name\":\"yashav\"}";
Url url = JsonUtils.readValue(json, Url.class);
System.out.format("name = %s", url.getName());
Another way is to use Regular Expression:
public static void main(String... args) {
String url = "{name:yashav}";
Pattern pattern = Pattern.compile("\\{(?<key>[^:]+):(?<value>[^\\}]+)\\}");
Matcher matcher = pattern.matcher(url);
if (matcher.matches())
System.out.format("%s = %s\n", matcher.group("key"), matcher.group("value"));
}
And finally, you can use plain old String operations:
public static void main(String... args) {
String url = "{name:yashav}";
int colon = url.indexOf(':');
String key = url.substring(1, colon);
String value = url.substring(colon + 1, url.length() - 1);
System.out.format("%s = %s\n", key, value);
}
I want to convert each integer/double value to String present in json request before storing in MongoDB database.
There can be multiple fields like amountValue in the json. I am looking for a generic way which can parse json with any number of such attributes value to string. My request will have around 200 fields.
ex: "amountValue": 200.00, to "amountValue": "200.00",
{
"templateName": "My DC Template 14",
"templateDetails": {
"beneficiaryName": "Snow2",
"dcOpenAmount": {
"amountValue": 200.00,
}
}
}
My mongoDB Document is of the form
#Document
public class TemplateDetails {
#Id
private long templateId;
private String templateName;
private Object templateDetail;
}
Because we are storing document in mongodb as an object(Which can accept any type of json request) we dont have field level control on it.
In my controller, converting the request object to json.
This is how I tried. But its not meeting my expectation. It is still keeping the amount value to its original double form.:
ObjectMapper mapper = new ObjectMapper();
try {
String json = mapper.writeValueAsString(templateRequestVO);
System.out.println("ResultingJSONstring = " + json);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
Output :
ResultingJSONstring = {"id":null,"userId":"FU.ZONKO","txnType":"LCI","accessIndicator":"Public","templateId":null,"templateName":"My DC Template 14","tags":null,"templateDetails":{"applicantDetail":{"applicantName":"Tom","applicantAddress":{"addressLine1":"Infosys, Phase 2","city":"PUNE","state":"MAHARASHTRA","country":"INDIA","zip":"40039"},"accountId":"Account1234","customerId":"JPMORGAN"},"beneficiaryName":"Snow2","dcOpenAmount":{"amountValue":200.0,"currency":"USD"}}}
Is there any way to accomplish the result ? Or anything which can help to store documents in mongodb with attribute type as String ?
You can use Json manipulation avaliable in "org.json.JSONObject" to convert Double value to Stirng .
If your Json structure won't change and will remain as said above , you can do the following.
import org.json.JSONObject;
public static void main(String args[]) {
String j = "{ \"templateName\": \"My DC Template 14\", \"templateDetails\": { \"beneficiaryName\": \"Snow2\", \"dcOpenAmount\": { \"amountValue\": 200.00 } } }";
JSONObject jo = new JSONObject(j);
jo.getJSONObject("templateDetails")
.getJSONObject("dcOpenAmount")
.put("amountValue", String.valueOf(jo.getJSONObject("templateDetails").getJSONObject("dcOpenAmount").getDouble("amountValue")));
System.out.println(jo.toString());
}
Following will be the output
{"templateDetails":{"dcOpenAmount":{"amountValue":"200.0"},"beneficiaryName":"Snow2"},"templateName":"My DC Template 14"}
I don't know for mongodb but for a json string you can replace them with a regex and the function replace like this :
public class Test {
public static void main(String[] args) {
String json = "{\"id\":null,\"userId\":\"FU.ZONKO\",\"txnType\":\"LCI\",\"accessIndicator\":\"Public\",\"templateId\":null,\"templateName\":\"My DC Template 14\",\"tags\":null,\"templateDetails\":{\"applicantDetail\":{\"applicantName\":\"Tom\",\"applicantAddress\":{\"addressLine1\":\"Infosys, Phase 2\",\"city\":\"PUNE\",\"state\":\"MAHARASHTRA\",\"country\":\"INDIA\",\"zip\":\"40039\"},\"accountId\":\"Account1234\",\"customerId\":\"JPMORGAN\"},\"beneficiaryName\":\"Snow2\",\"dcOpenAmount\":{\"amountValue\":200.0,\"currency\":\"USD\"}}}";
System.out.println(replaceNumberByStrings(json));
}
public static String replaceNumberByStrings(String str){
return str.replaceAll("(?<=:)\\d+(\\.\\d+)?(?=(,|}))","\"$0\"");
}
}
It will look for all fields with a numeric value in the json string and add quotes to the value. This way they will be interpreted as strings when the json willl be parsed.
It will not work if the value is in an array though, but in this case it should not be a problem.
In Python we can do this easily:
data = {'name':'Felix'}
s = 'Hello, %(name)s' % data
s
'Hello, Felix'
Is there a similar way in Java to implement the same thing?
PS:
Sorry for the unclear question. the use case is : we have a map which stores the key-values, the Template only need to specify a key in the map, then the value of the key will be in the place where the key is in the template.
AFAIK you can use String#format for this:
String name = "Felix";
String s = String.format("Hello, %s", name);
System.out.println(s);
This will print
Hello, Felix
More info about how to use the formatting of String#format can be found on java.util.Formatter syntax
You want String.format method.
String data = "Hello, %s";
String updated = String.format(data, "Felix");
If you want to replace only Strings with Strings then code from second part of my answer will be better
Java Formatter class doesn't support %(key)s form, but instead you can use %index$s where index is counted from 1 like in this example
System.out.format("%3$s, %2$s, %1s", "a", "b", "c");
// indexes 1 2 3
output:
c, b, a
So all you need to do is create some array that will contain values used in pattern and change key names to its corresponding indexes (increased by 1 since first index used by Formatter is written as 1$ not as 0$ like we would expect for arrays indexes).
Here is example of method that will do it for you
// I made this Pattern static and put it outside of method to compile it only once,
// also it will match every (xxx) that has % before it, but wont include %
static Pattern formatPattern = Pattern.compile("(?<=%)\\(([^)]+)\\)");
public static String format(String pattern, Map<String, ?> map) {
StringBuffer sb = new StringBuffer();
List<Object> valuesList = new ArrayList<>();
Matcher m = formatPattern.matcher(pattern);
while (m.find()) {
String key = m.group(1);//group 1 contains part inside parenthesis
Object value = map.get(key);
// If map doesn't contain key, value will be null.
// If you want to react somehow to null value like throw some
// Exception
// now is the good time.
if (valuesList.contains(value)) {
m.appendReplacement(sb, (valuesList.indexOf(value) + 1) + "\\$");
} else {
valuesList.add(value);
m.appendReplacement(sb, valuesList.size() + "\\$");
}
}
m.appendTail(sb);
return String.format(sb.toString(), valuesList.toArray());
}
usage
Map<String, Object> map = new HashMap<>();
map.put("name", "Felix");
map.put("age", 70);
String myPattern =
"Hi %(emptyKey)s! My name is %(name)s %(name)s and I am %(age)s years old";
System.out.println(format(myPattern, map));
output:
Hi null! My name is Felix Felix and I am 70 years old
As you can see you can use same key few times (in our case name) and if your map wont contain key used in your String pattern (like emptyKey) it will be replaced with null.
Above version was meant to let you set type of data like s d and so on, but if your data will always be replaced with Strings, then you can skip String.format(sb.toString(), valuesList.toArray()) and replace all your keys with values earlier.
Here is simpler version that will accept only map with <String,String> key-value relationship.
static Pattern stringsPattern = Pattern.compile("%\\(([^)]+)\\)s\\b");
public static String formatStrings(String pattern, Map<String, String> map) {
StringBuffer sb = new StringBuffer();
Matcher m = stringsPattern.matcher(pattern);
while (m.find()) {
// we can't use null as replacement so we need to convert it to String
// first. We can do it with String.valueOf method
m.appendReplacement(sb, String.valueOf(map.get(m.group(1))));
}
m.appendTail(sb);
return sb.toString();
}
Under this use case, you need a template engine like velocity or freemarker to use a Map-like data structure to render a string template, there is no builtin module in java to do that. like this(with velocity):
public static void main(String[] args) {
Context context = new VelocityContext();
context.put("appid", "9876543d1");
context.put("ds", "2013-09-11");
StringWriter sw = new StringWriter();
String template = "APPID is ${appid} and DS is ${ds}";
Velocity.evaluate(context, sw, "velocity", template);
System.out.println(sw.toString());
}
If you want more advanced techniques like i18n support, you can use the advanced Message Format features
ex:
in langage properties files you add the property 'template' wich is your message
template = At {2,time,short} on {2,date,long}, \
we detected {1,number,integer} spaceships on \
the planet {0}.
then you can format your valriables pass the arguments in an array:
Object[] messageArguments = {
"Mars",
new Integer(7),
new Date()
};
You call the formatter it this way:
MessageFormat formatter = new MessageFormat("");
formatter.setLocale(currentLocale);
formatter.applyPattern(messages.getString("template"));
String output = formatter.format(messageArguments);
the detailed example is here
http://docs.oracle.com/javase/tutorial/i18n/format/messageFormat.html
I use Jackson library to generate json string like this:
ObjectMapper objectMapper = new ObjectMapper();
String str = objectMapper.writeValueAsString(model);
and this snipped code for instance generate somtething like this:
{"x" : "This is x", "y" : "This is y"}
but I want to generate something like this:
{'x' : 'This is x', 'y' : 'This is y'}
I mean how can I change the double quote string with single quote string.I try to change code like this:
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true);
String str = objectMapper.writeValueAsString(model);
but this snipped code generate the first one.
and of course I can handle this problem with replace method but I want Jackson library do this for me.
How can I handle this problem?
objectMapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true); is about allowing single quotes in deserializing (JSON to Objects), not serializing (Objects to JSON) as you want.
In serializing, the issue seems to be with Jackson 1.X's default serializer. Below is the code that Jackson uses to write the String values. As you can see, the double quotes are hard coded and thus unchangeable through configuration:
#Override
public void writeString(String text)
throws IOException, JsonGenerationException
{
_verifyValueWrite("write text value");
if (text == null) {
_writeNull();
return;
}
if (_outputTail >= _outputEnd) {
_flushBuffer();
}
_outputBuffer[_outputTail++] = '"'; // <----------------- opening quote
_writeString(text); // <----------------- string actual value
// And finally, closing quotes
if (_outputTail >= _outputEnd) {
_flushBuffer();
}
_outputBuffer[_outputTail++] = '"'; // <----------------- closing quote
}
To achieve what you want, there are at least two options:
1: Replace the quotes using Regex:
This is a safe approach because Jackson gives the double quotes (") already escaped (\"). All you have to do is escape the single quotes and switch the " around properties names and values:
ObjectMapper objectMapper = new ObjectMapper();
String str = objectMapper.writeValueAsString(model);
System.out.println("Received.: "+str);
str = str.replaceAll("'", "\\\\'"); // escapes all ' (turns all ' into \')
str = str.replaceAll("(?<!\\\\)\"", "'"); // turns all "bla" into 'bla'
System.out.println("Converted: "+str);
Output:
Received.: {"x":"ab\"c","y":"x\"y'z","z":15,"b":true}
Converted: {'x':'ab\"c','y':'x\"y\'z','z':15,'b':true}
Or 2: User a custom JsonSerializer on every String field
Declare the custom serializer:
public class SingleQuoteStringSerializer extends JsonSerializer<String> {
#Override
public void serialize(String str, JsonGenerator jGen, SerializerProvider sP)
throws IOException, JsonProcessingException {
str = str.replace("'", "\\'"); // turns all ' into \'
jGen.writeRawValue("'" + str + "'"); // write surrounded by single quote
}
}
Use it in the fields you want to single quote:
public class MyModel {
#JsonSerialize(using = SingleQuoteStringSerializer.class)
private String x;
...
And proceed as usual (QUOTE_FIELD_NAMES == false is used to unquote the field names):
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(JsonGenerator.Feature.QUOTE_FIELD_NAMES, false);
String str = objectMapper.writeValueAsString(model);
System.out.println("Received.: "+str);
Output:
Received.: {x:'ab"c',y:'x"y\'z',z:15,b:true}
Note: Since you seem to be wanting to embed the JSON into another, this last approach may also require escaping the "s (see x:'ab"c').
Configure ObjectMapper in the following way:
final ObjectMapper mapper = new ObjectMapper();
mapper.configure(JsonGenerator.Feature.QUOTE_FIELD_NAMES, false);
mapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
//this may be what you need
mapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true);
Try looking into gson. It would look like this in gson.
House myHouse = new House();
Gson gson = new Gson();
String json = gson.toJson(myHouse);
Done...
http://code.google.com/p/google-gson/
As I said in the comment that's not valid JSON and it doesn't make any sense to escape it. You should handle it in a different way.
You should put that object inside a property.
I think you want to have something like
{"myString":"{\"fake json\":\"foo\"}"}
instead you should have:
{"myString":{"fake json":"foo"}}
That should be the proper way to handle this.