Most probably this issue is because of JSONObject(org.json.JSONObject) is incompatible with cloudant library.
Is any alternative way to use any other Object?
I am using below cloudant libraries,
<dependency>
<groupId>com.cloudant</groupId>
<artifactId>cloudant-client</artifactId>
<version>2.6.2</version>
</dependency>
Here is my code
package data.repositories;
import org.json.JSONObject;
import com.cloudant.client.api.*;
import com.cloudant.client.api.CloudantClient;
import com.cloudant.client.api.Database;
import com.cloudant.client.api.model.Response;
import util.Config;
public class DatabaseRepository {
CloudantClient client = ClientBuilder.account(Config.CLOUDANT_ACCOUNT_NAME)
.username(Config.CLOUDANT_USER_NAME)
.password(Config.CLOUDANT_PASSWORD).build();
public DatabaseRepository() {
JSONObject
}
public void Save(String dbName) {
Database db = client.database("dbTempName", true);
JSONObject jsonObject = new JSONObject("{hello: data}");
db.save(jsonObject);
}
}
Document saved in cloudant database is,
{
"_id": "1c7f223f74a54e7c9f4c8a713feaa537",
"_rev": "1-a3cd12379eec936b61f899c8278c9d62",
"map": {
"hello": "data"
}
}
I'm not familiar with cloudant but my guess is JsonObject has a property called "map" that holds your json string data (probably there's a myArray property too), and cloudant serializes it into json, thus adding those unnecessary values.
my suggestions:
1) try to save your json string directly like db.save("{hello: data}") to avoid serialization
2) if you really need to create a JsonObject try to customize cloudant's serialization process to avoid that extra fields.
in response to comment:
from what I read here, then I think you need a pojo, which when serialized into json would look like:
{ 'hello' : 'data' }
which is something like:
public class MyClass implements Serializable {
String hello;
public MyClass(String hello) {
this.hello = hello;
}
public String getHello() {
return hello;
}
}
then save it like:
db.save(new MyClass("data"));
or you can use a hashmap instead of a pojo:
Map<String, Object> map = new Hashmap ...
map.put("hello", "data");
db.save(map);
Look at the example in the README for the repo. It shows that you want a POJO, but you don't have to implement Serializable. Just create a class that has _id and _rev properties that are Strings. Then add Javascript object compatible properties as desired.
// A Java type that can be serialized to JSON
public class ExampleDocument {
private String _id = "example_id";
private String _rev = null;
private boolean isExample;
public ExampleDocument(boolean isExample) {
this.isExample = isExample;
}
public String toString() {
return "{ id: " + _id + ",\nrev: " + _rev + ",\nisExample: " + isExample + "\n}";
}
}
// Create an ExampleDocument and save it in the database
db.save(new ExampleDocument(true));
Although I haven't tried it, the Hashmap approach may work also, as discussed in this tutorial: https://www.ibm.com/blogs/bluemix/2014/07/cloudant_on_bluemix/.
// create a simple doc to place into your new database
Map<String, Object> doc = new HashMap<String, Object>();
doc.put("_id", UUID.randomUUID().toString());
doc.put("season", "summer");
doc.put("climate", "arid");
dbc.create(doc);
In question It seems org.json.JSONObject used And it is not compatible with cloudant client library. I tried with google object it is working good for me.
Issue got resolved by using google com.google.gson.JsonObject instead of org.json.JSONObject.
Correct Full code is given below,
Database db = client.database("dbTempName", true);
// Used google.gson.JsonObject instead of org.json.JSONObject.
com.google.gson.JsonParser parser = new com.google.gson.JsonParser();
com.google.gson.JsonObject jsonObject = parser.parse("{\"hello\": \"data\"}").getAsJsonObject();
db.save(jsonObject);
Related
I want to make a POST request with URL Query Params set to the values of an object.
For example
http://test/data?a=1&b=2&c=3
I want to make a post request to this URL with a class like this:
public class Data {
private Integer a;
private Integer b;
private Integer c;
}
I do NOT want to do each field manually, like this:
public void sendRequest(Data data) {
String url = UriComponentsBuilder.fromHttpUrl("http://test/")
.queryParam("a", data.getA())
.queryParam("b", data.getB())
.queryParam("c", data.getC())
.toUriString();
restTemplate.postForObject(url, body, Void.class);
}
Instead, I want to use the entire object:
public void sendRequest(Data data) {
String url = UriComponentsBuilder.fromHttpUrl("http://test/")
.queryParamsAll(data) //pseudo
.toUriString();
restTemplate.postForObject(url, body, Void.class);
}
Your requirement is like QS in js. Thx qianshui423/qs . It is implementation QS in java. It is coded by a Chinese guy. At first git clone it and use below cmd to build. You will get a jar called "qs-1.0.0.jar" in build/libs (JDK required version 8)
# cd qs directory
./gradlew build -x test
Import it, I do a simple demo as below. For your requirement, you can build class to transfer your Obj into QSObject. Besides toQString, QS can parse string to QSObject. I think it powerful.
import com.qs.core.QS;
import com.qs.core.model.QSObject;
public class Demo {
public static void main(String[] args) throws Exception{
QSObject qsobj = new QSObject();
qsobj.put("a",1);
qsobj.put("b",2);
qsobj.put("c",3);
String str = QS.toQString(qsobj);
System.out.println(str); // output is a=1&b=2&c=3
}
}
I am trying to deserialize some Kafka messages that were serialized by Nifi, using Hortonworks Schema Registry
Processor used on the Nifi Side as RecordWritter: AvroRecordSetWriter
Schema write strategy: HWX COntent-Encoded Schema Reference
I am able to deserialize these messsages in other Nifi kafka consumer. However I am trying to deserialize them from my Flink application using Kafka code.
I have the following inside the Kafka deserializer Handler of my Flink Application:
final String SCHEMA_REGISTRY_CACHE_SIZE_KEY = SchemaRegistryClient.Configuration.CLASSLOADER_CACHE_SIZE.name();
final String SCHEMA_REGISTRY_CACHE_EXPIRY_INTERVAL_SECS_KEY = SchemaRegistryClient.Configuration.CLASSLOADER_CACHE_EXPIRY_INTERVAL_SECS.name();
final String SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_SIZE_KEY = SchemaRegistryClient.Configuration.SCHEMA_VERSION_CACHE_SIZE.name();
final String SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_EXPIRY_INTERVAL_SECS_KEY = SchemaRegistryClient.Configuration.SCHEMA_VERSION_CACHE_EXPIRY_INTERVAL_SECS.name();
final String SCHEMA_REGISTRY_URL_KEY = SchemaRegistryClient.Configuration.SCHEMA_REGISTRY_URL.name();
Properties schemaRegistryProperties = new Properties();
schemaRegistryProperties.put(SCHEMA_REGISTRY_CACHE_SIZE_KEY, 10L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_CACHE_EXPIRY_INTERVAL_SECS_KEY, 5000L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_SIZE_KEY, 1000L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_EXPIRY_INTERVAL_SECS_KEY, 60 * 60 * 1000L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_URL_KEY, "http://schema_registry_server:7788/api/v1");
return (Map<String, Object>) HWXSchemaRegistry.getInstance(schemaRegistryProperties).deserialize(message);
And here is the HWXSchemaRegistryCode to deserialize the message:
import com.hortonworks.registries.schemaregistry.avro.AvroSchemaProvider;
import com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient;
import com.hortonworks.registries.schemaregistry.errors.SchemaNotFoundException;
import com.hortonworks.registries.schemaregistry.serdes.avro.AvroSnapshotDeserializer;
public class HWXSchemaRegistry {
private SchemaRegistryClient client;
private Map<String,Object> config;
private AvroSnapshotDeserializer deserializer;
private static HWXSchemaRegistry hwxSRInstance = null;
public static HWXSchemaRegistry getInstance(Properties schemaRegistryConfig) {
if(hwxSRInstance == null)
hwxSRInstance = new HWXSchemaRegistry(schemaRegistryConfig);
return hwxSRInstance;
}
public Object deserialize(byte[] message) throws IOException {
Object o = hwxSRInstance.deserializer.deserialize(new ByteArrayInputStream(message), null);
return o;
}
private static Map<String,Object> properties2Map(Properties config) {
Enumeration<Object> keys = config.keys();
Map<String, Object> configMap = new HashMap<String,Object>();
while (keys.hasMoreElements()) {
Object key = (Object) keys.nextElement();
configMap.put(key.toString(), config.get(key));
}
return configMap;
}
private HWXSchemaRegistry(Properties schemaRegistryConfig) {
_log.debug("Init SchemaRegistry Client");
this.config = HWXSchemaRegistry.properties2Map(schemaRegistryConfig);
this.client = new SchemaRegistryClient(this.config);
this.deserializer = this.client.getDefaultDeserializer(AvroSchemaProvider.TYPE);
this.deserializer.init(this.config);
}
}
But I am getting a 404 HTTP Error code(schema not found). I think this is due to incompatible "protocols" between Nifi configuration and HWX Schema Registry Client implementation, so schema identifier bytes that the client is looking for does not exist on the server, or something like this.
Can someone help on this?
Thank you.
Caused by: javax.ws.rs.NotFoundException: HTTP 404 Not Found
at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:1069)
at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:866)
at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$1(JerseyInvocation.java:750)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:748)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:404)
at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:300)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$14.run(SchemaRegistryClient.java:1054)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$14.run(SchemaRegistryClient.java:1051)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getEntities(SchemaRegistryClient.java:1051)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getAllVersions(SchemaRegistryClient.java:872)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getAllVersions(SchemaRegistryClient.java:676)
at HWXSchemaRegistry.(HWXSchemaRegistry.java:56)
at HWXSchemaRegistry.getInstance(HWXSchemaRegistry.java:26)
at SchemaService.deserialize(SchemaService.java:70)
at SchemaService.deserialize(SchemaService.java:26)
at org.apache.flink.streaming.connectors.kafka.internals.KafkaDeserializationSchemaWrapper.deserialize(KafkaDeserializationSchemaWrapper.java:45)
at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.runFetchLoop(KafkaFetcher.java:140)
at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:712)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:93)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:302)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
at java.lang.Thread.run(Thread.java:745)
I found a workaround. Since I wasn't able to get this working. I take the first bytes of the byte array to make several calls to schema registry and get the avro schema to deserialize later the rest of the byte array.
First byte (0) is protocol version (I figured out this is a Nifi-specific byte, since I didn't need it).
Next 8 bytes are the schema Id
Next 4 bytes are the schema version
The rest of the bytes are the message itself:
import com.hortonworks.registries.schemaregistry.SchemaMetadataInfo;
import com.hortonworks.registries.schemaregistry.SchemaVersionInfo;
import com.hortonworks.registries.schemaregistry.SchemaVersionKey;
import com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient;
try(SchemaRegistryClient client = new SchemaRegistryClient(this.schemaRegistryConfig)) {
try {
Long schemaId = ByteBuffer.wrap(Arrays.copyOfRange(message, 1, 9)).getLong();
Integer schemaVersion = ByteBuffer.wrap(Arrays.copyOfRange(message, 9, 13)).getInt();
SchemaMetadataInfo schemaInfo = client.getSchemaMetadataInfo(schemaId);
String schemaName = schemaInfo.getSchemaMetadata().getName();
SchemaVersionInfo schemaVersionInfo = client.getSchemaVersionInfo(
new SchemaVersionKey(schemaName, schemaVersion));
String avroSchema = schemaVersionInfo.getSchemaText();
byte[] message= Arrays.copyOfRange(message, 13, message.length);
// Deserialize [...]
}
catch (Exception e)
{
throw new IOException(e.getMessage());
}
}
I also thought that maybe I had to remove the first byte before calling the hwxSRInstance.deserializer.deserialize in my question code, since this byte seems to be a Nifi specific byte to communicate between Nifi processors, but it didn't work.
Next step is to build a cache with the schema texts to avoid calling multiple times the schema registry API.
New info: I will extend my answer to include the avro deserialization part, since it was some troubleshooting for me and I had to inspect Nifi Avro Reader source code to figure out this part (I was getting not valid Avro data exception when trying to use the basic avro deserialization code):
import org.apache.avro.Schema;
import org.apache.avro.file.SeekableByteArrayInput;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.io.BinaryDecoder;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DecoderFactory;
private static GenericRecord deserializeMessage(byte[] message, String schemaText) throws IOException {
InputStream in = new SeekableByteArrayInput(message);
Schema schema = new Schema.Parser().parse(schemaText);
DatumReader<GenericRecord> datumReader = new GenericDatumReader<>(schema);
BinaryDecoder decoder = DecoderFactory.get().binaryDecoder(in, null);
GenericRecord genericRecord = null;
genericRecord = datumReader.read(genericRecord, decoder);
in.close();
return genericRecord;
}
If you want to convert GenericRecord to map, note that String values are not Strings objects, you need to cast the Keys and values of types string:
private static Map<String, Object> avroGenericRecordToMap(GenericRecord record)
{
Map<String, Object> map = new HashMap<>();
record.getSchema().getFields().forEach(field ->
map.put(String.valueOf(field.name()), record.get(field.name())));
// Strings are maped to Utf8 class, so they need to be casted (all the keys of records and those values which are typed as string)
if(map.get("value").getClass() == org.apache.avro.util.Utf8.class)
map.put("value", String.valueOf(map.get("value")));
return map;
}
This is my code:
MongoDBSingleton dbSingleton = MongoDBSingleton.getInstance();
MongoDatabase db;
try {
db = dbSingleton.getTestdb();
MongoIterable<String> mg = db.listCollectionNames();
MongoCursor<String> iterator=mg.iterator();
while (iterator.hasNext()) {
MongoCollection<Document> table = db.getCollection(iterator.next());
for (Document doc: table.find()) {
System.out.println(doc.toJson());
}
}
}
This the output of toJson:
"modified" : { "$date" : 1475789185087}
This is my output of toString:
{"modified":"Fri Oct 07 02:56:25 IST 2016"}
I want String date format in Json, how to do it?
Sadly, IMO, MongoDB Java support is broken.
That said, there is a #deprecated class in the mongo-java-driver that you can use:
String json = com.mongodb.util.JSON.serialize(document);
System.out.println("JSON serialized Document: " + json);
I'm using this to produce fasterxml (jackson) compatible JSON from a Document object that I can deserialize via new ObjectMapper().readValue(json, MyObject.class).
However, I'm not sure what they expect you to use now that the JSON class is deprecated. But for the time being, it is still in the project (as of v3.4.2).
I'm importing the following in my pom:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-driver-async</artifactId>
<version>3.4.2</version>
</dependency>
<!-- Sadly, we need the mongo-java-driver solely to serialize
Document objects in a sane manner -->
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>3.4.2</version>
</dependency>
I'm using the async driver for actually fetching and pushing updates to mongo, and the non-async driver solely for the use of the JSON.serialize method.
No, it is not possible to produce the plain JSON. Please refer this link.
However, it can produce JSON in two modes.
1) Strict mode - Output that you have already got
2) Shell mode
Shell Mode:-
JsonWriterSettings writerSettings = new JsonWriterSettings(JsonMode.SHELL, true);
System.out.println(doc.toJson(writerSettings));
Output:-
"createdOn" : ISODate("2016-07-16T16:26:51.951Z")
MongoDB Extended JSON
In theory we are supposed to use toJSON() per...
https://jira.mongodb.org/browse/JAVA-1770
However, it seems that, at least up through 3.6, toJSON() isn't supported on various types the old JSON.serialize() method handled without issue, such as the AggregateIterable<Document> objects output by aggregate().
Here is a 2020 update to answer exactly your question, i.e. getting this exact format:
"modified":"2016-07-16T16:26:51.951Z"
You have to use writerSettings like notionquest suggested, but with a custom date converter and DateTimeFormatter.ISO_INSTANT:
public class JsonDateTimeConverter implements Converter<Long> {
private static final Logger LOGGER = LoggerFactory.getLogger(JsonDateTimeConverter.class);
static final DateTimeFormatter DATE_TIME_FORMATTER = DateTimeFormatter.ISO_INSTANT
.withZone(ZoneId.of("UTC"));
#Override
public void convert(Long value, StrictJsonWriter writer) {
try {
Instant instant = new Date(value).toInstant();
String s = DATE_TIME_FORMATTER.format(instant);
writer.writeString(s);
} catch (Exception e) {
LOGGER.error(String.format("Fail to convert offset %d to JSON date", value), e);
}
}
}
Use it like this:
doc.toJson(JsonWriterSettings
.builder()
.dateTimeConverter(new JsonDateTimeConverter())
.build())
if the bson.jar version is > 3.0.0 you may try document.toJson()
I used following
try {
MongoDatabase db = mongoClient.getDatabase("dbname");
MongoCollection<Document> collection = db.getCollection("nameofcollect");
Gson gson = new Gson();
ArrayList<JsonObject> array = new ArrayList<JsonObject>();
String jsonString = null;
/*WARNING Gson lib serialize string ->means add slash if you convert "json string" into "json string"*/
for (Document doc : collection.find()) {
jsonString = gson.toJson(doc);
array.add(new Gson().fromJson(jsonString, JsonObject.class));
}
//String finalarry = gson.toJson(array);
Map<Object, ArrayList<JsonObject>> seedMap = new HashMap<Object, ArrayList<JsonObject>>();
// String encode = coCoRy.encryptAndEncode(jsonString);
seedMap.put("seed", array);
String seedJsonString = gson.toJson(seedMap);
mongoClient.close();
return seedJsonString;
} catch (MongoException | ClassCastException e) {
e.printStackTrace();
return null;
}
Result will be like following
{
"seed": [
{
"_id": {
"timestamp": 1590914828,
"counter": 10457170,
"randomValue1": 5587428,
"randomValue2": -25784
},
"FIR_EVID_NUM": "3436345346",
"FIR_REG_NUM": "89678967",
"LOGIN_ID": "pc_admin",
"MEDIA_PATH": "C:\\Users\\ALPHAMALE\\Documents\\ShareX\\Screenshots\\2020-05\\1590211570.png"
},
{
"_id": {
"timestamp": 1590924463,
"counter": 7254997,
"randomValue1": 5012578,
"randomValue2": 24700
},
"FIR_EVID_NUM": "999999",
"FIR_REG_NUM": "888888",
"LOGIN_ID": "32323",
"MEDIA_PATH": "C:/uploads/c46847c7e2d130ffd746c789c0f0932e.png"
}
]
}
try this:
final JsonWriterSettings settings = JsonWriterSettings.builder( ).outputMode( JsonMode.SHELL ).build( );
System.out.println(doc.toJson(settings));
You can change the JsonMode is you wish
I am trying several examples from Jest to use as a POC for ElasticSearch integration.
Right now, I am trying just a basic GET. I created a POJO called Document. In there are some basic setters and getters are some fields. I populate it and then use GSON to generate the JSON text.
From this generated JSON, I go into ElasticSearch Sense and do the following:
PUT /reports/documents/3
{
// JSON code
}
This generates just fine. I then try using Get to pull the values out from Java, like so:
JestClientFactory factory = new JestClientFactory();
factory.setHttpClientConfig(new HttpClientConfig
.Builder("http://localhost:9200")
.multiThreaded(true)
.build());
client = factory.getObject();
Get get = new Get.Builder("reports", "3").type("documents").build();
try {
JestResult result = client.execute(get);
String json = result.getJsonString();
System.out.println(json);
Document doc = null;
doc = result.getSourceAsObject(Document.class);
System.out.println("is doc null? " + doc == null);
}catch (Exception e) {
System.err.println("Error getting document");
e.printStackTrace();
}
The String json returns what I would expect (showing _index, _type, _id and of course _source). However, doc always comes out as NULL. I am not sure why that is happening.
Just to see if this was just a Get problem, I proceeded to try to Search.I did the following code snippet:
try {
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchQuery("reportNumber", "101221895CRT-004"));
Search search = new Search.Builder(searchSourceBuilder.toString())
// multiple index or types can be added.
.addIndex("reports")
.addType("documents")
.build();
SearchResult result = client.execute(search);
//List<Document> results = result.getSourceAsObjectList(Document.class);
List<SearchResult.Hit<Document, Void>> hits = result.getHits(Document.class);
for (SearchResult.Hit hit : hits) {
Document source = (Document) hit.source;
Void ex = (Void) hit.explanation;
System.out.println();
}
System.out.println("Result size: " + hits.size());
}catch (Exception e) {
System.err.println("Error searching");
e.printStackTrace();
}
When looking at result, the JSON of the object is shown. However, the List<Document> results comes out as NULL. When using hits, the size of hits is correct, but the "source" and "ex" are both NULL.
Any ideas on what I am doing wrong with this?
UPDATE
After reading Cihat's comment, I went ahead and added in logging. It turns out I am getting an error when trying to convert a date (hence why it's always coming back as NULL).
I get the following error message:
Unhandled exception occurred while converting source to the object .com.someCompanyName.data.Document
com.google.gson.JsonSyntaxException: java.text.ParseException: Unparseable date: "Nov 6, 2014 8:29:00 AM"
I have tried all different formats:
11/06/2014 8:29:00 AM (and without time and making year just 14)
06-NOV-2014 8:29:00 AM (and without time and making year just 14)
2014-11-06 8:29:00 AM (same thing with time and year changes)
2014-NOV-06 8:29:00 AM (same thing with time and year changes)
06/11/2014 8:29:00 AM (same thing)
All of those failed. I am sure I tried some other formats, so not sure what format the date should be in. I even tried the exact date from DateFormat JavaDocs and it still failed. Every time I do a search, it says to define the Dateformat in the GsonBuilder, but in Jest I do not have access to that.
This test case demonstrates indexing a document with Jest and then getting the same document back out. Not a complete answer, but hopefully it is useful to see something that is known to work.
import io.searchbox.client.JestClient;
import io.searchbox.client.JestClientFactory;
import io.searchbox.client.JestResult;
import io.searchbox.client.config.HttpClientConfig;
import io.searchbox.core.Get;
import io.searchbox.core.Index;
import static org.hamcrest.Matchers.*;
import static org.hamcrest.MatcherAssert.*;
import org.junit.Test;
public class JestRoundtripIT {
public static final String INDEX = "reports";
public static final String TYPE = "documents";
public static final String ID = "3";
#Test
public void documentRoundTrip() throws Exception {
JestClientFactory factory = new JestClientFactory();
factory.setHttpClientConfig(new HttpClientConfig
.Builder("http://localhost:9200")
.multiThreaded(true)
.build());
JestClient client = factory.getObject();
Document original = new Document()
.withAuthor("Shay Banon")
.withContent("You know, for search...");
JestResult indexResult = client.execute(
new Index.Builder(original)
.index(INDEX)
.type(TYPE)
.id(ID)
.build());
assertThat(indexResult.isSucceeded(), equalTo(true));
JestResult getResult = client.execute(
new Get.Builder(INDEX, ID)
.type(TYPE)
.build());
assertThat(getResult.isSucceeded(), equalTo(true));
Document fromEs = getResult.getSourceAsObject(Document.class);
assertThat(fromEs, notNullValue());
assertThat(fromEs.getAuthor(), equalTo(original.getAuthor()));
assertThat(fromEs.getContent(), equalTo(original.getContent()));
}
public static class Document {
protected String author;
protected String content;
public Document withAuthor( String author ) {
this.author = author;
return this;
}
public Document withContent( String content ) {
this.content = content;
return this;
}
public String getAuthor() {
return author;
}
public void setAuthor( String author ) {
this.author = author;
}
public String getContent() {
return content;
}
public void setContent( String content ) {
this.content = content;
}
}
}
When using regular JSP forms for printing to the client, configuring the web.xml properly works for me (http://stackoverflow.com/questions/2147958/how-do-i-prevent-people-from-doing-xss-in-java).
Is there any "best practice" on how to escape/entityze strings which will be send via JSON to a jQuery function, which then populates the DOM with these values? Any recommended libraries or Spring Web Framework build-ins?
jQuery $.ajax-call to Spring MVC
Spring MVC responds in JSON
(magic encoding happens, e.g. <a> becomes <a> ) <= this one
jQuery receives the JSON and populates the DOM XSS-safe
Thanks in advance!
edit: I am also sometimes sending HTML on purpose, so the solution would need to be able to only handle the user input. It will probably turn out that every user-poisoned string will have to be sanitized manually?
try this class which I wrote for my use .
it may be useful check wether any case is missing . . . as no detailed testing is done on this yet.
If any issue arise please let me know. . .
(add corresponding jar Apache commons and net.sf.json)
package myutil;
import java.util.Iterator;
import net.sf.json.JSONArray;
import net.sf.json.JSONObject;
import org.apache.commons.lang.StringEscapeUtils;
public class JSONCleaner {
public static void main(String[] args) {
// TODO Auto-generated method stub
JSONObject jsonchild2=new JSONObject();
jsonchild2.put("subchlidkey1", "subchildvalue1");
jsonchild2.put("subchlidkey2", "subchildvalue2");
jsonchild2.put("subchlidkey3", "subchildvalue3");
JSONObject jsonchild=new JSONObject();
jsonchild.put("chlidkey1", "childvalue1");
jsonchild.put("chlidkey2", "childvalue2");
jsonchild.put("chlidkey3", "childvalue3");
JSONArray jsonarray=new JSONArray();
jsonarray.add("option1");
jsonarray.add("<p>option2</p>");
jsonarray.add(jsonchild2);
JSONObject json=new JSONObject();
json.put("name", "<b>nirbhay</b>");
json.put("age", 23);
json.put("jsonChildObject", jsonchild);
json.put("weight", 65);
json.put("array", jsonarray);
System.out.println(cleanJSONObject(json));
//System.out.println(json.getString("name"));
}
public static JSONObject cleanJSONObject(JSONObject jsonObject)
{
JSONObject returnJson=new JSONObject();
Iterator<?> keys = jsonObject.keys();
while( keys.hasNext() ){
String key = (String)keys.next();
//System.out.println(jsonObject.get(key));
if(jsonObject.optJSONObject(key)==null)
{
if(jsonObject.optJSONArray(key)!=null)
{
returnJson.put(key, cleanJSONArray(jsonObject.getJSONArray(key)));
}
else
{
returnJson.put(key, StringEscapeUtils.escapeHtml(jsonObject.getString(key)));
}
}
else
{
returnJson.put(key,cleanJSONObject(jsonObject.optJSONObject(key)));
}
}
return returnJson;
}
private static JSONArray cleanJSONArray(JSONArray array)
{
JSONArray returnArray=new JSONArray();
for(int i=0,j=array.size();i<j;i++)
{
if(array.optJSONObject(i)==null)
{
if(array.optJSONArray(i) != null)
{
returnArray.add(cleanJSONArray((JSONArray) array.get(i)));
}
else
{
returnArray.add(StringEscapeUtils.escapeHtml(array.getString(i)));
}
}
else
{
returnArray.add(cleanJSONObject((JSONObject) array.get(i)));
}
}
return returnArray;
}
}