I am searching through solr. it gives me response in json. like following:
{response
{numfound:# , docs{
[
{
id="#"
model="#"
}
{
id="#"
model="#"
}
]
}
}
I want to extract just the ids from this and make java array list from them.
Can someone please tell me how i can do that in coding language?
question: how to just extract id from son string and convert then into java array list if i am using hashmap or objectmapper in java?
thanx
If you want to convert into java objects, you can work with Solr Client Solrj
Solrj will give you and easy option to query Solr and read the xml to convert into java objects
For JSON you can use jackson library for parsing Solr response.
Here is a small program that will read the data using the Gson library.
#Test
public void testRead() throws IOException {
final String testUrl = "http://localhost:8983/solr/collection1/select?q=*%3A*&wt=json&indent=true";
String out = new Scanner(new URL(testUrl).openStream(), "UTF-8").useDelimiter("\\A").next();
SolrResult result = new Gson().fromJson(out, SolrResult.class);
List<String> ids = new ArrayList<String>();
for (Doc doc : result.response.docs) {
ids.add(doc.id);
}
System.err.println(ids);
}
It uses the following three small classes as a Java representation of the response:
public class SolrResult {
public Response response;
}
class Response {
public int numFound;
public int start;
public List<Doc> docs;
}
class Doc {
public String id;
}
I hope this helps.
As Jayendra pointed you can use SolrJ if you are querying Solr using Java. I don't suggest to use XML as response format due to XML parsing overhead. SolrJ supports java binary object encoding (using commons-codec) out-of-the-box and by default on queries.
To use it on updates you have to set it up manually using
server.setRequestWriter(new BinaryRequestWriter());
and you have to enable the BinaryUpdateHandler on the server too, using:
<requestHandler name="/update/javabin" class="solr.BinaryUpdateRequestHandler" />
In the case you don't want to use SolrJ you can parse it using GSON. Here you have and example.
Related
I'm working with the MarkLogic POJO Databinding Interface at the moment. I'm able to write POJOs to MarkLogic. Now I want to search those POJOs and retrieve the search results. I'm following the instructions from: https://docs.marklogic.com/guide/java/binding#id_89573 However, the search results don't seem to return the correct objects. I'm getting a JSONMappingException. Here's the code:
HashMap<String, MatchedPropertyInfo> matchedProperties = new HashMap<String, MatchedPropertyInfo>();
PropertyMatches PM = new PropertyMatches(123,"uri/prefix/location2", "uri/prefix", 1234,0,"/aKey","/aLocation",true,matchedProperties);
MatchedPropertyInfo MPI1 = new MatchedPropertyInfo("matched/property/uri1", "matched/property/key1", "matched/property/location1", true,"ValueMatch1", 12, 1*1.0/3, true);
MatchedPropertyInfo MPI2 = new MatchedPropertyInfo("matched/property/uri2", "matched/property/key2", "matched/property/location2", true,"ValueMatch2", 14, 1.0/2.0, true);
PM.getMatchedProperties().put("matched/property/prefix/location1", MPI1);
PM.getMatchedProperties().put("matched/property/prefix/location2", MPI2);
PojoRepository myClassRepo = client.newPojoRepository(PropertyMatches.class, Long.class);
myClassRepo.write(PM);
PojoQueryBuilder qb = myClassRepo.getQueryBuilder();
PojoPage<PropertyMatches> matches = myClassRepo.search(qb.value("uri", "uri/prefix/location2"),1);
if (matches.hasContent()) {
while (matches.hasNext()) {
PropertyMatches aPM = matches.next();
System.out.println(" " + aPM.getURI());
}
} else {
System.out.println(" No matches");
}
The PropertyMatches (PM) object is succesfully written to the MarkLogic database. This class contains a member: private String URI which is initiated with "uri/prefix/location2". The matches.hasContent() returns true in the example above. However, I'm getting an error on PropertyMatches aPM = matches.next();
Searching POJOs in MarkLogic and read them into your Java program requires the POJOs to have an empty constructor. In this case PropertyMatches should have public PropertyMatches(){} and MatchedPropertyInfo should have public MatchedPropertyInfo(){}
Thanks #sjoerd999 for posting the answer you found. Just to add some documentation references, this topic is discussed here: http://docs.marklogic.com/guide/java/binding#id_54408 and here: https://docs.marklogic.com/javadoc/client/com/marklogic/client/pojo/PojoRepository.html.
Also worth noting is you can have multiple parameters in the consructor, you just have to do it the Jackson way. Here are examples of two ways (with annotations and without): https://manosnikolaidis.wordpress.com/2015/08/25/jackson-without-annotations/
I'd recommend using annotations as that's built-in with Jackson. But if you want to do it without annotations, here's the code:
ObjectMapper mapper = new ObjectMapper();
// Avoid having to annotate the Person class
// Requires Java 8, pass -parameters to javac
// and jackson-module-parameter-names as a dependency
mapper.registerModule(new ParameterNamesModule());
// make private fields of Person visible to Jackson
mapper.setVisibility(FIELD, ANY);
If you want to do this with PojoRepository you'll have to use the unsupported getObjectMapper method to get the ObjectMapper and call registerModule and setVisibility on that:
ObjectMapper objectMapper = ((PojoRepositoryImpl) myClassRepo).getObjectMapper();
I want to access Domino data via the Domino Access Services (DAS) as REST provider in java e.g.
String url = "http://malin1/fakenames.nsf/api/data/collections/name/groups";
ObjectMapper mapper = new ObjectMapper();
JsonFactory factory = new JsonFactory();
JsonParser parser = factory.createParser(new URL(url));
JsonNode rootNode = mapper.readTree(parser);
however, I notice DAS binds the JSON in square brackets:
[
{
"#entryid":"1-D68BB54DEA77AC8085256B700078923E",
"#unid":"D68BB54DEA77AC8085256B700078923E",
"#noteid":"1182",
"#position":"1",
"#read":true,
"#siblings":3,
"#form":"Group",
"name":"LocalDomainAdmins",
"description":"This group should contain all Domino administrators in your domain. Most system databases and templates give people in this group Manager access."
},
{
"#entryid":"3-9E6EABBF405A1A9985256B020060E64E",
"#unid":"9E6EABBF405A1A9985256B020060E64E",
"#noteid":"F46",
"#position":"3",
"#read":true,
"#siblings":3,
"#form":"Group",
"name":"OtherDomainServers",
"description":"You should add all Domino servers in other domains with which you commonly replicate to this group."
}
]
How can I easily get rid of these brackets?
As already mentioned you should leave them intact. You can parse theJSON array for example with Jackson.
find an example snippet below
import org.codehaus.jackson.JsonNode;
import org.codehaus.jackson.JsonProcessingException;
import org.codehaus.jackson.map.ObjectMapper;
...
String response = ... your posted string
ObjectMapper mapper = new ObjectMapper();
try {
JsonNode taskIdsjsonNode = mapper.readTree(response);
for (JsonNode next : taskIdsjsonNode) {
System.out.printf("%s: %s%n", "#entryid", next.get("#entryid"));
System.out.printf("%s: %s%n", "name", next.get("name"));
}
} catch (.... ) {
// your exception handling goes here
}
output
#entryid: "1-D68BB54DEA77AC8085256B700078923E"
name: "LocalDomainAdmins"
#entryid: "3-9E6EABBF405A1A9985256B020060E64E"
name: "OtherDomainServers"
The brackets are not nasty but a correct notation. To access the contens just use [0] in your client side script or with your JSON parser in Java you like.
Perhaps the explanation here can help:
https://quintessens.wordpress.com/2015/05/08/processing-json-data-from-domino-access-services-with-jackson/
Basically you establish a call to DAS via the Jersey client and then you parse the json via Jackson library to a map in java.
During the parsing process you can define which values you want to parse and transform them.
Take a look at the Person class...
I'm trying to retrieve a list of indices using Jest, but I just got as far as:
Stats statistics = new Stats.Builder().build();
result = client.execute(statistics);
How can i retrieve the list of indices from the result? Do I have to use something else than Stats?
It would also help if someone could show me a detailed documentation of Jest. The basics are really well documented, but with the different kinds of builders I'm really lost at the moment.
Get Aliases will give you all the aliases for the indices on a node.
One can simply navigate a browser to the following URL to get the indexes available on an ElasticSearch cluster.
http://elasticsearch.company.com/_aliases
This will return an array of indexes and their aliases in JSON. Here's an example:
{
"compute-devzone1": { },
"compute-den2": { },
"compute-den1": { },
...
}
To get the list of indexes with Jest, use this code...
HttpClientConfig config;
JestClientFactory factory;
JestClient client;
GetAliases aliases;
JestResult result;
String json;
config = new HttpClientConfig.
Builder("http://elasticsearch.company.com").
build();
aliases = new GetAliases.
Builder().
build();
factory = new JestClientFactory();
factory.setHttpClientConfig(config);
client = factory.getObject();
result = client.execute(aliases);
json = result.getJsonString();
Use your favorite JSON processor to extract the indexes from json.
Use:
JestResult result = elasticSearchClient.execute(new Cat.IndicesBuilder().build());
This will return a JSON response just like curl -XGET "localhost:9200/_cat/indices?format=json"
I am trying to deserialize the following incoming JSON data:
{"TimeTable":[{"PersonID":"100649771",
..,..,..,"xx":null},
{"PersonID":"100631701",
..,..,..,"xx":{"abc":1234,"xyz":5678}}],
"xxx":"","xxxx":0,"xxxxx":false}
But I am facing a problem while parsing using a custom deserialization block made up of:
jParser.nextToken();
while ((jParser.nextToken() != JsonToken.END_ARRAY)) {
String innerField = jParser.getCurrentName();
jParser.nextToken();
But in this way I am skipping the array contents While parsing for the second row in the array (as illustrated in the JSON sample above^).
UPDATE: Here is the method(PasteBin Link) which is trying to parse the JSON data coming in the described format. Is there a way I can bind the JSON data directly to my bean? (IMO it appeared way more complex to me as because of the JSON structure; Moreover I can't change the JSON structure nor the bean structure. So, I just dropped the idea of binding directly :| ) Anyways here(PasteBin Link) is the bean as well.
Following is the sample of the incoming JSON Data:
{"Schedules":[{"PersonID":"100649771",
"HasSchedule":false,
"TripType":null,
"StickerNumber":null,
"VehicleRegNo":null,
"ExpectedStartDate":null,
"ActualStartDate":null,
"ActualEndDate":null,
"PersonScheduledDate":null,
"Shift":null,
"ColdCall":null,
"PickupLocationCoord":null},
{"PersonID":"100631701",
"HasSchedule":true,
"TripType":"P",
"StickerNumber":"PC0409",
"VehicleRegNo":"ASJHAHSP1758",
"ExpectedStartDate":"16 Aug 2013, 10:00:00",
"ActualStartDate":"16 Aug 2013, 10:02:52",
"ActualEndDate":"16 Aug 2013, 14:14:12",
"PersonScheduledDate":null,
"Shift":"02:30 PM",
"ColdCall":"N",
"PickupLocationCoord":{"Latitude":92.01011101,"Longitude":48.01011101}}],
"ErrorMessage":"","ErrorCode":0,"HasError":false}
Please can anyone fireup some pointers for me here in order- to deserialize 'em correctly?
Thanks
UPDATED
Among other things, you are mixing up the two aproaches to read a JSON stream: using readTree() to get all your JSON data in a memory tree (like XML's DOM) but also using a JsonParser to read a JSON stream token by token (like XML's JAX). The following is a method that does almost the same using readTree(), which seems to be more appropriate to you as you are reading JSON already loaded in a String:
public List<VehicleInformationBean> getAllVehiclesInTree(String response) {
List<VehicleInformationBean> vehicleList = new ArrayList<VehicleInformationBean>();
try {
PersonInformationBean mPersonInformationBean;
DatabaseHelper mDatabaseHelper = DatabaseHelper.getInstance(sContext);
ObjectMapper mapper = new ObjectMapper();
ObjectNode root = (ObjectNode) mapper.readTree(response);
if ((root.get(ServiceConstant.ErrorCode).asInt()) != 0 || !root.has(ServiceConstant.Schedules)) {
return vehicleList;
}
for(JsonNode element: root.get(ServiceConstant.Schedules)) {
VehicleInformationBean lstVehicleInformation = new VehicleInformationBean();
if (element.has(ServiceConstant.PersonID)) {
String personId = element.get(ServiceConstant.PersonID).asText();
mPersonInformationBean = mDatabaseHelper.getPersonDetailById(personId);
lstVehicleInformation.setPersonID(personId);
lstVehicleInformation.setName(mPersonInformationBean.getName());
lstVehicleInformation.setPickupLocation(mPersonInformationBean.getPickupLocation());
lstVehicleInformation.setDropLocation(mPersonInformationBean.getDropLocation());
}
lstVehicleInformation.setTripType(element.get(ServiceConstant.TripType).textValue());
lstVehicleInformation.setStickerNumber(element.get(ServiceConstant.StickerNumber).textValue());
lstVehicleInformation.setVehicleRegNo(element.get(ServiceConstant.VehicleRegNo).textValue());
lstVehicleInformation.setExpectedStartDate(element.get(ServiceConstant.ExpectedStartDate).textValue());
lstVehicleInformation.setActualStartDate(element.get(ServiceConstant.ActualStartDate).textValue());
lstVehicleInformation.setActualEndDate(element.get(ServiceConstant.ActualEndDate).textValue());
lstVehicleInformation.setPersonScheduledDate(element.get(ServiceConstant.PersonScheduledDate).textValue());
lstVehicleInformation.setShift(element.get(ServiceConstant.Shift).textValue());
if (element.has("PickupLocationCoord")) {
JsonNode coords = element.get("PickupLocationCoord");
if(coords.has(ServiceConstant.Latitude)) {
lstVehicleInformation.setLatitude(coords.get(ServiceConstant.Latitude).asDouble());
}
if(coords.has(ServiceConstant.Longitude)) {
lstVehicleInformation.setLongitude(coords.get(ServiceConstant.Longitude).asDouble());
}
} else if (element.has(ServiceConstant.ColdCall)) {
lstVehicleInformation.setColdCall(element.get(ServiceConstant.ColdCall).textValue());
}
vehicleList.add(lstVehicleInformation);
}
} catch (Exception e) {
// TODO doing something with exception or throw it if it can't be handled here
e.printStackTrace();
}
return vehicleList;
}
There are some validations and extra code you need to add to this method for it to do exactly what your original method does. This method only shows you the main idea of how to do it.
Need a quick help. I am a newbie in QuickFixJ. I have a FIX message in a txt file. I need to convert that into FIX50SP2 format. I am enclosing the code snippet.
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
System.out.println("FixMsg String:"+fixMsg);
Message FIXMessage = new Message();
DataDictionary dd = new DataDictionary("FIX50SP2.xml");
FIXMessage.fromString(fixMsg, dd, false);
System.out.println("FIXMessage Output:" + FIXMessage.toString()); // Print message after parsing
MsgType msgType = new MsgType();
System.out.println(FIXMessage.getField(msgType));
Here is the output:
FixMsg String:1128=99=15835=X49=CME34=47164052=2012090312102051175=20120904268=1279=122=848=336683=607745107=ESU2269=1270=140575271=123273=121020000336=2346=501023=110=205
FIXMessage Output:9=6135=X34=47164049=CME52=2012090312102051175=20120904268=110=117
quickfix.FieldNotFound: Field [35] was not found in message.
at quickfix.FieldMap.getField(FieldMap.java:216)
at quickfix.FieldMap.getFieldInternal(FieldMap.java:353)
at quickfix.FieldMap.getField(FieldMap.java:349)
at MainApp.main(MainApp.java:52)
I want to extract MsgType field (field 35). Could you please tell me where I am wrong? The thing I have observed is that after parsing to FIX50SP2 format, the convert FIX message is missing many data element (for details see the output)
Thanks
Like others mentioned the MsgType is an header field and you get it by using the following
String msgType = null;
if(FIXMessage.getHeader().isSetField(MsgType.FIELD)) {
msgType = FIXMessage.getHeader().getString(MsgType.FIELD);
}
System.out.println("MsgType is " + msgType);`
The reason you are missing many data element after parsing is, probably your message have some custom tags(like tag 2346), which is not defined in your data dictionary(FIXSP02.xml). hence the parsing of those tags failed and missing in the output.
To fix this, get the data dictionary from the party that is sending you the message and use it to parse the message
I'm not familiar with FIX messages and QuickFixJ, but glancing at the Javadoc, it seems like you should use the identifyType method :
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
MsgType msgType = Message.identifyType(fixMsg);
You may find FixB framework useful as it deals well with non-standard use cases of FIX.
As in your case, to extract only data you are interested in, you need to define a class that will represent this data and to bind it to FIX using annotations. E.g.:
#FixBlock
public class MDEntry {
#FixField(tag=269) public int entryType; // you could define an enum type for it as well
#FixField(tag=278) public String entryId;
#FixField(tag=55) public String symbol;
}
...
FixFieldExtractor fixExtractor = new NativeFixFieldExtractor();
List<MDEntry> mdEntries = fixExtractor.getGroups(fixMsg, List.class, 268, FixMetaScanner.scanClass(MDEntry.class))
In more common cases, FixSerializer interface should be used, but it requires a message with MsgType(35) tag and a class annotated with #FixMessage(type="...") accordingly. E.g.:
#FixMessage(type="X")
public class MarketData {
#FixGroup(tag=268) public List<MDEntry> entries;
}
...
FixMetaDictionary fixMetaDictionary = FixMetaScanner.scanClassesIn("my.fix.classes.package");
FixSerializer fixSerializer = new NativeFixSerializer("FIX.5.0.SP2", fixMetaDictionary);
MarketData marketData = fixSerializer.deserialize(fixMsg);
I hope you will find it useful.
If you need just a MsgTyp, you're sure the message is correct and you do not need any other field from the message, then I would recommend extracting MsgType from string using regexp.
e.g.: \u000135=(\w+)\u0001
It is MUCH FASTER than parsing (and validating) a string via QuickFix.