Tell me what the problem is, and how to solve it. I need to get a response array json format. On the server, the response is formed in about 30 minutes, collects an array of 150K elements. Then it comes to dispatch, and hangs on hold for more than 24 hours. But even after such a long time, the answer does not come. There are no errors on the server. If you take 10K, then the answer comes without problems. I tried to save the answer to a TXT file, its size is +-35MB, and contains 35K stock.
This is how I execute the request:
Connection.Response response = Jsoup.connect(URLs.sayt + "/" + URLs.api + "/hs/CLIENTS/LIST/")
.method(Connection.Method.GET)
.maxBodySize(0)
.timeout(0)
.userAgent(URLs.getUserAgentMobile())
.header("Authorization", "Basic " + base64login)
.header("Accept-Language", "*")
.header("mode", "tobase")
.execute();
if(response != null){
ArrayList<Client> clients = new Gson().fromJson(response.body(), new TypeToken<List<Client>>() {}.getType());
for (Client client:clients){
sqLiteWork.AddToClient(client);
}
}
Here is an example of an element from the json array:
{"id": "9812d904-2d8a-11e8-80bb-a45d36c35311", "name": "Afaq", "code": "", "isGroup": false, "parent": "null", "address": "", "phone": "+994(12) 436 71 88", "phoneMore": "+994(50)2409696", "phoneHome": "", "debt": 0 }
I haven't found a solution to my question. I've tried both okhttp and httpurlconnection. Apparently, the android client itself does not support such a large string response in the body. Maybe it would be a solution via a file. But this is not a solution when you need actual data. Therefore, I made a response from the server for 10K elements. Full loading of 150K elements takes 2 hours.
Please note, I'm talking up-to-date data, if 2 hours is a long time, then online mode is also supported in my application.
Connection connection = Jsoup.connect(URLs.sayt + "/" + URLs.api + "/hs/CLIENTS/LIST/")
.method(Connection.Method.GET)
.ignoreContentType(true)
.maxBodySize(0)
.timeout(0)
.userAgent(URLs.getUserAgentMobile())
.header("Authorization", "Basic " + base64login)
.header("Accept-Language", "*")
.header("mode", "tobase");
Connection.Response response = connection.execute();
ArrayList<Client> clients = new Gson().fromJson(response.body(), new TypeToken<List<Client>>() {}.getType());
Client last = null;
for (Client client:clients){
last = client;
sqLiteWork.AddToClient(client);
}
while (clients.size() > 0){
response = connection.header("last", last.id).execute();
clients = new Gson().fromJson(response.body(), new TypeToken<List<Client>>() {}.getType());
for (Client client:clients){
last = client;
sqLiteWork.AddToClient(client);
}
}
Related
I want to form a new MSTeam from a newly created group.
I created a new GraphServiceClient
I created a new AuthProvider
I created a new Group with the client like this:
graphApplicationClient.groups().buildRequest().post(group);
It works till here. I get a Group object as a response and can work with it. It has an Id and so on
Right after that I want to create a new team out of this group, so I call
ITeamRequest iTeamRequest = graphApplicationClient.groups(pGroupId).team().buildRequest();
iTeamRequest.setMaxRetries(3);
iTeamRequest.setDelay(10);
Team newTeam = iTeamRequest.post(team);
where pGroupId is the Id I got from the call before.
(In this case: "team" is a new Team-Object created right before this call, like in the MS-Docs.)
I know the sentence from MS-Docs:
If the group was created less than 15 minutes ago, it's possible for
the Create team call to fail with a 404 error code due to replication
delays. The recommended pattern is to retry the Create team call three
times, with a 10 second delay between calls.
thats why I added the maxRetries and Delay..
I watched this video:
https://youtu.be/ybGm1qWVi-k?t=650
in which two MS Employees do exactly the same as I do...
What am I missing? Or is there a workaround for it?
This is the response:
404 : Not Found
Strict-Transport-Security : max-age=31536000
Cache-Control : private
x-ms-ags-diagnostic : {"ServerInfo":{"DataCenter":"West Europe","Slice":"SliceC","Ring":"5","ScaleUnit":"002","RoleInstance":"AGSFE_IN_51"}}
client-request-id : b39c16e4-f786-4d42-865e-9f0cf23ed46f
request-id : 07efc77c-02d3-45f7-85ab-6ca2241d1859
Content-Length : 198
Date : Thu, 23 Apr 2020 12:39:30 GMT
Content-Type : application/json
{
"error": {
"code": "UnknownError",
"message": "",
"innerError": {
"request-id": "07efc77c-02d3-45f7-85ab-6ca2241d1859",
"date": "2020-04-23T12:39:30"
}
}
}
It also does not work, if I put more retries (10 = Max) and more Delay (180 = Max) in it...
And there is no PUT in java..
Best regards
EDIT1: I tried to get it done with two other options.. first with ScribeJava and second with a normal httpRequest.. none of them works. I only get it done with graphExplorer.. With the other options it says BadRequest, but when i copy everything I got in code to GraphExplorer it works... help me :(
EDIT2: I have a group that's been around for a day. From this I wanted to create a new team. Still 404 Not Found. I tried it with GraphAPI and ScribeJava.. None of these calls are working. Debugged through, copied all information, put it in Postman, works fine.
For everyone who has the same problem as I. GraphAPI provides a customRequest() function.
I use the same token, the same url, the same properties, but using the customRequest with PUT. Works...
CustomRequest<JsonObject> createTeamCustomRequest = graphApplicationClient.customRequest("/groups/" + pGroupId + "/team").buildRequest();
createTeamCustomRequest.setMaxRetries(3);
createTeamCustomRequest.setDelay(10);
JsonObject newTeam = _createTeamInJson();
JsonObject createdTeam = createTeamCustomRequest.put(newTeam);
This is the _createTeamInJson function. Returns the same body as I create a Team with GraphAPI
private JsonObject _createTeamInJson()
{
JsonObject teamPayload = new JsonObject();
JsonObject memberSettings = new JsonObject();
JsonObject messagingSettings = new JsonObject();
JsonObject funSettings = new JsonObject();
memberSettings.addProperty("allowCreateUpdateChannels", true);
messagingSettings.addProperty("allowUserEditMessages", true);
messagingSettings.addProperty("allowUserDeleteMessages", true);
funSettings.addProperty("allowGiphy", true);
funSettings.addProperty("giphyContentRating", "strict");
teamPayload.add("memberSettings", memberSettings);
teamPayload.add("messagingSettings", messagingSettings);
teamPayload.add("funSettings", funSettings);
return teamPayload;
}
This isn't the right solution for it, because it should work with the normal GraphAPI, that's why I don't mark this as an answer. But it is a workaround for everyone who has the same problem..
I am storing emails to ElasticSearch but whenever they have a character not from iso latin like "コニカミノルタ", they get changed to ? in ES:
"content": """
???????
?? ?
?????????????????
I am using the following code to post to ES:
HttpPost post = new HttpPost(url);
try {
// set import date
doc.setImportTimestamp(new Date());
// convert item to json string
post.setEntity(new StringEntity(JsonUtils.toJson(doc)));
// index
post.setHeader("Content-Type", "application/json; charset=UTF-8");
// perform http post
LOG.info("Index doc to " + url);
LOG.info("JSON : [" + JsonUtils.toJson(doc) + "]");
try (CloseableHttpResponse res = httpClient.execute(post)) {
LOG.debug(IOUtils.toString(res.getEntity().getContent(), StandardCharsets.UTF_8));
}
This is the string in the JSON object before it gets posted:
> "content": "コニカミノルタ\r\n宮脇 様\r\n\r\nいつも大変お世話になっております。\r\n\r\n今回表示されたエラーを教えていただけますでしょうか。\r\nスクリーンショットをお送りいただけますと情報として問題解決にとても\r\n助かります。\r\n\r\n検証会議につきまして、明日でも宜しければ参加をさせていただきます。\r\nお手数ですが、それまでに先ほどお送りしました証明書の追加作業が\r\n御社IT部門の方にて適用されていることをご確認いただけましたら幸甚です。\r\n\r\nご連絡をお待ちしております。\r\n\r\nどうぞよろしくお願い致します。\r\n\r\n山本
美奈子\r\nMinako Yamamoto\r\nWeb Support APAC \r\nUnified Communications
Services\r\nAU +612 8295 9015 / 1800 766 770 \r\nHK 800 964 180\r\nIN
1800 3010 3990 \r\nJP 0120300532\r\nKR 0808080501\r\nSG 800 616 \r\ne
WebSupport-APAC#west.com
\r\nwestuc.com\r\n\r\n\r\n\r\n--------------------------------------------------------------------------------------\r\n",
Any help is welcomed.
Thanks
Raúl
I've been trying to place labels over newly created nodes using Neo4J restful api. The following is the CURL request I've tried.
Client client = ClientBuilder.newClient();
WebTarget target = client.target("http://localhost:7474/db/data/");
String propertyUri_labels = "http://localhost:7474/db/data/node/9/labels";
Response response = target
.path(propertyUri_labels)
.request(MediaType.APPLICATION_JSON)
.header("application/xml", "true")
.accept(MediaType.APPLICATION_JSON)
.put(Entity.entity("\"" + "Artist" + "\"", MediaType.APPLICATION_JSON_TYPE));
System.out.println( String.format( "PUT to [%s], status code [%d]",
propertyUri_labels, response.getStatus() ) );
However, I have not been successful and got an status code [400]. Is there something wrong with my URL "http://localhost:7474/db/data/node/9/labels" ?
Pls help.
Thanks!
Try a POST instead of PUT.
Check the documentation => http://neo4j.com/docs/rest-docs/current/#rest-api-node-labels
I am working on a fragment in android studio whose purpose is to display To/From, Subject, and the body of the message. So far, I am able to retrieve, decode, and display the body. I tried using a similar method for the headers but for some reason it isn't decoding properly, or my method calls aren't getting the correct information. Here is the code I am working with:
String user = "me";
String query = "in:inbox is:unread";
textView.setText("Inbox");
ListMessagesResponse messageResponse =
mService.users().messages().list(user).setQ(query).setMaxResults(Long.valueOf(1)).execute();
List<Message> messages = messageResponse.getMessages();
for(Message message : messages){
Message message2 = mService.users().messages().get(user, message.getId()).execute();
//Get Headers
byte[] headerBytes = Base64.decodeBase64(message2.getPayload().getParts().get(0).getHeaders().get(0).getName().toString().trim()); // get headers
String header = new String(headerBytes, "UTF-8");
//Get Body
byte[] bodyBytes = Base64.decodeBase64(message2.getPayload().getParts().get(0).getBody().getData().trim().toString()); // get body
String body = new String(bodyBytes, "UTF-8");
messageList.add(header);
messageList.add(body);
}
return messageList;
The section under // get body works. But the section under //Get Headers returns data with weird symbols which include black diamonds with white question marks inside and letters in random order. I have tried many different combinations and orders for the method calls in the Base64.decodeBase64 statement for headerBytes but wasn't able to succeed. Is there something I am missing?
Edit: I looked at the gmail-api documentation on the google developers site and I still am confused on how the header information is stored and how to retrieve specific things such as To, From, and Subject. That might be my problem since I may not be targeting the correct data.
If I list messages and get the first one, we can see what the message looks like:
Request
format = metadata
metadataHeaders = From,To,Subject
fields = payload/headers
GET https://www.googleapis.com/gmail/v1/users/me/messages/15339f3d12042fec?format=metadata&metadataHeaders=To&metadataHeaders=From&metadataHeaders=Subject&fields=payload%2Fheaders&access_token={ACCESS_TOKEN}
Response
{
"payload": {
"headers": [
{
"name": "To",
"value": "Emil <emtholin#gmail.com>"
},
{
"name": "From",
"value": "\"BernieSanders.com\" <info#berniesanders.com>"
},
{
"name": "Subject",
"value": "5,000,000"
}
]
}
}
As you can see, the values you are looking for are in the headers. You just have to sort them out in Java and you are done. The headers are not encoded like the body, so there is no need to do any decoding.
I am creating some nodes within a transaction in neo4j using the rest api. After all nodes have been created (typically between 3 and 5 in one transaction), I have to create some relationships between them. To do this I need, of course the location of the nodes, and this is the source of my problem. I can't figure out how to get this location.
According to documentation, I should be able to get the location of a node from the response-object, after creating the node, like so:
nodeLocation = response.getLocation();
But in a transaction, this of course returns the url of the transaction:
http://localhost:7474/db/data/transaction/108
Then I thought, if I query for the just created node, maybe in that response I can find the location. Again, according to documentation, the node location should be presented in the json-structure extensions in the field self.
"self" : "http://localhost:7474/db/data/node/357",
But my response to the query does not seem to contain an extension structure.
This is the query I'm using:
{"statements": [ {"statement": "MATCH (p:POST {sn_id: 'TW', id: '536982477664190465'} ) RETURN p"} ] }
I send it to the open transaction, and I get this back:
GET to http://localhost:7474/db/data/transaction/108 returned status code 200, returned data: {"commit":"http://localhost:7474/db/data/transaction/108/commit","results":[{"columns":["p"],"data":[]}],"transaction":{"expires":"Mon, 24 Nov 2014 20:40:34 +0000"},"errors":[]}
Just for completeness, this is the code for my query:
String payload = "{\"statements\": "
+ "[ "
+ "{\"statement\": "
+ "\"MATCH (p:POST {sn_id: 'TW', id: '536982477664190465'} ) RETURN p\""
+ "} "
+ "] "
+ "}";
logger.trace("sending cypher {} to endpoint {}", payload, endpointLoc);
WebResource resource = Client.create().resource( endpointLoc );
ClientResponse response = resource
.accept( MediaType.APPLICATION_JSON )
.type( MediaType.APPLICATION_JSON )
.entity( payload )
.get(ClientResponse.class);
//.post( ClientResponse.class );
String responseEntity = response.getEntity(String.class).toString();
int responseStatus = response.getStatus();
logger.trace("GET to {} returned status code {}, returned data: {}",
endpointLoc, responseStatus,
responseEntity);
JSONParser reponseParser = new JSONParser();
Object responseObj = reponseParser.parse(responseEntity);
JSONObject jsonResponseObj = responseObj instanceof JSONObject ?(JSONObject) responseObj : null;
if(jsonResponseObj == null)
throw new ParseException(0, "returned json object is null");
String result = (String) jsonResponseObj.get("results").toString();
logger.trace("result is {} ", result);
String error = (String) jsonResponseObj.get("errors").toString();
Am I missing something? Do I need to use a special call?
Can someone help me with this? Thanks in advance,
Christian
What do you need the node-URL for?
That's the old RESTful representation. You can either get it by using the old HTTP endpoint /db/data/cypher or better by specifying the (very verbose) resultDataContents type REST
You can also specify other types like "row" and "graph" in parallel.
{"statements": [
{"statement": "MATCH (p:POST {sn_id: 'TW', id: '536982477664190465'} ) RETURN p",
"resultDataContents":["REST"]}
] }