I have been trying to put data in elastic search through java using the following code:
String url = "http://localhost:9200/testindex2/test/2";
HttpClient client = new DefaultHttpClient();
HttpPut put = new HttpPut(url);
JSONObject json = new JSONObject();
json.put("email", "abc#abof.com");
json.put("first_name", "abc");
StringEntity se = new StringEntity("JSON: " + json.toString());
se.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE,"Text"));
put.setEntity(se);
HttpResponse response = client.execute(put);
System.out.println("\nSending 'PUT' request to URL : " + url);
System.out.println("Put parameters : " + put.getEntity());
System.out.println("Response Code : " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
And I am getting the following error:
Sending 'PUT' request to URL : http://localhost:9200/testindex2/test/2
Put parameters : [Content-Type: text/plain; charset=ISO-8859-1,Content- Encoding: Text,Content-Length:
52,Chunked: false]
Response Code : 400
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed
to parse"}],"type":"mapper_parsing_exception","reason":"failed to
parse","caused_by":{"type":"not_x_content_exception","reason":"Compressor
detection can only be called on some xcontent bytes or compressed
xcontent bytes"}},"status":400}
Also when I try the same code from a rest client it runs just fine, not sure why this problem is happening.
Replaced
StringEntity se = new StringEntity("JSON: " + json.toString());
se.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE,"Text"));
with this:
StringEntity se = new StringEntity(json.toString(),ContentType.APPLICATION_JSON);
and its working now
Elastic search has special client to work with Java. And you don't need to generate JSON manually. Moreover you didn't describe import section, so a bit hard to understand what libraries you use.
I was getting the same error, but in my case, I was actually doing something like this in a Kubernetes configuration for an init container:
- args:
- -XPUT
- -k
- {{.Values.kibana.env.ELASTICSEARCH_URL}}/.logtrail/config/1?pretty
- -H
- 'Content-Type: application/json'
- --data
- /etc/logtrail/logtrail.json
The problem here is that when you specify a file in a curl POST/PUT, it needs to be appended by '#'. So the below configuration worked!
- args:
- -XPUT
- -k
- {{.Values.kibana.env.ELASTICSEARCH_URL}}/.logtrail/config/1?pretty
- -H
- 'Content-Type: application/json'
- --data
- '#/etc/logtrail/logtrail.json'
uri = 'http://projects.local:9200/'
_index = 'mydata/'
_type = '_doc/'
head = {'Content-Type': 'application/json'}
body = {
"capital" : "boston",
"state" : "massachusetts"
}
Using json.dumps to convert body to jbody worked for me
jbody = json.dumps(body)
response = requests.post(uri+_index+_type, headers=head, data=jbody)
Related
I would like to send POST request using Java equivalent of the following CURL example:
echo "param value" | curl --data-binary #- -uuser:pass https://url
I've tried apache http setEntity(FileEntity entity), 400 bad request
I've tried apache http setEntity(MultiPartEntity entity), 400 bad request
// ----------------
// General part
String url = "https://url";
String content = "param" + " " + "value";
File file = new File("test.txt");
try {
FileUtils.writeStringToFile(file, content, StandardCharsets.UTF_8);
} catch (IOException e) {
e.printStackTrace();
}
String encoding = Base64.getEncoder().encodeToString(("user:pass").getBytes());
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
post.setHeader(HttpHeaders.AUTHORIZATION, "Basic " + encoding);
// -----------------
// 1. FileEntity try
FileEntity reqEntity = new FileEntity (file, ContentType.DEFAULT_BINARY);
post.setEntity(reqEntity);
HttpResponse response = httpclient.execute(post);
// ----------------
// 2. Multipart try
MultipartEntity mpEntity = new MultipartEntity();
ContentBody cbFile = new FileBody(file, org.apache.http.entity.ContentType.DEFAULT_BINARY);
mpEntity.addPart("userfile", cbFile);
post.setEntity(mpEntity);
HttpResponse response = httpclient.execute(post);
I've expected to get 200, but got 400 Bad request.
Original CURL works as expected
Problem with boundary parameter is not in the Content-Type header
Actually If you are using one of multipart/ content types, you are actually required to specify the boundary parameter in the Content-Type header ,But in here with curl request not trying generate any boundary values , without boundary values the server (in the case of an HTTP request) will not be able to parse the payload
I would like to run this specific curl command with a HTTP POST request in java
curl --location --request POST "http://106.51.58.118:5000/compare_faces?face_det=1" \
--header "user_id: myid" \
--header "user_key: thekey" \
--form "img_1=https://cdn.dnaindia.com/sites/default/files/styles/full/public/2018/03/08/658858-577200-katrina-kaif-052217.jpg" \
--form "img_2=https://cdn.somethinghaute.com/wp-content/uploads/2018/07/katrina-kaif.jpg"
I only know how to make simple POST requests by passing a JSON object, But i've never tried to POST based on the above curl command.
Here is a POST example that I've made based on this curl command:
curl -X POST TheUrl/sendEmail
-H 'Accept: application/json' -H 'Content-Type: application/json'
-d '{"emailFrom": "smth#domain.com", "emailTo":
["smth#gmail.com"], "emailSubject": "Test email", "emailBody":
"708568", "generateQRcode": true}' -k
Here is how i did it using java
public void sendEmail(String url) {
try {
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
//add reuqest header
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json; utf-8");
con.setRequestProperty("Accept", "application/json");
con.setDoOutput(true);
// Send post request
JSONObject test = new JSONObject();
test.put("emailFrom", emailFrom);
test.put("emailTo", emailTo);
test.put("emailSubject", emailSubject);
test.put("emailBody", emailBody);
test.put("generateQRcode", generateQRcode);
String jsonInputString = test.toString();
System.out.println(jsonInputString);
System.out.println("Email Response:" + returnResponse(con, jsonInputString));
} catch (Exception e) {
System.out.println(e);
}
System.out.println("Mail sent");
}
public String returnResponse(HttpURLConnection con, String jsonInputString) {
try (OutputStream os = con.getOutputStream()) {
byte[] input = jsonInputString.getBytes("utf-8");
os.write(input, 0, input.length);
} catch (Exception e) {
System.out.println(e);
}
try (BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream(), "utf-8"))) {
StringBuilder response = new StringBuilder();
String responseLine = null;
while ((responseLine = br.readLine()) != null) {
response.append(responseLine.trim());
}
return response.toString();
} catch (Exception e) {
System.out.println("Couldnt read response from URL");
System.out.println(e);
return null;
}
}
I've found this useful link but i can't really understand how to use it in my example.
Is it any different from my example? and if yes how can i POST the following data?
Note: Required Data
HEADERS:
user_id myid
user_key mykey
PARAMS:
face_det 1
boxes 120,150,200,250 (this is optional)
BODY:
img_1
multipart/base64 encoded image or remote url of image
img_2
multipart/base64 encoded image or remote url of image
Here is the complete documentation of the API
There are three things that your HttpURLConnection needs:
The request method. You can set this with setRequestMethod.
The headers. You can set them with setRequestProperty.
The content type. The HTML specification requires that an HTTP request containing a form submission have application/x-www-form-urlencoded (or multipart/form-data) as its body’s content type. This is done by setting the Content-Type header using the setRequestProperty method, just like the other headers.
It’s not clear what you’re trying to do here. As Boris Verkhovskiy points out, curl’s --form option includes data as a part of a multipart request. In your command, the content of that request would be the characters of the URLs themselves. If you really want to submit URLs, not the images at those locations, you could use an application/x-www-form-urlencoded request body to do it. The body itself needs to URL-encoded, as the content type indicates. The URLEncoder class exists for this purpose.
The steps look like this:
String img1 = "https://cdn.dnaindia.com/sites/default/files/styles/full/public/2018/03/08/658858-577200-katrina-kaif-052217.jpg";
String img2 = "https://cdn.somethinghaute.com/wp-content/uploads/2018/07/katrina-kaif.jpg";
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setRequestProperty("user_id", myid);
con.setRequestProperty("user_key", thekey);
con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
String body =
"img_1=" + URLEncoder.encode(img1, "UTF-8") + "&" +
"img_2=" + URLEncoder.encode(img2, "UTF-8");
try (OutputStream os = con.getOutputStream()) {
byte[] input = body.getBytes(StandardCharsets.UTF_8);
os.write(input);
}
However, if you want to submit the actual images, you will need to create a MIME request body. Java SE cannot do this, but the MimeMultipart class of JavaMail, which is part of the Java EE specification, can.
Multipart multipart = new MimeMultipart("form-data");
BodyPart part;
part = new MimeBodyPart();
part.setDataHandler(new DataHandler(new URL(img1)));
multipart.addBodyPart(part);
part = new MimeBodyPart();
part.setDataHandler(new DataHandler(new URL(img2)));
multipart.addBodyPart(part);
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setRequestProperty("user_id", myid);
con.setRequestProperty("user_key", thekey);
con.setRequestProperty("Content-Type", multipart.getContentType());
try (OutputStream os = con.getOutputStream()) {
multipart.writeTo(os);
}
You should remove all catch blocks from your code, and amend your method signatures to include throws IOException (or throws IOException, MessagingException). You don’t want users of your application to think the operation was successful if in fact it failed, right?
I use command "curl -X DELETE --header 'Accept: application/json' 'http://10.10.1.29:8181/onos/v1/flows/application/olsrflow' -u karaf:karaf"
It Can work.
but use JAVA Code does't work.
have any problem in my JAVA code?
URL dc0ContrailUrl2 = new URL("http://10.10.1.29:8181/onos/v1/flows/application/olsrflow");
HttpURLConnection dcConn2 = (HttpURLConnection) dc0ContrailUrl2.openConnection();
dcConn2.setDoOutput(true);
String login = "karaf:karaf";
String content = URLEncoder.encode (login) ;
String basicAuth = "Basic " + new String(new Base64().encode(login.getBytes()));
dcConn2.setRequestProperty("Authorization",basicAuth);
dcConn2.setRequestProperty("Content-Type", "application/json");
dcConn2.setRequestMethod("DELETE");
BufferedReader in2 = new BufferedReader(new InputStreamReader(dcConn2.getInputStream()));
String inputLine2;
while ((inputLine2 = in2.readLine()) != null){ //while response is not null, assign response to inputLine and print inputLine
System.out.println(inputLine2);
}
in2.close();
error HTTP response code: 415
You did not add Accept type the same as curl. Add the following line and remove the Content-Type:
dcConn2.setRequestProperty("Accept", "application/json");
Typically it means that server does not support MediaType has been passed. You have to figure out what does it expect and setup it in your request accordingly.
i want to code a java http request for posting audio file (wav) for speech to text transformation.
I´m quite new to http requests, and could not find any useful hints how to achieve that.
In https://www.microsoft.com/cognitive-services/en-us/Speech-api/documentation/GetStarted/GetStarted-cURL I was able to get token (Step 1 in url), but now I am struggeling with step 2.
Can someone help or even provide java code for step 2 (post wav file to cognitive services).
CURL Request given:
curl -v -X POST "https://speech.platform.bing.com/recognize? scenarios=smd&appid=D4D52672-91D7-4C74-8AD8- 42B1D98141A5&locale=your_locale&device.os=your_device_os&version=3.0&format=json&instanceid=your_instance_id&requestid=your_request_id" -H 'Authorization: Bearer your_access_token' -H 'Content-type: audio/wav; codec="audio/pcm"; samplerate=16000' --data-binary #your_wave_file
My Java HTTP code so far (Using Apache):
HttpClient client = HttpClientBuilder.create().build();
String url = "https://speech.platform.bing.com/recognize";
String appId = "D4D52672-91D7-4C74-8AD8-42B1D98141A5"; //Always use this. See Docu
String token = "12345"; // received from step 1 (see MS documentation)
String locale = "de-DE";
String deviceOS = "Windows";
String version = "3.0";
String format = "json";
String instanceid = UUID.randomUUID().toString();
String scenarios = "smd";
// setting up post parameters
Map<String, String> postParameters = new HashMap<>();
postParameters.put("scenarios", scenarios);
postParameters.put("appid", appId);
postParameters.put("locale", locale);
postParameters.put("device.os", deviceOS);
postParameters.put("format", format);
postParameters.put("instanceid", instanceid);
postParameters.put("requestid", instanceid);
postParameters.put("version", version);
// setting up HttpPost
HttpPost httpPost = new HttpPost(url);
// PARAMETERS
List<NameValuePair> qparams = new ArrayList<>();
for (Map.Entry<String, String> s : postParameters.entrySet()) {
qparams.add(new BasicNameValuePair(s.getKey(), s.getValue()));
}
httpPost.setEntity(new UrlEncodedFormEntity(qparams));
// HEADERS
String wavFile = "C:\\Folder\\AudioData.wav";
Map<String, String> postHeaders = new HashMap<>();
postHeaders.put("Authorization", "Bearer " + token);
postHeaders.put("Content-Type", "audio/wav; codec=\"audio/pcm\"; samplerate=16000");
for (Map.Entry<String, String> entry : postHeaders.entrySet()) {
httpPost.setHeader(entry.getKey(), entry.getValue());
}
// WAV FILE
File file = new File(wavFile);
FileBody bin = new FileBody(file, ContentType.DEFAULT_BINARY);
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.addPart("bin", bin);
HttpEntity reqEntity = builder.build();
httpPost.setEntity(reqEntity);
// RESPONSE
HttpResponse response = client.execute(httpPost);
int responseCode = response.getStatusLine().getStatusCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuilder result = new StringBuilder();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
My Response code is 200, but my response.getEntity().getContent() is empty. I expect a JSON there.
btw: if i fire this up with curl (with set parameters of course) it works, and I get a JSON back with recognized speech to text.
Can you help me with that?
Nycon
I am trying to submit a json post request using HttpURLConnection in Scala. I followed along to two tutorials and produced this:
def sendPost(url: String, jsonHash: String) {
val conn: HttpURLConnection = new URL(url).openConnection().asInstanceOf[HttpURLConnection]
conn.setRequestMethod("POST")
conn.setRequestProperty("Content-Type", "application/json")
conn.setRequestProperty("Accept", "application/json")
conn.setDoOutput(true)
conn.connect()
val wr = new DataOutputStream(conn.getOutputStream)
wr.writeBytes(jsonHash)
wr.flush()
wr.close()
val responseCode = conn.getResponseCode
println("Sent: " + jsonHash + " to " + url + " received " + responseCode)
val in = new BufferedReader(new InputStreamReader(conn.getInputStream))
var response: String = ""
while(response != null) {
response = in.readLine()
println(response)
}
in.close()
}
It responds with:
Sent: '{"schedule":"R/2014-02-02T00:00:00Z/PT24H", "name":"Scala-Post-Test", "command":"which scalac", "epsilon":"PT15M", "owner":"myemail#thecompany.com", "async":false}' to http://localhost:4040/scheduler/iso8601 received 500
java.io.IOException: Server returned HTTP response code: 500 for URL: http://localhost:4040/scheduler/iso8601
stemming from
val in = new BufferedReader(new InputStreamReader(conn.getInputStream))
but if I rebuild it as a curl request, it works fine:
curl -X POST -H 'Content-Type: application/json' -d '{"schedule":"R/2014-02-02T00:00:00Z/PT24H", "name":"Scala-Post-Test", "command":"which scalac", "epsilon":"PT15M", "owner":"myemail#thecompany.com", "async":false}' http://localhost:4040/scheduler/iso8601
requirement failed: Vertex already exists in graph Scala-Post-Test
(which is what I expect)
Any insight to what is wrong? I'm trying to sniff the packets now to determine what is different.
(Note: I had previously given up on sys.process._)
The issue is here:
Sent: '{"schedule":"R/2014-02-02T00:00:00Z/PT24H", "name":"Scala-Post-Test", "command":"which scalac", "epsilon":"PT15M", "owner":"myemail#thecompany.com", "async":false}' to http://localhost:4040/scheduler/iso8601 received 500
You'll note your JSON is surrounded by single quotes. This makes it invalid.
Also worth noting is that while this code works, you are using a DataOutputStream.writeBytes() to output your data. This would be problematic if your string including anything but single-byte characters; it strips the high 8 bits off each char (Java uses 2-byte chars to hold UTF-16 codepoints).
It's better to use something more suited for String output. The same technique you use for input, for example:
BufferedWriter out =
new BufferedWriter(new OutputStreamWriter(conn.getOutputStream));
out.write(jsonString);
out.close();