I want to add a new storage plugin using java code. Currently I am creating a json file and uploading it on drill web ui. But it fails. here is my code
def creatplugin() {
val httpclient = new DefaultHttpClient()
val httpPost = new HttpPost("http://ip:port/storage/hdfs1.json")
val uploadFilePart = new FileBody(new File("D:/plugin.json"))
val reqEntity = new MultipartEntity()
reqEntity.addPart("hdfs1.json", uploadFilePart)
httpPost.setEntity(reqEntity)
httpPost.setHeader("Content-type", "application/json")
val response = httpclient.execute(httpPost)
println(response.getStatusLine().getStatusCode())
}
In this case response code is 400 with bad request.
Any suggestion, what's going wrong? is there any other way to add plugin dynamically using java code instead of rest api?
Thanks
The problem was multipart entity as pointed by Jim. Here is working code
def creatplugin() {
val source = scala.io.Source.fromFile("D:/plugin.json").mkString
val httpclient = new DefaultHttpClient()
val httpPost = new HttpPost("http://ip:port/storage/hdfs1.json")
val reqEntity = new StringEntity(source)
httpPost.setEntity(reqEntity)
httpPost.setHeader("content-type", "application/json")
httpPost.setHeader("Accept", "application/json")
val response = httpclient.execute(httpPost)
println(response.getStatusLine().getStatusCode())
}
I would think it may be the multipart entry. I would just post the json data as part of the body of the post message. Here is an example curl that works.Use a StringEntity instead.
curl -X POST -H "Authorization: Basic bWFwcjpyb290NG1hcHI=" -H "Content-Type: application/json" -d '{"name":"nfl","config":{"type":"file","enabled":true,"connection":"maprfs:///","workspaces":{"views":{"location":"/mapr/demo.mapr.com/data/views","writable":true,"defaultInputFormat":null},"json":{"location":"/mapr/demo.mapr.com/data/nfl/json","writable":false,"defaultInputFormat":"json"},"csv":{"location":"/mapr/demo.mapr.com/data/nfl/csv","writable":false,"defaultInputFormat":"csv"},"tab":{"location":"/mapr/demo.mapr.com/data/nfl/txt","writable":false,"defaultInputFormat":"tsv"},"xml":{"location":"/mapr/demo.mapr.com/data/nfl/xml","writable":false,"defaultInputFormat":null}},"formats":{"csv":{"type":"text","extensions":["csv"],"delimiter":","},"tsv":{"type":"text","extensions":["tsv","txt"],"delimiter":"\t"},"json":{"type":"json"}}}}' http://maprdemo:8047/storage/nfl.json
Related
I have a very large json file (about 500MB) that I am trying to upload using REST and java. It works using curl like this -
curl -H "content-type: application/json" --data-binary #2018-02-28.json http://md01:8086/Gateway/rest/gateway-service/ABC/invocations
However, I get "Software caused connection abort: socket write error" when I do it using REST like this-
String filePath = "C:\\2018-02-28.json";
String filename = "2018-02-28.json";
File uploadedFile = new File("C:\\2018-02-28.json");
try {
// HttpClient httpclient = HttpClientBuilder.create().build();
String authHeader = authToken();
HttpEntity entity = MultipartEntityBuilder
.create()
.addTextBody("name", "fileDate")
.addTextBody("fileName", filename)
.addTextBody("Content-Type", "application/json")
.addBinaryBody("fileData", new File(filePath), ContentType.create("application/json"), filename)
.build();
HttpClient httpClient = HttpClients.custom()
.setConnectionTimeToLive(2700, TimeUnit.SECONDS)
.setMaxConnTotal(400).setMaxConnPerRoute(400)
.setDefaultRequestConfig(RequestConfig.custom()
.setSocketTimeout(30000).setConnectTimeout(5000).build())
.setRetryHandler(new DefaultHttpRequestRetryHandler(5, true))
.build();
HttpPost request = new HttpPost("http://md:8086/InputGateway/rest/input-gateway-service/ABC/invocations");
request.setEntity(entity);
request.setHeader(HttpHeaders.AUTHORIZATION, authHeader);
HttpResponse resp = httpClient.execute(request);
HttpEntity entity1 = resp.getEntity();
System.out.println(EntityUtils.toString(entity1, "utf-8"));
System.out.println("File has been Uploaded successfully: " + uploadedFile);
} catch (Exception ex) {
throw new Exception( ex.toString());
}
}
what am I doing wrong here
curl -H "content-type: application/json" --data-binary #2018-02-28.json http://md01:8086/Gateway/rest/gateway-service/ABC/invocations
One thing you may want to do is run this command in verbose mode, to see what is really happening. It is possible that the example that works because curl is using an Expect header that allows the server to prepare for the data dump.
A packet analyzer, to see if the remote server is actually sending you a RST, or if something else is going on in the network stack.
(Obvious, but just in case: try using the same code to send a smaller file. Make sure that the size it the limiting factor. Doing a binary search to find out how big a file is acceptable might provide an extra clue.)
The below code is working fine to attach file in JIRA, only one problem is here
I can't use MultipartEntityBuilder as it needed to add new dependency in pom and that is not permissible , can any one please suggest which basic API I can use there? thanks in advance
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpPost postRequest = new HttpPost("https://xxxx.zzzz.net/rest/api/2/issue/" + issueID +"/attachments");
postRequest.setHeader("Authorization", "Basic <AUTHSTRING>");
postRequest.setHeader("X-Atlassian-Token", "nocheck");
File file = new File("C:\\Users\\MKumar\\Desktop\\Oauth_JIRA.rtf");
URL url = new URL("C:\\Users\\MKumar\\Desktop\\Oauth_JIRA.rtf");
MultipartEntityBuilder builder = MultipartBodyBuilder.create();
// This attaches the file to the POST:
builder.addBinaryBody(
"file",
new FileInputStream(file),
ContentType.MULTIPART_FORM_DATA,
file.getName()
);
HttpEntity multipart = builder.build();
postRequest.setEntity(multipart);
HttpResponse response = httpClient.execute(postRequest);
org.apache.http.entity.mime.MultipartEntity is deprecated as a result you'll have to use MultipartEntityBuilder.
For more related post please see this thread
I want to using docker remote api in this.
I have succeed in this command to create container:
curl -v -X POST -H "Content-Type: application/json" -d '{"Image": " registry:2",}' http://192.168.59.103:2375/containers/create?name=test_remote_reg
Then,I use HttpClient(4.3.1) in java to try to create container via this code:
String containerName = "test_remote_reg";
String imageName = "registry:2";
String url = DockerConfig.getValue("baseURL")+"/containers/create?name="+containerName;
List<NameValuePair> ls = new ArrayList<NameValuePair>();
ls.add(new BasicNameValuePair("Image",imageName));
UrlEncodedFormEntity fromEntity = new UrlEncodedFormEntity(ls, "uTF-8");
HttpPost post = new HttpPost(url);
post.setHeader("Content-Type", " application/json");
if(null!=fromEntity) post.setEntity(fromEntity);
HttpClient client = new DefaultHttpClient();
client.execute(post);
It didn't work, and throw error:
invalid character 'I' looking for beginning of value
I just add header information and add param pair about "Image:test_remote_reg".
What is wrong about my java code? What difference is between they? What should I edit for my java code?
Considering this is a json call, you could use one of the answers of HTTP POST using JSON in Java, like (replace the JSON part by your JSON parameters):
HttpClient httpClient = new DefaultHttpClient();
try {
HttpPost request = new HttpPost("http://yoururl");
StringEntity params =new StringEntity("details={\"name\":\"myname\",\"age\":\"20\"} ");
request.addHeader("content-type", "application/json");
request.addHeader("Accept","application/json");
request.setEntity(params);
HttpResponse response = httpClient.execute(request);
// handle response here...
}catch (Exception ex) {
// handle exception here
} finally {
httpClient.getConnectionManager().shutdown();
}
I've some code, sending a multipart/form-data request to an API. Using Apache's commons-httpclient 3.1 it works, however switching over to httpclient 4.3.5, I face problems with the API. Below you can find both code samples. Since it has to do with the Salesforce API, I've also posted a question to SFSE, since I'm still not sure if it's a problem on my or their side. However, my question here is: Have I migrated the code to 4.3.5 correctly? If yes, is there anything which changed in httpclient's behavior related to executing multipart/form-data requests?
Code samples follow:
commons-httpclient 3.1
String json = "{ \"body\":{ \"messageSegments\":[ { \"type\":\"Text\", \"text\":\"Here is another receipt.\" } ] }, \"capabilities\":{ \"content\":{ \"title\":\"receipt2\"} } }";
PostMethod filePost = new PostMethod("https://eu3.salesforce.com/services/data/v32.0/chatter/feed-elements/<some_feed_element_id>/capabilities/comments/items");
filePost.addRequestHeader("Authorization", token());
StringPart jsonPart = new StringPart("json", json);
jsonPart.setContentType(ContentType.APPLICATION_JSON.getMimeType());
FilePart filePart = new FilePart("feedElementFileUpload", file);
filePart.setContentType(ContentType.APPLICATION_OCTET_STREAM.getMimeType());
Part[] parts = { jsonPart, filePart };
filePost.setRequestEntity(new MultipartRequestEntity(parts, filePost.getParams()));
int response = httpclient.executeMethod(filePost);
Wire / context logs: http://pastebin.com/RCg20Ygn
httpclient 4.3.5
String json = "{ \"body\":{ \"messageSegments\":[ { \"type\":\"Text\", \"text\":\"Here is another receipt.\" } ] }, \"capabilities\":{ \"content\":{ \"title\":\"receipt2\"} } }";
String attachmentName = "package.xml";
CloseableHttpClient client = HttpClientBuilder
.create()
.setDefaultHeaders(Lists.newArrayList())
.build();
HttpPost post = new HttpPost(
"https://eu3.salesforce.com/services/data/v32.0/chatter/feed-elements/<feed_element_id>/capabilities/comments/items"
);
post.addHeader(HttpHeaders.AUTHORIZATION, token());
post.addHeader(HttpHeaders.CONTENT_TYPE, ContentType.MULTIPART_FORM_DATA.getMimeType());
post.setEntity(
MultipartEntityBuilder.create()
.setStrictMode()
.addPart(
"json",
new StringBody(
json,
ContentType.APPLICATION_JSON
)
)
.addPart(
"feedElementFileUpload",
new FileBody(
new File(attachmentName),
ContentType.APPLICATION_OCTET_STREAM,
attachmentName
)
)
.build()
);
CloseableHttpResponse response = client.execute(post);
Wire / context logs: http://pastebin.com/EHXd1y50
UPDATE 1:
I've tried all three available modes for MultipartEntityBuilder (STRICT, BROWSER_COMPATIBLE, RFC6532), but it still doesn't work.
Try using 'browser compatible' mode instead of 'strict' when constructing the request entity with MultipartEntityBuilder
UPDATE 1:
"Content-Type: multipart/form-data[\r][\n]"
This is clearly wrong (boundary attribute is missing) and likely to be reason for the request being rejected.
Please remove this line and try again
post.addHeader(HttpHeaders.CONTENT_TYPE, ContentType.MULTIPART_FORM_DATA.getMimeType());
I am using Grails Plugin rest=0.7 for consuming a Rest Webservice.
Everything works fine when the response from the service is xml but in case if response is file type like pdf it must start downloading on sending the request but the downloading is not starting at all.
The below code in implemented in a grails service.
String httpUrl = 'http://abc.com/myService'
String data = '<methodcall protocol="2" method="avalidmethodname"><cmdid/><data><project_id>1</project_id><user_id>2</user_id><operation>ABC</operation><filter><status_type_id>1</status_type_id><scope_bits>00</scope_bits></filter></data></methodcall>'
String respText = ''
HttpClient httpClient = new DefaultHttpClient()
HttpResponse response
try {
HttpPost httpPost = new HttpPost(httpUrl)
httpPost.setHeader("Content-Type", "text/xml")
HttpEntity reqEntity = new StringEntity(data, "UTF-8")
reqEntity.setContentType("text/xml")
reqEntity.setChunked(true)
httpPost.setEntity(reqEntity)
response = httpClient.execute(httpPost)
// HttpEntity resEntity = response.getEntity()
// respText = resEntity.getContent().text
}
finally {
httpClient.getConnectionManager().shutdown()
}
return response
// return respText
The commented lines in code is for the case of xml response.
Please help me to resolve this problem, i am not sure the approach i am using is valid in case of file response from the webservice.
Try adding
response.setContentType("text/pdf");
before
response = httpClient.execute(httpPost);