Develop Reference

How to write / convert CURL for Android java

I am trying to implement the MOT history API https://dvsa.github.io/mot-history-api-documentation/ and they give an example using CURL which works with the supplied api key successfully when using an online CURL tool.
I am trying to implement this in Android and realise I have to use something like HttpPost rather than CURL, this is my code:
//Tried with full URL and by adding the registration as a header.
//HttpPost httpPost = new HttpPost("https://beta.check-mot.service.gov.uk/trade/vehicles/mot-tests?registration=" + reg_selected);
HttpPost httpPost = new HttpPost("https://beta.check-mot.service.gov.uk/trade/vehicles/mot-tests");
httpPost.addHeader("Content-Type", "application/json");
httpPost.addHeader("Accept", "application/json+v6");
httpPost.addHeader("x-api-key", "abcdefgh123456");
httpPost.addHeader("registration", reg_selected);
StringEntity entity = new StringEntity(jsonObj.toString(), HTTP.UTF_8);
httpPost.setEntity(entity);
HttpClient client = new DefaultHttpClient();
try {
HttpResponse response = client.execute(httpPost);
if (response.getStatusLine().getStatusCode() == 200) {
InputStream inputStream = response.getEntity().getContent();
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
String readLine = bufferedReader.readLine();
String jsonStr = readLine;
JSONObject myJsonObj = new JSONObject(jsonStr);
}else if (response.getStatusLine().getStatusCode() == 400){
//Bad Request Invalid data in the request. Check your URL and parameters
error_text = "Bad Request";
}else if (response.getStatusLine().getStatusCode() == 403){
//Unauthorised – The x-api-key is missing or invalid in the header
error_text = "Authentication error"; //<<<< FAILS HERE 403
}
response.getStatusLine().getStatusCode() returns • "403 – Unauthorised – The x-api-key is missing or invalid in the header".
However the x-api-key that I use works correctly with the online CURL test so the actual key is correct but how I am adding it to my android code request must be invalid or similar.
Can anyone throw any light as to the correct way to convert the CURL into Android java so that the server does not return 403?
Thanks

It's easy to do with Jsoup:
// CREATE CONNECTION
Connection conn=Jsoup.connect("URL_GOES_HERE");
// ADD POST/FORM DATA
conn.data("KEY", "VALUE");
// ADD HEADERS HERE
conn.header("KEY", "VALUE");
// SET METHOD AS POST
conn.method(Connection.Method.POST);
// ACCEPT RESPONDING CONTENT TYPE
conn.ignoreContentType(true);
try
{
// GET RESPONSE
String response = conn.execute().body();
// USE RESPONSE HERE
// CREATE JSON OBJECT OR ANYTHING...
} catch(HttpStatusException e)
{
int status = e.getStatusCode();
// HANDLE HTTP ERROR HERE
} catch (IOException e)
{
// HANDLE IO ERRORS HERE
}
Ps: I guess you are confused with Header and Post Data. The key etc (Credentials) must be used as Post Data and Content Type etc as Header.

Uploading a very large JSON file via REST using Java

I have a very large json file (about 500MB) that I am trying to upload using REST and java. It works using curl like this -
curl -H "content-type: application/json" --data-binary #2018-02-28.json http://md01:8086/Gateway/rest/gateway-service/ABC/invocations
However, I get "Software caused connection abort: socket write error" when I do it using REST like this-
String filePath = "C:\\2018-02-28.json";
String filename = "2018-02-28.json";
File uploadedFile = new File("C:\\2018-02-28.json");
try {
// HttpClient httpclient = HttpClientBuilder.create().build();
String authHeader = authToken();
HttpEntity entity = MultipartEntityBuilder
.create()
.addTextBody("name", "fileDate")
.addTextBody("fileName", filename)
.addTextBody("Content-Type", "application/json")
.addBinaryBody("fileData", new File(filePath), ContentType.create("application/json"), filename)
.build();
HttpClient httpClient = HttpClients.custom()
.setConnectionTimeToLive(2700, TimeUnit.SECONDS)
.setMaxConnTotal(400).setMaxConnPerRoute(400)
.setDefaultRequestConfig(RequestConfig.custom()
.setSocketTimeout(30000).setConnectTimeout(5000).build())
.setRetryHandler(new DefaultHttpRequestRetryHandler(5, true))
.build();
HttpPost request = new HttpPost("http://md:8086/InputGateway/rest/input-gateway-service/ABC/invocations");
request.setEntity(entity);
request.setHeader(HttpHeaders.AUTHORIZATION, authHeader);
HttpResponse resp = httpClient.execute(request);
HttpEntity entity1 = resp.getEntity();
System.out.println(EntityUtils.toString(entity1, "utf-8"));
System.out.println("File has been Uploaded successfully: " + uploadedFile);
} catch (Exception ex) {
throw new Exception( ex.toString());
}
}
what am I doing wrong here

curl -H "content-type: application/json" --data-binary #2018-02-28.json http://md01:8086/Gateway/rest/gateway-service/ABC/invocations
One thing you may want to do is run this command in verbose mode, to see what is really happening. It is possible that the example that works because curl is using an Expect header that allows the server to prepare for the data dump.
A packet analyzer, to see if the remote server is actually sending you a RST, or if something else is going on in the network stack.
(Obvious, but just in case: try using the same code to send a smaller file. Make sure that the size it the limiting factor. Doing a binary search to find out how big a file is acceptable might provide an extra clue.)

How to use docker remote api to create container In java?

I want to using docker remote api in this.
I have succeed in this command to create container:
curl -v -X POST -H "Content-Type: application/json" -d '{"Image": " registry:2",}' http://192.168.59.103:2375/containers/create?name=test_remote_reg
Then,I use HttpClient(4.3.1) in java to try to create container via this code:
String containerName = "test_remote_reg";
String imageName = "registry:2";
String url = DockerConfig.getValue("baseURL")+"/containers/create?name="+containerName;
List<NameValuePair> ls = new ArrayList<NameValuePair>();
ls.add(new BasicNameValuePair("Image",imageName));
UrlEncodedFormEntity fromEntity = new UrlEncodedFormEntity(ls, "uTF-8");
HttpPost post = new HttpPost(url);
post.setHeader("Content-Type", " application/json");
if(null!=fromEntity) post.setEntity(fromEntity);
HttpClient client = new DefaultHttpClient();
client.execute(post);
It didn't work, and throw error:
invalid character 'I' looking for beginning of value
I just add header information and add param pair about "Image:test_remote_reg".
What is wrong about my java code? What difference is between they? What should I edit for my java code?

Considering this is a json call, you could use one of the answers of HTTP POST using JSON in Java, like (replace the JSON part by your JSON parameters):
HttpClient httpClient = new DefaultHttpClient();
try {
HttpPost request = new HttpPost("http://yoururl");
StringEntity params =new StringEntity("details={\"name\":\"myname\",\"age\":\"20\"} ");
request.addHeader("content-type", "application/json");
request.addHeader("Accept","application/json");
request.setEntity(params);
HttpResponse response = httpClient.execute(request);
// handle response here...
}catch (Exception ex) {
// handle exception here
} finally {
httpClient.getConnectionManager().shutdown();
}

Android JAVA $_FILE returms empty at files bigger than ~ 50 mb

Problem: Trying to upload bigger than ~ 50 mb files, $_FILES return empty array.
I try change limits on php (a lot of combination) but it useleass..
my class at java:
public void upload() throws Exception {
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(this.url);
try {
FileBody bin = new FileBody(this.file);
#SuppressWarnings("deprecation")
MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
reqEntity.addPart("file", bin);
httppost.setEntity(reqEntity);
System.out.println("Requesting : " + httppost.getRequestLine());
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpclient.execute(httppost, responseHandler);
System.out.println("responseBody : " + responseBody);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
httpclient.getConnectionManager().shutdown();
}
}
my php code to print array for respawn:
<?php
ini_set('file_uploads', 1);
ini_set('max_file_uploads', '9999999999M');
ini_set('post_max_size', '2000M');
ini_set('max_input_time', 3600);
ini_set('max_execution_time', 3600);
print_r($_FILES);
?>

You could try to add the mime type of the file when you create the FileBody. For example:
FileBody bin = new FileBody(this.file, "application/octect-stream");
Another solution is to use the HttpURLConnection client that is easier to use and less error prone. Also, Google recommends to use it for new applications.

One side or the other is breaking the http connection being used by the POST action.
Do you know which side ( client or server )
If you do not know , maybe you should try to rule out the server by using CURL to do the M-Multipart Post of one of your big ( +50 ) files.
Invoke CURL to upload and add the -v switch for verbose log so you can analyse the process.
Android POST action on files of 50 - 100 meg should work fine.
Since you are doing big uploads , you would want a client that you can send headers "expect 100 Continue" and where you can employ chunked transfer encoding along with a client side thread that can provide data to a buffer that is wrapped by something like a byteArrayEntity
apache or nginX on std config should handle this fine. If your php process is abending then maybe you need to check phpinfo.

twitter streaming api - set filter in http post request (apache httpcomponents)

I'm trying out the twitter streaming api. I could succesfully filter tweets by using curl, as stated here:
curl -d #tracking http://stream.twitter.com/1/statuses/filter.json -u <user>:<pass>
where tracking is a plain file with the content:
track=Berlin
Now I tried to do the same thing in JavaSE, using Apache's HTTPComponents:
UsernamePasswordCredentials creds = new UsernamePasswordCredentials(<user>, <pass>);
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getCredentialsProvider().setCredentials(AuthScope.ANY, creds);
HttpPost httpPost = new HttpPost("http://stream.twitter.com/1/statuses/filter.json");
HttpParams params = new BasicHttpParams();
params = params.setParameter("track", "Berlin");
httpPost.setParams(params);
try {
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity entity = httpResponse.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
String t;
BufferedReader br = new BufferedReader(new InputStreamReader(instream));
while(true) {
t = br.readLine();
if(t != null) {
linkedQueue.offer(t);
}
}
}
} catch (IOException ioe) {
System.out.println(ioe.getMessage());
}
finally{
httpClient.getConnectionManager().shutdown();
}
When I run that, I get:
No filter parameters found. Expect at least one parameter: follow track
as a single entry in my linkedQueue. Seems the api wants the parameter in a different form, but cannot find any hint in the documentation. Can somebody share some experiences with the api or see any other problem with the code? Thanks!
EDIT
Putting the filter parameter into the params was a bad idea. As it's post data, it needs to be defined as an Entity before the request is being made:
StringEntity postEntity = new StringEntity("track=Berlin", "UTF-8");
postEntity.setContentType("application/x-www-form-urlencoded");
httpPost.setEntity(postEntity);
That's what I was doing wrong. Thanks Brian!

I suspect you need to post the data as the contents of your HTTP post. The man page for curl -d says:
(HTTP) Sends the specified data in a
POST request to the HTTP server, in
the same way that a browser does when
a user has filled in an HTML form and
presses the submit button. This will
cause curl to pass the data to the
server using the content-type
application/x-www-form-urlencoded.
so I believe you have to set that content type and put the contents of the tracking file in the body of your post.