Develop Reference

Converting HTTP POST from curl (PHP) to HttpURLConnection (Java)

I tried to convert the below PHP code (taken from https://www.cryptocoincharts.info/tools/api) to java
// define pairs
$post = array("pairs" => "ltc_usd,ppc_btc");
// fetch data
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "http://api.cryptocoincharts.info/tradingPairs");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, $post);
$rawData = curl_exec($curl);
curl_close($curl);
// decode to array
$data = json_decode($rawData);
// show data
echo "<pre>";
foreach ($data as $row)
{
echo "Price of ".$row->id.": ".$row->price."\n";
echo "Trade this pair on ".$row->best_market."\n";
}
echo "</pre>";
Java Code
URL url = new URL("http://api.cryptocoincharts.info/tradingPairs");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
// CURLOPT_POST
con.setRequestMethod("POST");
// CURLOPT_FOLLOWLOCATION
con.setInstanceFollowRedirects(true);
String postData = "ltc_usd,ppc_btc";
con.setRequestProperty("Content-length", String.valueOf(postData.length()));
con.setDoOutput(true);
con.setDoInput(true);
DataOutputStream output = new DataOutputStream(con.getOutputStream());
output.writeBytes(postData);
output.close();
// "Post data send ... waiting for reply");
int code = con.getResponseCode(); // 200 = HTTP_OK
System.out.println("Response (Code):" + code);
System.out.println("Response (Message):" + con.getResponseMessage());
// read the response
DataInputStream input = new DataInputStream(con.getInputStream());
int c;
StringBuilder resultBuf = new StringBuilder();
while ( (c = input.read()) != -1) {
resultBuf.append((char) c);
}
input.close();
System.out.println("resultBuf.toString() " + resultBuf.toString());
As per the API, after converting this to java I should get only the details of LTC and PPC details. Instead I am getting a strange Json with all trading pairs.
2 $post = array("pairs" => "ltc_usd,ppc_btc"); Posted the PHP code as I am not known the exact equivalent in Java
Could you please point out if my conversion from PHP to Java is correct ?

As far as I see, the main difference between the two implementation is related to the $post variable.
In the PHP implementation $post is a key/value array but in Java I only see the value part.
I suggest to change the postData variable content into pairs=ltc_usd,ppc_btc

You didn't mentioned key part, only value is mentioned. And when we fetch data from PHP API, we have an associative array. If u want to display the output, u need to know the key and value of the particular associative array.
And the InputStream and OutputStream should be inside try-resources

you can try curl-to-java lib to convert curl php code to java code
https://github.com/jeffreyning/curl-to-java
demo like this
public Object curl(String url, Object postData, String method) {
CurlLib curl = CurlFactory.getInstance("default");
ch = curl.curl_init();
curl.curl_setopt(ch, CurlOption.CURLOPT_CONNECTTIMEOUT, 1000);
curl.curl_setopt(ch, CurlOption.CURLOPT_TIMEOUT, 5000);
curl.curl_setopt(ch, CurlOption.CURLOPT_SSL_VERIFYPEER, false);
curl.curl_setopt(ch, CurlOption.CURLOPT_SSL_VERIFYHOST, false);
String postDataStr = "key1=v1";
curl.curl_setopt(ch, CurlOption.CURLOPT_CUSTOMREQUEST, "POST");
curl.curl_setopt(ch, CurlOption.CURLOPT_POSTFIELDS, postDataStr);
curl.curl_setopt(ch, CurlOption.CURLOPT_URL, "https://xxxx.com/yyy");
Object html = curl.curl_exec(ch);
Object httpCode = curl.curl_getinfo(ch, CurlInfo.CURLINFO_HTTP_CODE);
if (httpCode != null && 200 == Integer.valueOf(httpCode.toString())) {
return null;
}
return html;
}

java api to add storage plugin in apache drill

I want to add a new storage plugin using java code. Currently I am creating a json file and uploading it on drill web ui. But it fails. here is my code
def creatplugin() {
val httpclient = new DefaultHttpClient()
val httpPost = new HttpPost("http://ip:port/storage/hdfs1.json")
val uploadFilePart = new FileBody(new File("D:/plugin.json"))
val reqEntity = new MultipartEntity()
reqEntity.addPart("hdfs1.json", uploadFilePart)
httpPost.setEntity(reqEntity)
httpPost.setHeader("Content-type", "application/json")
val response = httpclient.execute(httpPost)
println(response.getStatusLine().getStatusCode())
}
In this case response code is 400 with bad request.
Any suggestion, what's going wrong? is there any other way to add plugin dynamically using java code instead of rest api?
Thanks

The problem was multipart entity as pointed by Jim. Here is working code
def creatplugin() {
val source = scala.io.Source.fromFile("D:/plugin.json").mkString
val httpclient = new DefaultHttpClient()
val httpPost = new HttpPost("http://ip:port/storage/hdfs1.json")
val reqEntity = new StringEntity(source)
httpPost.setEntity(reqEntity)
httpPost.setHeader("content-type", "application/json")
httpPost.setHeader("Accept", "application/json")
val response = httpclient.execute(httpPost)
println(response.getStatusLine().getStatusCode())
}

I would think it may be the multipart entry. I would just post the json data as part of the body of the post message. Here is an example curl that works.Use a StringEntity instead.
curl -X POST -H "Authorization: Basic bWFwcjpyb290NG1hcHI=" -H "Content-Type: application/json" -d '{"name":"nfl","config":{"type":"file","enabled":true,"connection":"maprfs:///","workspaces":{"views":{"location":"/mapr/demo.mapr.com/data/views","writable":true,"defaultInputFormat":null},"json":{"location":"/mapr/demo.mapr.com/data/nfl/json","writable":false,"defaultInputFormat":"json"},"csv":{"location":"/mapr/demo.mapr.com/data/nfl/csv","writable":false,"defaultInputFormat":"csv"},"tab":{"location":"/mapr/demo.mapr.com/data/nfl/txt","writable":false,"defaultInputFormat":"tsv"},"xml":{"location":"/mapr/demo.mapr.com/data/nfl/xml","writable":false,"defaultInputFormat":null}},"formats":{"csv":{"type":"text","extensions":["csv"],"delimiter":","},"tsv":{"type":"text","extensions":["tsv","txt"],"delimiter":"\t"},"json":{"type":"json"}}}}' http://maprdemo:8047/storage/nfl.json

'Improperly formatted request' error after switching from Commons HttpClient to HttpComponents

I've some code, sending a multipart/form-data request to an API. Using Apache's commons-httpclient 3.1 it works, however switching over to httpclient 4.3.5, I face problems with the API. Below you can find both code samples. Since it has to do with the Salesforce API, I've also posted a question to SFSE, since I'm still not sure if it's a problem on my or their side. However, my question here is: Have I migrated the code to 4.3.5 correctly? If yes, is there anything which changed in httpclient's behavior related to executing multipart/form-data requests?
Code samples follow:
commons-httpclient 3.1
String json = "{ \"body\":{ \"messageSegments\":[ { \"type\":\"Text\", \"text\":\"Here is another receipt.\" } ] }, \"capabilities\":{ \"content\":{ \"title\":\"receipt2\"} } }";
PostMethod filePost = new PostMethod("https://eu3.salesforce.com/services/data/v32.0/chatter/feed-elements/<some_feed_element_id>/capabilities/comments/items");
filePost.addRequestHeader("Authorization", token());
StringPart jsonPart = new StringPart("json", json);
jsonPart.setContentType(ContentType.APPLICATION_JSON.getMimeType());
FilePart filePart = new FilePart("feedElementFileUpload", file);
filePart.setContentType(ContentType.APPLICATION_OCTET_STREAM.getMimeType());
Part[] parts = { jsonPart, filePart };
filePost.setRequestEntity(new MultipartRequestEntity(parts, filePost.getParams()));
int response = httpclient.executeMethod(filePost);
Wire / context logs: http://pastebin.com/RCg20Ygn
httpclient 4.3.5
String json = "{ \"body\":{ \"messageSegments\":[ { \"type\":\"Text\", \"text\":\"Here is another receipt.\" } ] }, \"capabilities\":{ \"content\":{ \"title\":\"receipt2\"} } }";
String attachmentName = "package.xml";
CloseableHttpClient client = HttpClientBuilder
.create()
.setDefaultHeaders(Lists.newArrayList())
.build();
HttpPost post = new HttpPost(
"https://eu3.salesforce.com/services/data/v32.0/chatter/feed-elements/<feed_element_id>/capabilities/comments/items"
);
post.addHeader(HttpHeaders.AUTHORIZATION, token());
post.addHeader(HttpHeaders.CONTENT_TYPE, ContentType.MULTIPART_FORM_DATA.getMimeType());
post.setEntity(
MultipartEntityBuilder.create()
.setStrictMode()
.addPart(
"json",
new StringBody(
json,
ContentType.APPLICATION_JSON
)
)
.addPart(
"feedElementFileUpload",
new FileBody(
new File(attachmentName),
ContentType.APPLICATION_OCTET_STREAM,
attachmentName
)
)
.build()
);
CloseableHttpResponse response = client.execute(post);
Wire / context logs: http://pastebin.com/EHXd1y50
UPDATE 1:
I've tried all three available modes for MultipartEntityBuilder (STRICT, BROWSER_COMPATIBLE, RFC6532), but it still doesn't work.

Try using 'browser compatible' mode instead of 'strict' when constructing the request entity with MultipartEntityBuilder
UPDATE 1:
"Content-Type: multipart/form-data[\r][\n]"
This is clearly wrong (boundary attribute is missing) and likely to be reason for the request being rejected.
Please remove this line and try again
post.addHeader(HttpHeaders.CONTENT_TYPE, ContentType.MULTIPART_FORM_DATA.getMimeType());

Android multipart upload + blueimp UploadHandler: filetype not allowed

I'm trying to get my Android multipart image upload to work with the default blueimp PHP UploadHandler, which I'm using for the web version of my app.
It didn't work out of the box, so I experimented with the header of the requests.
I played around with it for quite a while, but since nothing worked I ended up firing up Wireshark and comparing the packets of the blueimp demo and the ones that are sent by the Android app.After some further tweaking, they look exactly the same (well the important parts):
Blueimp demo: http://i.imgur.com/T9styyR.png
Android MultipartEntity: http://i.imgur.com/OuDuyJ0.png
This is what my customized generate_response function returns:
Array
(
[0] => stdClass Object
(
[name] => 1387215600-9207
[size] => 97894
[type] => multipart/form-data; boundary=-----Don'tmindmeI'mjustaboundary
[error] => Filetype not allowed
)
)
It seems to load the whole packet instead of just it's multipart part.
Is there any way to fix that?
Here's how I create the request (not quite but these are the relevant lines):
public static String BOUNDARY = "-----Don'tmindmeI'mjustaboundary";
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE, BOUNDARY, null);
File file = new File(params[2]); //params[2] contains the absolute path + filename
String filename = file.getName();
entity.addPart(filename, new FileBody(file, getMimeType(params[2]))); //getMimeType returns the MIME Type of the image e.g. image/png
HttpPost request = new HttpPost(params[0]); //params[0] contains the URL
request.setHeader("Content-Type", "multipart/form-data; boundary="+BOUNDARY);
request.setEntity(entity);
HttpResponse response = client.execute(request, context);
I've been working on this for hours and it's really driving me nuts.
What am I doing wrong?
Edit:
Nevermind I got it to work
I forgot to set the name of the form to files[], which is what the UploadHandler is looking for ...
So now it looks like this:
...
public static String BOUNDARY = "-----Don'tmindmeI'mjustaboundary";
...
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE, BOUNDARY, null);
File file = new File(params[2]);
String filename = file.getName();
FileBody body = new FileBody(file, getMimeType(params[2]));
FormBodyPart part = new FormBodyPart("files[]", body);
part.addField("Content-Type", getMimeType(params[2]));
part.addField("Content-Disposition", "form-data; filename=\""+filename+"\"");
entity.addPart(part);
HttpPost request = new HttpPost(params[0]);
request.setHeader("Content-Type", "multipart/form-data; boundary="+BOUNDARY);
request.addHeader("Accept", "application/json, text/javascript, */*; q=0.01");
request.addHeader("Accept-Language", "en-US,en;q=0.8,de;q=0.6,es;q=0.4");
request.setEntity(entity);
HttpResponse response = client.execute(request, context);
...

How to POST to node.js from applet?

I have an applet (not our choice, it's the MarioAI engine) that I'd like to connect to a node.js app that uses express...but I can't seem to get mongodb to accept the values I'm sending in my POST request through localhost. I keep getting 200 response from node, but 'undefined' from mongooose, which I suspect means the URLEncoder I'm using in Java is mangling the String I'm sending through somehow.
I read this:
Problem with Java Applet to connect our server to call a PHP file
and tried the following OutputStreamWriter call in Java:
//EvaluateFrustration() takes an int but should come back with a float value
String frustrationString = Double.toString(EvaluateFrustration(this.periods));
try {
URL url = new URL("http://127.0.0.1:8888/mario");
final HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
System.out.println(conn.getResponseCode());
conn.setUseCaches (false);
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
OutputStreamWriter writer;
writer = AccessController
.doPrivileged(new PrivilegedExceptionAction<OutputStreamWriter>() {
public OutputStreamWriter run() throws IOException {
return new OutputStreamWriter(conn.getOutputStream());
}
});
String data = URLEncoder.encode("frustrationValueFirstRound="
+ frustrationString,"UTF-8");
writer.write(data);
writer.flush();
} catch (Exception e) {
}
In the node app (using express and mongoose/mongodb), I wrote:
var express = require('express');
var mongoose = require('mongoose');
var Schema = mongoose.Schema
, ObjectId = Schema.ObjectId;
var ExperimentSchema = new Schema({
experiment : ObjectId
, frustrationValueFirstRound : Number
});
mongoose.connect('mongodb://localhost/mariopaper');
mongoose.model('Experiment', ExperimentSchema);
var Experiment = mongoose.model('Experiment');
app.post('/mario', function(req, res){
var exp = new Experiment();
exp.frustrationValueFirstRound = req.body.frustrationValueFirstRound;
exp.save(function(err){ if (err) { throw err; }
res.send('ok');
});
For reference, I'd like to point out that this curl call works just fine:
curl -d "frustrationValueFirstRound=99" http://localhost:8888/mario
Anyone have any ideas whether I've simply written the POST wrong in Java, or perhaps I'm missing something in how URLEncoder.encode() works?

I think that's because you don't have the body-parser node module, that "Parse incoming request bodies in a middleware before your handlers, available under the req.body property."
Try this (after installing body-parser module):
var express = require('express');
var mongoose = require('mongoose');
var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));
var Schema = mongoose.Schema
, ObjectId = Schema.ObjectId;
var ExperimentSchema = new Schema({
experiment : ObjectId
, frustrationValueFirstRound : Number
});
mongoose.connect('mongodb://localhost/mariopaper');
mongoose.model('Experiment', ExperimentSchema);
var Experiment = mongoose.model('Experiment');
app.post('/mario', function(req, res){
console.log(req.body); // Is there something here ?
var exp = new Experiment();
exp.frustrationValueFirstRound = req.body.frustrationValueFirstRound;
exp.save(function(err){ if (err) { throw err; }
res.send('ok');
});
source : https://github.com/expressjs/body-parser