Develop Reference

Upload report to BA server with Pentaho PDI (Kettle)

I am currently working on a Pentaho PDI project where I need to upload a report to the BA/BI server repository (URL: ***/api/repo/publish/file). I would like to achieve this by using the HTTP Post step and an User Defined Java Class step that produces the request entity field.
However I didn't manage to come up with a working code. Since my boss does not want me to use external libraries, I am sticking to the org.apache.commons.httpclient classes which are deployed with kettle.
My approach is to create a Part[] array containing the FilePart and StringParts. The next step is to create a MultipartRequestEntity which is then writen to a ByteArrayOutputStream.
File filePart = new File(fileReport);FilePart fileUpload = new FilePart("fileUpload", filePart);
StringPart applyAclPermissions = new StringPart("applyAclPermissions","true");
StringPart overwriteAclPermissions = new StringPart("overwriteAclPermissions","true");
StringPart overwriteFile = new StringPart("overwriteFile", "true");
StringPart logLevel = new StringPart("logLevel","TRACE");
StringPart retainOwnership = new StringPart("retainOwnership", "false");
StringPart fileNameOverride = new StringPart("fileNameOverride","blablub.prpt");
StringPart importDir = new StringPart("importDir", "/public");
Part[] parts = {
fileUpload,
overwriteFile,
logLevel,
retainOwnership,
fileNameOverride,
importDir
};
HttpMethodParams params = new HttpMethodParams();
MultipartRequestEntity requestEntity = new MultipartRequestEntity(
parts, params
);
ByteArrayOutputStream bOutput = new ByteArrayOutputStream();
requestEntity.writeRequest(bOutput);
String requestEntityValue = new String(bOutput.toByteArray());
String contentType = requestEntity.getContentType();
String contentLength = String.valueOf(requestEntity.getContentLength());
Object[] outputRow = createOutputRow(r, data.outputRowMeta.size());
get(Fields.Out, "requestEntityValue").setValue(outputRow, requestEntityValue);
get(Fields.Out, "contentType").setValue(outputRow, contentType);
get(Fields.Out, "contentLength").setValue(outputRow, contentLength);
putRow(data.outputRowMeta, outputRow);
return true;
In the next step the data is sent with the HTTP Post Step. However the server is not satisfied with this approach.
Do you guys have any idea what I am doing wrong?
Thanks for your help!

Since 5.4, there is a special plugin for interacting with BA server: https://github.com/pentaho/pdi-platform-utils-plugin . I highly recommend you to look at it.
As for implementing the upload by yourself, you can either look at the plugin sources or, for instance, this utility from Pentaho Report Designer: https://github.com/pentaho/pentaho-reporting/blob/master/libraries/libpensol/source/org/pentaho/reporting/libraries/pensol/PublishRestUtil.java
Hope this will help.

zip file size limitation on ckan java client upload

I have created a java program that can upload a file to a ckan installation, (currently my local testing installation, haven't tested it on a live one).
The text files that i tested my application are uploaded to ckan properly. I do have a problem with some zip files.
After some test and failure attempts, i realized that the problem is in the size of the file. For zip files less than 4Kb it works, but for larger files it fails with the error "server Error".
The files failing to upload via my java application, upload just fine when using the ckan front end, so i am guessing the problem is with my java application and not the ckan installation. This is the code I am using:
StringBuilder sb = new StringBuilder();
CloseableHttpClient httpclient = HttpClientBuilder.create().build();
File file = new File(uploadFileName);
SimpleDateFormat dateFormatGmt = new SimpleDateFormat("yyyyMMdd_HHmmss");
String date=dateFormatGmt.format(new Date());
HttpPost postRequest;
try {
ContentType zipType=ContentType.create("zip");
ContentBody cbFile = new FileBody(file,zipType);
HttpEntity reqEntity = MultipartEntityBuilder.create()
.addPart("file", cbFile)
.addPart("url",new StringBody(HOST+"/files/"+date+"/"+uploadFileName,ContentType.TEXT_PLAIN))
.addPart("upload",cbFile)
.addPart("title",new StringBody(TITLE,ContentType.TEXT_PLAIN))
.addPart("description",new StringBody(DESCRIPTION+date,ContentType.TEXT_PLAIN))
.addPart("comment",new StringBody(COMMENT,ContentType.TEXT_PLAIN))
.addPart("key", new StringBody(uploadFileName+date,ContentType.TEXT_PLAIN))
.addPart("package_id",new StringBody("test2",ContentType.TEXT_PLAIN))
.addPart("notes", new StringBody("notes",ContentType.TEXT_PLAIN))
.build();
postRequest = new HttpPost(HOST+"/api/action/resource_create");
postRequest.setEntity(reqEntity);
postRequest.setHeader("X-CKAN-API-Key", myApiKey);
HttpResponse response = httpclient.execute(postRequest);
int statusCode = response.getStatusLine().getStatusCode();
BufferedReader br = new BufferedReader(
new InputStreamReader((response.getEntity().getContent())));
}
catch (IOException ioe) {
System.out.println(ioe);
}
finally {
httpclient.getConnectionManager().shutdown();
}

Android - How to fill data in an HTML form on an external website and hit the upload button?

I have an Android device. I want to fill a form in my app, with edittexts etc (one of these fields would take the path of an image on the SDCard). I want these form contents to be the data for an HTML form in an external website where this file (from the SD Card) needs to be uploaded. The HTML form has an upload button. I do not want to show this HTML webpage to my android app users. Is there any way to do this? Please let me know! Thanks!
EDIT: I've looked through many websites and I understand that I should use a HttpPost. I have a few doubts though:
1. What is the url that you use in HttpPost- Is it the url which contains the form, or the url which the form redirects to.
2. In a multipartentity, what is the first parameter in addPart? Is it the ID given to the field or the name?
3. How does the HttpPost know which form it should go to?

Well, you need to make a MultiPart Http Post. You could use this sample:
HttpClient httpClient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost("target_link");
MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
reqEntity.addPart("data1", new StringBody("Data1"));
reqEntity.addPart("data2", new StringBody("Data2"));
reqEntity.addPart("data3",new StringBody("Data3"));
try{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bitmap.compress(CompressFormat.JPEG, 80, bos);
byte[] data = bos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(data, "forest.jpg");
reqEntity.addPart("picture", bab);
}
catch(Exception e){
//Log.v("Exception in Image", ""+e);
reqEntity.addPart("picture", new StringBody(""));
}
postRequest.setEntity(reqEntity);
HttpResponse response = httpClient.execute(postRequest);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
String sResponse;
StringBuilder s = new StringBuilder();
while ((sResponse = reader.readLine()) != null) {
s = s.append(sResponse);
}
Personally, I prefer to use Spring for Android as that is easier to configure. Here's a link with a multi-part Http Post.
Good luck!

Google Documents List API file upload 400 response

I am writing my own implementation of the Google Drive client for the Android and I am using the docs list api. Recently I've encountered a following problem:
At first I was using the HttpURLConnection to upload the file but it seems like it writes the data to the socket after a call to getResponseCose(), not when I am writing to the connection's OutputStream, which is a must for me.
Then I've switched to the Apache HttpClient but I'm still getting a 400 response, not sure why. Maybe you will be able to help me. Here is the code used to upload a file.
String putUrl = conn.getHeaderField("Location");//from the previous request
final HttpClient client = new DefaultHttpClient();
final HttpPut put = new HttpPut(putUrl);
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
put.addHeader("Content-Type", mime==null?"file":mime);
//put.addHeader("Content-Length", String.valueOf(length));
put.addHeader("Content-Range", "bytes 0-"+(length-1)+"/"+length);
put.addHeader("GData-Version", "3.0");
put.addHeader("Authorization", getAuthorizationProperty());
entity.addPart("content", new InputStreamBody(in, name));
put.setEntity(entity);
HttpResponse resp = client.execute(put);
int response = resp.getStatusLine().getStatusCode();
if(response == HttpStatus.SC_CREATED){
lastCreated = parseSingleXMLEntry(resp.getEntity().getContent());
}
Exactly the same headers worked for HttpURLConnection. Maybe the entity is wrong?

Ok, the solution is quite simple, hope it will be useful for someone.
I had to delete all lines which added headers to the request. After that I've added the mime type to the InputStreamBody constructor and overriden the getContentLength() method to provide stream length. Finally it looks like this:
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
entity.addPart("content", new InputStreamBody(in, ,mime, name){
#Override
public long getContentLength() {
return length;
}
});
put.setEntity(entity);
HttpResponse resp = client.execute(put);
And that's all.

HTTP Post Multipart on Android not posting image.

The whole thing works perfectly, except image won't show, no errors, Using RoR. What am I missing? All called by async class btw. Been trying several different methods with no avail, if someone could help me out that would be great. Willing to post more if needed.
Thanks!
public static void multiPart(Bitmap image, String topicid, String topost, Context c){
String responseString = "";
{
try {
String imageName = System.currentTimeMillis() + ".jpg";
HttpClient httpClient = new MyHttpClient(c);
HttpPost postRequest = new HttpPost("https://urlofmyapi");
if (image==null){
Log.d("TAG", "NULL IMAGE");
}
ByteArrayOutputStream bos = new ByteArrayOutputStream();
image.compress(CompressFormat.JPEG, 75, bos);
byte[] data = bos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(data, imageName);
MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
reqEntity.addPart("feed", new StringBody(topost));
reqEntity.addPart("post_to", new StringBody(topicid));
reqEntity.addPart("upload_file", bab);
postRequest.setEntity(reqEntity);
HttpResponse response = httpClient.execute(postRequest);
BufferedReader reader = new BufferedReader(
new InputStreamReader(
response.getEntity().getContent(), "UTF-8"));
String sResponse;
StringBuilder s = new StringBuilder();
while ((sResponse = reader.readLine()) != null) {
s = s.append(sResponse);
}
responseString = s.toString();
System.out.println("Response: " + responseString);
} catch (Exception e) {
Log.e(e.getClass().getName(), e.getMessage());
}
}
}

Perhaps you can do following step to import library into your Android.
requirement library:
apache-mime4j-0.6.jar
httpmime-4.0.1.jar
Right click your project and click properties
select java build path
select tab called "Order and Export"
Apply it
Fully uninstall you apk file with the adb uninstall due to existing apk not cater for new library
install again your apk
run it
Thanks,
Jenz

HTTP POSTing images from Java to RoR always seems to have undue issues for me. Have you tried attaching the binary as a org.apache.http.entity.mime.content.FileBody object, like this Android Multipart Upload question?