I am currently working on a Pentaho PDI project where I need to upload a report to the BA/BI server repository (URL: ***/api/repo/publish/file). I would like to achieve this by using the HTTP Post step and an User Defined Java Class step that produces the request entity field.
However I didn't manage to come up with a working code. Since my boss does not want me to use external libraries, I am sticking to the org.apache.commons.httpclient classes which are deployed with kettle.
My approach is to create a Part[] array containing the FilePart and StringParts. The next step is to create a MultipartRequestEntity which is then writen to a ByteArrayOutputStream.
File filePart = new File(fileReport);FilePart fileUpload = new FilePart("fileUpload", filePart);
StringPart applyAclPermissions = new StringPart("applyAclPermissions","true");
StringPart overwriteAclPermissions = new StringPart("overwriteAclPermissions","true");
StringPart overwriteFile = new StringPart("overwriteFile", "true");
StringPart logLevel = new StringPart("logLevel","TRACE");
StringPart retainOwnership = new StringPart("retainOwnership", "false");
StringPart fileNameOverride = new StringPart("fileNameOverride","blablub.prpt");
StringPart importDir = new StringPart("importDir", "/public");
Part[] parts = {
fileUpload,
overwriteFile,
logLevel,
retainOwnership,
fileNameOverride,
importDir
};
HttpMethodParams params = new HttpMethodParams();
MultipartRequestEntity requestEntity = new MultipartRequestEntity(
parts, params
);
ByteArrayOutputStream bOutput = new ByteArrayOutputStream();
requestEntity.writeRequest(bOutput);
String requestEntityValue = new String(bOutput.toByteArray());
String contentType = requestEntity.getContentType();
String contentLength = String.valueOf(requestEntity.getContentLength());
Object[] outputRow = createOutputRow(r, data.outputRowMeta.size());
get(Fields.Out, "requestEntityValue").setValue(outputRow, requestEntityValue);
get(Fields.Out, "contentType").setValue(outputRow, contentType);
get(Fields.Out, "contentLength").setValue(outputRow, contentLength);
putRow(data.outputRowMeta, outputRow);
return true;
In the next step the data is sent with the HTTP Post Step. However the server is not satisfied with this approach.
Do you guys have any idea what I am doing wrong?
Thanks for your help!
Since 5.4, there is a special plugin for interacting with BA server: https://github.com/pentaho/pdi-platform-utils-plugin . I highly recommend you to look at it.
As for implementing the upload by yourself, you can either look at the plugin sources or, for instance, this utility from Pentaho Report Designer: https://github.com/pentaho/pentaho-reporting/blob/master/libraries/libpensol/source/org/pentaho/reporting/libraries/pensol/PublishRestUtil.java
Hope this will help.
Related
I need to convert the following code from Java to C# when I'm using restAPI in C#.
In java :
HttpGet statusGet = new HttpGet(fileUrl);
statusGet.setHeader("X-API-TOKEN", API_TOKEN);
HttpResponse response = httpClient.execute(statusGet);
// Extract exported file
ZipInputStream zs = new ZipInputStream(response.getEntity().getContent());
In C# this is what I have:
var client1 = new RestClient(fileUrl);
var request1 = new RestRequest(Method.GET);
request1.AddHeader("X-API-TOKEN", "API_TOKEN");
request1.AddHeader("content-type", "application/json");
request1.AddParameter("application/json", "{\n\t\"format\" : \"csv\",\n\t\"surveyId\" : \"_surveyId\"\n}", ParameterType.RequestBody);
IRestResponse responsedata = client1.Execute(request1);
var download=client1.DownloadData(request1);
MemoryStream stream = new MemoryStream(download);
ZipInputStream zs = new ZipInputStream(stream);
using (ZipFile zip1 = ZipFile.Read(zs))
I have no clue how to implement response.getEntity().getContent(). I believe it is getting the Stream(Containing a zip file?)
Updated: So I get the byte array from client1.DownloadData(request1), looks like it is not right to convert it to stream (has readtimeout exception). and it will not be able to read from zipfile.read
Thank you so much for your help
Are you getting any specific errors? It looks like you are implementing this using RestSharp. Have you followed their examples and read through their documentation?
I have not personally used this third-party solution, but immediately on their front page they have the following example that does exactly what you are trying to do:
var client = new RestClient("http://example.com");
// client.Authenticator = new HttpBasicAuthenticator(username, password);
var request = new RestRequest("resource/{id}", Method.POST);
request.AddParameter("name", "value"); // adds to POST or URL querystring based on Method
request.AddUrlSegment("id", "123"); // replaces matching token in request.Resource
// easily add HTTP Headers
request.AddHeader("header", "value");
// add files to upload (works with compatible verbs)
request.AddFile(path);
// execute the request
IRestResponse response = client.Execute(request);
var content = response.Content; // raw content as string
// or automatically deserialize result
// return content type is sniffed but can be explicitly set via RestClient.AddHandler();
RestResponse<Person> response2 = client.Execute<Person>(request);
var name = response2.Data.Name;
// easy async support
client.ExecuteAsync(request, response => {
Console.WriteLine(response.Content);
});
// async with deserialization
var asyncHandle = client.ExecuteAsync<Person>(request, response => {
Console.WriteLine(response.Data.Name);
});
// abort the request on demand
asyncHandle.Abort();
It looks like you would want to access the IRestResponse.Content property, or to deserialize using the RestClient.Execute<T>(RestRequest request) function.
Dropwizard (Version 0.8.2) uses Jersey internally to provide HTTP client. I am using this client to send a Multipart POST request to an external Rest Endpoint to a SMS Service. Code is given below but it doesn't seems to be working because i am not receiving any message through this method also it does not throw any error.
URI for the first sample is http://enterprise.com/GatewayAPI/rest?userid=%s&password=%s&method=xlsUpload&filetype=zip&msg_type=TEXT&auth_scheme=PLAIN&v=1.1
FileDataBodyPart fileDataBodyPart = new FileDataBodyPart(fileName, file,
MediaType.APPLICATION_OCTET_STREAM_TYPE);
FormDataMultiPart multiPart = new FormDataMultiPart();
multiPart.field("fileName", fileName).bodyPart(fileDataBodyPart);
Entity<FormDataMultiPart> entity =
Entity.entity(multiPart, multiPart.getMediaType());// MediaType.MULTIPART_FORM_DATA_TYPE)
Client tenacityClient = TenacityJerseyClientBuilder
.builder(AppDependencyKeys.BULK_SMS)
.usingTimeoutPadding(Duration.milliseconds(500)).build(client)
.register(MultiPartFeature.class);
Invocation invocation = getResourceBuilder(tenacityClient, uri).buildPost(entity);
Future<Response> futureResponse = invocation.submit();
long start = System.currentTimeMillis();
futureResponse.get();
But the same works with below method when i use Apache Commons Httpclient. working code for the same is given below.
HttpClient client = new HttpClient();
PostMethod method = new
PostMethod("http://enterprise.com/GatewayAPI/rest");
Part[] parts = {
new StringPart("method", "xlsUpload"),
new StringPart("userid", "*******"),
new StringPart("password", "*******"),
new StringPart("filetype", "zip"),
new StringPart("v", "1.1"),
new StringPart("auth_scheme", "PLAIN"),
new FilePart(file.getName(), file)
};
method.setRequestEntity(new MultipartRequestEntity(parts, method.getParams()));
int statusCode = client.executeMethod(method);
log.info("Status code: {}", statusCode);
But i want to use the first way as that suits my infrastructure better.
I think you should set up properly media type for entity. Currently, you created new FormDataMultiPart but, you did not set and media type and it uses "text/plain" y default.
So, you should set up MediaType.APPLICATION_OCTET_STREAM_TYPE to your FormDataMultiPart as media type.
Just what the title says, if it helps in any way I have this java code (multipart consists of json object and file):
// Construct a MultiPart
MultiPart multiPart = new MultiPart();
multiPart.bodyPart(new BodyPart(inParams, MediaType.APPLICATION_JSON_TYPE));
multiPart.bodyPart(new BodyPart(fileToUpload, MediaType.APPLICATION_OCTET_STREAM_TYPE));
// POST the request
final ClientResponse clientResp = resource.type("multipart/mixed").post(ClientResponse.class, multiPart);
(using com.sun.jersey.multipart ) and I want to create the same in .NET (C#)
So far I managed to POST the json object like this:
Uri myUri = new Uri("http://srezWebServices/rest/ws0/test");
var httpWebRequest = (HttpWebRequest)WebRequest.Create(myUri);
httpWebRequest.Proxy = null;
httpWebRequest.Accept = "application/json";
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
Console.Write("START!");
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream())){
string json = new JavaScriptSerializer().Serialize(new
{
wsId = "0",
accessId = "101",
accessCode = "x#ds!2"
});
streamWriter.Write(json);
streamWriter.Flush();
streamWriter.Close();
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
}
But I want to send the file together. The content type has to be "multipart/mixed" because that's what the web service gets. I tried to find some package that supports multiparts but I found nothing except maybe this http://www.example-code.com/csharp/mime_multipartMixed.asp (which is not free so I can't use it).
I finally managed to do it like this:
HttpContent stringStreamContent = new StringContent(json);
stringStreamContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
HttpContent fileStreamContent = new StreamContent(fileStream);
fileStreamContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
// Construct a MultiPart
// 1st : JSON Object with IN parameters
// 2nd : Octet Stream with file to upload
var content = new MultipartContent("mixed");
content.Add(stringStreamContent);
content.Add(fileStreamContent);
// POST the request as "multipart/mixed"
var result = client.PostAsync(myUrl, content).Result;
I am trying to upload an image file using http-client from my Google Glass to my server but it always gets stuck at the httpclient.execute() method. I am not sure how should I approach uploading files from my Glass. This is what I have so far:
httpClient = HttpUtils.getNewHttpsClient();
postRequest = new HttpPost(strURL);
final File file= new File("mnt/sdcard/DCIM/Camera/12232.jpg");
s = new StringBuilder();
try
{
if(file.exists())
{
final FileBody bin = new FileBody(file);
final MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
reqEntity.addPart("uid", new StringBody(username,Charset.forName("UTF-8")));
reqEntity.addPart("pwd", new StringBody(password,Charset.forName("UTF-8")));
if(encKey!=null && !encKey.equals(""))
reqEntity.addPart("pvtkey", new StringBody(encKey,Charset.forName("UTF-8")));
reqEntity.addPart("p", new StringBody(selectedDrivePath,Charset.forName("UTF-8")));
reqEntity.addPart("ornt", new StringBody(fornt,Charset.forName("UTF-8")));
reqEntity.addPart("file_size", new StringBody(strfilesize,Charset.forName("UTF-8")));
reqEntity.addPart("data", bin);
contentLength=reqEntity.getContentLength();
postRequest.setEntity(reqEntity);
final HttpResponse response = httpClient.execute(postRequest);
...
}
...
}
...
Where am I going wrong?
You may want to ensure you have reviewed the following description for media uploading. from the developer site for Google Glass.
I know this is basic (many stack*overflow* community members prefer that you already have researched a problem before posting here), so you really should visit https://developers.google.com/glass.
I've got XML data in AS3 that needs to be compressed, validated on my Java Google App Engine servlet then saved to a file in Google Cloud Storage. Later that file will be opened and decompressed by the AS3 client. The process works if I do it with plain XML or text, but if I ByteArray#compress the data, it dies during ByteArray#uncompress with "There was an error decompressing the data".
I've tried setting the content type and mime type at various points, as well as encoding with Base64, but every attempt seems to break in a different way and I never get the same XML back that I sent in. Do I need to use multipart? Should I compress on the server? What's the best practice for doing this?
Sending the data from AS3:
// compress xml using zlib
var xml:XML = <contents><thing>value</thing></contents>;
var bytes:ByteArray = new ByteArray();
bytes.writeObject(xml);
bytes.position = 0;
bytes.compress();
var request:URLRequest = new URLRequest(url);
var urlVariables :URLVariables = new URLVariables();
urlVariables.filename = "somefile.bin";
urlVariables.contents = bytes;
request.data = urlVariables;
request.method = URLRequestMethod.POST;
loader = new URLLoader();
loader.load(request);
Receiving it in the Java servlet and creating the file:
public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
String filename = req.getParameter("filename");
byte[] contents = req.getParameter("contents").getBytes();
GSFileOptionsBuilder optionsBuilder = new GSFileOptionsBuilder()
.setBucket("bucketname")
.setKey(filename)
.setAcl("public-read")
.setMimeType("binary/octet-stream");
AppEngineFile writableFile = fileService.createNewGSFile(optionsBuilder.build());
boolean lockForWrite = true;
FileWriteChannel writeChannel = fileService.openWriteChannel(writableFile, lockForWrite);
writeChannel.write(ByteBuffer.wrap(contents));
writeChannel.closeFinally();
}
Opening the new file in AS3:
var url :String = "http://commondatastorage.googleapis.com/bucketname/somefile.bin";
var request:URLRequest = new URLRequest(url);
request.method = URLRequestMethod.GET;
loader = new URLLoader();
loader.addEventListener(Event.COMPLETE, handleComplete);
loader.load(request);
protected function handleComplete (event:Event):void {
var bytes:ByteArray = new ByteArray();
bytes.writeObject(event.target.data);
// dies on this line with "There was an error decompressing the data."
bytes.uncompress();
var xml:XML = new XML(new String(bytes));
trace(xml);
}
Here is the code that I use to save an xml. I send the data to PHP but I would think it would work the same way for you... I haven't had any trouble with it.
var createXMLURL:URLRequest = new URLRequest("createXML.php");
createXMLURL.method = URLRequestMethod.POST;
var Variables:URLVariables = new URLVariables();
Variables.xmlString = xml.toXMLString();
Variables.filename = filename;
createXMLURL.data = Variables;
var loader:URLLoader = new URLLoader();
loader.dataFormat = "VARIABLES";
loader.addEventListener(Event.COMPLETE, xmlCreated);
loader.load(createXMLURL);
Let me know if you have any questions about what some of the variables are since I did not include their declarations (I think they are pretty easy to figure out).
Now this doesn't send that data in binary format like you were asking for, but I don't know why you wouldn't be able to convert the string to binary once you receive it in java if you really need the raw bytes.
I would base64 encode before you POST if from the client, store it that way in a TextProerty, then base64 decode / decompress when received back at the client. If you want to store it as binary on GAE, then base64 decode it into a Blob. Here are some code snippets I pieced together using your code, and something similar I do using HttpService -- apologies in advance for not extensively proofing it. HTH.
private var _serviceHttp:HTTPService = new HTTPService;
private function postBytes():void {
var xml:XML = <contents><thing>value</thing></contents>;
var bytes:ByteArray = new ByteArray();
bytes.writeObject(xml);
bytes.position = 0;
bytes.compress();
var enc:Base64Encoder = new Base64Encoder();
enc.encodeBytes(bytes, 0, bytes.length);
var myObj:Object = new Object();
myObj["bytes"] = enc.toString();
// myObj["other_variables"] = your_other_varaibles;
_serviceHttp.method = "POST";
_serviceHttp.resultFormat = "flashvars";
_serviceHttp.url = your_url_here;
_serviceHttp.addEventListener(ResultEvent.RESULT, urlHandler);
_serviceHttp.addEventListener(FaultEvent.FAULT, urlErrorHandler);
_serviceHttp.send(myObj);
}