Develop Reference

Using google-http-client to upload files via API

I'm trying to upload a file ("new.html", in the root of my project directory) via an API using Google's http-client.
The docs for the API (and the specific function) can be found at https://piathome.com/apidocs/#/asset/UploadFile.
My code is as follows
FileContent fileContent = new FileContent("text/html", new File("new.html"));
MultipartContent mpc = new MultipartContent().setMediaType(new HttpMediaType("multipart/form-data")
.setParameter("boundary", "__END_OF_PART__"));
MultipartContent.Part part = new MultipartContent.Part(fileContent);
part.setHeaders(new HttpHeaders().set("Upload ", "file=#new.html;type=text/html"));
mpc.addPart(part);
HttpRequest request;
try {
request = requestFactory.buildPostRequest(new GenericUrl(baseURL + "/files"), mpc)
.setParser(JsonFactory.createJsonObjectParser());
request.getHeaders().set("x-access-token", token);
request.getHeaders().set("accept", "application/json");
HttpResponse response = request.execute();
System.out.println(response.parseAsString());
Printing out response.parseAsString() yields {"stat_message":" Successfully uploaded files","data":[],"success":true}. Obviously something is wrong here, as the data field of the response is empty, but I'm not sure what I'm doing wrong, or why the API is returning success when something isn't working.
Any help is appreciated, thanks.

Different behavior of OkHttp3 on Linux and Windows

I have some code which works ok on windows platform, however, the code gives a different behavior on Linux.
I have used the following code to submit a request to an HTTP server to get some messages. what I have done as follows
deploy the code on my local windows machine, then trigger a request and get the server response.
parameters:
{"articleid":"","endtime":"2019-10-29T18:00:00","starttime":"2019-10-29T16:00:00","areaid":"","title":"","pageIndex":"1"}
server response:
{"result":1,"errorcode":"","message":"","pageindex":1,"nextpage":2,"pagesize":100,"data":[...
some data here ...]}
deploy the code on a Linux server, trigger the request with the same parameters in step 1, however, the server response is different.
parameters:
{"articleid":"","endtime":"2019-10-29T18:00:00","starttime":"2019-10-29T16:00:00","areaid":"","title":"","pageIndex":"1"}
server response:
{"result":1,"errorcode":"","message":"","pageindex":1,"nextpage":null,"pagesize":0,"data":[]}
We have looked through the code but can not find what causes the different behaviors.
I suppose there may exist one/some java class files with the same name in different jars, and windows/Linux load different class files then cause the problem, but after looking through the jar file, I also have no ideas. the okhttp related jar files are as following:
okhttp-3.10.0.jar
okio-1.14.0.jar
netty-codec-http-4.1.31.Final.jar
httpcore-nio-4.4.10.jar
httpcore-4.4.10.jar
httpclient-4.5.6.jar
httpasyncclient-4.1.4.jar
public static String okHttpPost(String requestUrl,Map<String,String> map,String orgId,String taskID) throws IOException {
String exceptionMessage="";
String responseResult="";
try {
FormBody.Builder newFormBody = new FormBody.Builder();
Set<String> keys = map.keySet();
for(String key:keys){
newFormBody.add(key,map.get(key));
}
RequestBody body = newFormBody.build();
log.info("server url : "+requestUrl+";paramters："+new ObjectMapper().writeValueAsString(map));
Request request = new Request.Builder()
.url(requestUrl)
.post(body)
.build();
Call call = okHttpClient.newCall(request);
Response response = call.execute();
if (response.code() != 200) {
exceptionMessage = "request failed, taskID：" + taskID + "orgid：" + orgId + "response mesage："+response.toString();
log.info(exceptionMessage);
}
responseResult = response.body().string();
log.info("server url : " + requestUrl + ", reponse messages："+responseResult);
return responseResult;
} catch (IOException e) {
e.printStackTrace();
}finally {
if (!responseResult.contains("token")) {
do some thing;
}
}
return null;
}
Can give any ideas on why the same code behaviors different on windows and Linux platform?
How to change the code to let it works well on Linux?

Problem with RestTemplate when including URL addresses in REST API request

I'm trying to obtain data from Botify's REST API to use it inside a project, which is also a REST API. I'm using an instance of Spring's RestTemplate class to make the actual requests to Botify, specifically the .exchange method as I need to pass Botify's key as a header parameter.
My problem comes when I need to call to a method of the endpoint which takes a URL as a part of the request's URI (not a parameter). Documentation of this endpoint is in https://developers.botify.com/api/reference/#!/Analysis/getUrlDetail
Basically the structure of the requests is like this:
/analyses/{username}/{project_slug}/{analysis_slug}/urls/{url}
The last part of that URI is a URL address, which needs to be encoded in UTF-8 to make it possible to separate it from the actual request.
The problem is (I believe) that the .exchange method always encodes the request, so what I try to send like this:
/analyses/myusername/myprojectname/myprojectslug/urls/https%3A%2F%2Fwww.example.com
...ends up like this:
/analyses/myusername/myprojectname/myprojectslug/urls/https%253A%252F%252Fwww.example.com'
Which obviously doesn't work. This is an excerpt from the method that makes the call to Botify:
public String callBotifyEndpoint(String reportType, String parameters) throws UnsupportedEncodingException {
String request = this.baseUri + "/analyses/myusername/myprojectname/myprojectslug/urls/https%3A%2F%2Fwww.example.com"
HttpHeaders headers = new HttpHeaders();
headers.set("Authorization", "Token " + this.apiKey);
HttpEntity<String> entity = new HttpEntity<>(headers);
UriComponentsBuilder botifyQueryBuilder = UriComponentsBuilder.fromUriString(request);
String queryStringBuild = botifyQueryBuilder.build(true).toUriString();
String botifyResult = null;
try {
System.out.println("Calling Botify API: " + queryStringBuild);
ResponseEntity<String> response = botifyTemplate.exchange(queryStringBuild, HttpMethod.GET, entity, String.class);
if(response.hasBody()) {
botifyResult = response.getBody();
}
} catch(RestClientException ex) {
ex.printStackTrace();
}
try {
} catch (Exception e) {
// TODO: handle exception
}
return botifyResult;
}
In this line:
botifyQueryBuilder.build(true).toUriString();
The "true" parameter indicates whether the data is already encoded or not. I've tried to disable it but the result is the same.
I've removed actual request generation process (along with my user and project's name) to simplify things, but this should return a response from Botify with the existing data for that URL.
Instead, it returns a 400 bad request error (which makes sense, because the URL is not correct).
I'm feeling like this may be a bug in RestTemplate's .exchange method, but maybe I'm not using it properly. Any suggestions?

Don't encode prematurly as you do here:
String request = this.baseUri + "/analyses/myusername/myprojectname/myprojectslug/urls/https%3A%2F%2Fwww.example.com";
Use parameter placeholders feature in RestTemplate instead of text concatenation.
Refer to:
Spring RestTemplate GET with parameters

How to send selected file list when submit button click in AngularJs

I am using ng-file-upload to send files to server.ng-file-upload worked perfectly with upload file one by one.I am using latest chrome browser.
Sample code snippet
jsfiddle
in there files are uploaded to the server when files are selected in the file input.But what i want is that files (selected file array) should be uploaded only to the server when submit button is clicked along with other form data.
Jersey REST service
#POST
#Path("/manual")
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Produces(MediaType.APPLICATION_JSON)
public boolean insertResults(#FormDataParam("file") InputStream uploadedInputStream,#FormDataParam("file") FormDataContentDisposition fileDetail,#FormDataParam("username") String username) throws IOException {
System.out.println(username);
StringWriter writer = new StringWriter();
IOUtils.copy(uploadedInputStream, writer,"UTF-8");
String theString = writer.toString();
System.out.println(theString);
return Boolean.TRUE;
}
I am new to AngularJs stuff please help me to overcome this problem.

According to the documentation on the site and from what I'd expect - you can attach the upload function to the button and trigger it as you'd do any other function. This is an example from the GitHub documentation on ng-file-upload:
app.controller('MyCtrl', ['$scope', 'Upload', function ($scope, Upload) {
// upload later on form submit or something similar
$scope.submit = function() {
if (form.file.$valid && $scope.file) {
$scope.upload($scope.file);
}
};
// upload on file select or drop
$scope.upload = function (file) {
Upload.upload({
url: 'upload/url',
data: {file: file, 'username': $scope.username}
}).then(function (resp) {
console.log('Success ' + resp.config.data.file.name + 'uploaded. Response: ' + resp.data);
}, function (resp) {
console.log('Error status: ' + resp.status);
}, function (evt) {
var progressPercentage = parseInt(100.0 * evt.loaded / evt.total);
console.log('progress: ' + progressPercentage + '% ' + evt.config.data.file.name);
});
};
// for multiple files:
$scope.uploadFiles = function (files) {
if (files && files.length) {
for (var i = 0; i < files.length; i++) {
Upload.upload({..., data: {file: files[i]}, ...})...;
}
// or send them all together for HTML5 browsers:
Upload.upload({..., data: {file: files}, ...})...;
}
}
}]);
Here you can see that he's calling the $scope.upload function when the forms $scope.submit is called plus he's even checking to see if the form is valid before calling upload.

I have decided to write a descriptive answer about how to send multiples files to back end and access file details one by one.There are lack of informative answers are on the internet regarding this so this answer maybe helpful for someone.To send multiple files in AngularJs to back end you can use ng-file-upload API.You can send files when submit button click like above mentioned answer.Let assume your front end is working perfectly and can send files to server.So most of you have doubt about how to manipulate files and other details if it is a multipart form data.Here is the way how to manipulate form data.
Server end received data something like below.
Sample files data along with other form attributes
{files[0]=[org.glassfish.jersey.media.multipart.FormDataBodyPart#40ce14c9],files[1]=[org.glassfish.jersey.media.multipart.FormDataBodyPart#40ce14c9],userName=sam}
REST endpoint(using Jersey) to manupulate multipart form data
#POST
#Consumes(MediaType.MULTIPART_FORM_DATA)
public void upload(FormDataMultiPart formParams){
Map<String, List<FormDataBodyPart>> fieldsByName = formParams.getFields();
for (List<FormDataBodyPart> fields : fieldsByName.values()) {
for (FormDataBodyPart field : fields){
// Check if fields are files(If any one knows better solution to check files[] please share your answers)
if (StringUtils.equals("files",
StringUtils.substringBefore(field.getName(), "["))) {
//To read file content
InputStream is = field.getEntityAs(InputStream.class);
String fileName = field.getName();
field.getMediaType();//File mimeType
//To get file details like size,file name,etc
FormDataContentDisposition f=field.getFormDataContentDisposition();
System.out.println(f.getFileName());
Note: f.getType() not return the actual file type it returns mime type as form-data to get actual mime type use FormDataBodyPart media type like above.
}
get other form data like user name
else if(StringUtils.equals(field.getName(),"userName")){
System.out.println(field.getValue());
}
}
}
}

Heap exception when write file on my server

I have some problem with file write on server. I have two approach to file upload, one with Spring Multipart file, and it doesn't have problem, and one with byte array and when the file is big it gives java.lang.OutOfMemoryError: Java heap space(I can see that the occupied ram increase with this method). I use byte array because I pass this file between web service REST and I put the stream into an object with also the field name.
This is my code for file write:
#Override
public void storeAcquisition(Response response, String toStorePath) throws Exception{
#SuppressWarnings("unchecked")
LinkedHashMap<String,String> result= (LinkedHashMap<String,String>)response.getResult();
Files.write(Paths.get(toStorePath + "/" + result.get("name")), DatatypeConverter.parseBase64Binary(result.get("content")));
}
This is the sender method
byte[] file = getFile(filePath);
if (file!=null){
FileTransfer fileTransfer= new FileTransfer(file, Paths.get(filePath).getFileName().toString());
//TODO POST TO SERVER
Response responseSend = new Response(true, true, fileTransfer, null);
RestTemplate restTemplate = new RestTemplate();
Response responseStatus = restTemplate.postForObject(serverIp + "ATS/client/file/?toStorePath={toStorePath}", responseSend, Response.class, toStorePath);
return responseStatus;
}
and
public byte[] getFile(String path) throws Exception{
if (Files.exists(Paths.get(path)))
return Files.readAllBytes(Paths.get(path));
else
return null;
}
Is there a problem in my code or the only solution is to use a better server?
The problem seems to be on restTemplate.postForObject and if I put a breakpoint on LinkedHashMap<String,String> result it doesn't come called.