How to download an image from a server and then write it as a response in my servlet.
What is the best way to do it keeping good performance?
Here's my code:
JSONObject imageJson;
... //getting my JSON
String imgUrl = imageJson.get("img");
if you don't need to hide your image source and if server is accessible from the client as well, I'd just point your response to remote server (as you already have the url) => you don't need to do a download to your server first, but possibly client could access it directly => you don't waste your resources.
However if you still need to download it to your server first, following post might help: Writing image to servlet response with best performance
It's important to avoid intermediate buffering of image in servlet. Instead just stream whatever was received to the servlet response:
InputStream is = new URL(imgUrl).openStream();
OutputStream os = servletResponse.getOutputStream();
IOUtils.copy(is, os);
is.close();
I'm using IOUtils from Apache Commons (not necessary, but useful).
The complete solution : download a map and save to file.
String imgUrl = "http://maps.googleapis.com/maps/api/staticmap?center=-15.800513,-47.91378&zoom=11&size=200x200&sensor=false";
InputStream is = new URL(imgUrl).openStream();
File archivo = new File("c://temp//mapa.png");
archivo.setWritable(true);
OutputStream output = new FileOutputStream(archivo);
IOUtils.copy(is, output);
IOUtils.closeQuietly(output);
is.close();
Related
There is an API that downloads Excel files. Whenever we click on the 'Download excel report' button, it calls this API which gets data from the database, transfers it to an Excel file, and downloads it.
excelName = key + ".xlsx";
response.setHeader("Content-Disposition", "attachment; filename=" + excelName);
ByteArrayInputStream in = getDownloadBusinessAnalysisKey(key, customerScopeId, response, getBusinessKeyResult, sheetName);
response.setHeader("Content-Length", String.valueOf(in.available()));
try {
FileCopyUtils.copy(in, response.getOutputStream());
} catch (IOException e) {
logger.error("Error in report " + e.getLocalizedMessage());
}
It downloads fine when there are about 10,000 records (or close to it).
However, when there are about 100,000 records it just doesn't download. I get no response from the endpoint. It works when trying to download the same file via Postman in local database.
Is there some limit on download size? Would this be a browser, Tomcat server, or HTTP header issue?
When you're using a ByteArrayInputSteam you're loading the whole data into memory. What needs to be done is to make getDownloadBusinessAnalysisKey store to a place you can stream, usually to a file. Then use FileInputStream to read the file and write it to the response output stream.
Based on your statement that you're using an API to download the Excel file, I am assuming it is not done inside your current JVM (otherwise I'd recommend you look up SXSSF which would require a bit of code change and rethinking to get it to work.
From your code sample it looks like you're using Servlet API as well based on the setHeader. So here's a bit of code that would make a get connection and more or less proxy it up. There's no need for a temporary file if it is just straight through, there's also no need to buffer unless you can confirm that the servlet engine does not provide you with buffered data.
protect void doGet(
HttpServletRequest req,
HttpServletResponse response)
throws ServletException, IOException {
var url = new URL("http://myapi");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
int responseCode = con.getResponseCode();
int len = con.getHeaderFieldInt("Content-Length",-1);
int contentType = con.getHeaderField("Content-Type", "application/octet-stream");
assert responseCode == 200
response.setIntHeader("Content-Length", len);
response.setHeader("Content-Type", contentType);
try (
InputStream is = con.getInputStream();
OutputStream os = response.getOutputStreeam();
) {
int c = is.read();
while (c != -1) {
os.write(c);
c = is.read();
}
}
}
This can be optimized by using Async Servlet APIs to reduce blocking I/O which would be needed if you are dealing with a lot of connections.
I think that the process to get data from db and fetch into excel taking too long time and it is longer than max response time of server. You should tuning the process getting data from db and fetching into excel, it will solve your issue.
So, I have a server application that returns ZIP files and I'm working with huge files (>=5GB). I am then using the jersey client to do a GET request from this application after which I want to basically extract the ZIP and save it as a folder. This is the client configuration:
Client client = ClientBuilder.newClient();
client.register(JacksonJaxbJsonProvider.class);
client.register(MultiPartFeature.class);
return client;
And here's the code fetching the response from the server:
client.target(subMediumResponseLocation).path("download?delete=true").request()
.get().readEntity(InputStream.class)
My code then goes through a bunch of (unimportant for this question) steps and finally gets to the writing of data.
try (ZipInputStream zis = new ZipInputStream(inputStream)) {
ZipEntry ze = zis.getNextEntry();
while(ze != null){
String fileName = ze.getName();
if(fileName.contains(".")) {
size += saveDataInDirectory(folder,zis,fileName);
}
is.closeEntry();
ze = zis.getNextEntry();
}
zis.closeEntry();
} finally {
inputStream.close();
}
Now the issue I'm getting is that the ZipInputStream refuses to work. I can debug the application and see that there are bytes in the InputStream but when it get to the while(ze != null) check, it returns null on the first entry, resulting in an empty directory.
I have also tried writing the InputStream from the client to a ByteArrayOutputStream using
the transferTo method, but I get a java heap space error saying the array length is too big (even though my heap space settings are Xmx=16gb and Xms=12gb).
My thoughts were that maybe since the InputStream is lazy loaded by Jersey using the UrlConnector directly, this doesn't react well with the ZipInputStream. Another possible issue is that I'm not using a ByteArrayInputStream for the ZipInputStream.
What would a proper solution for this be (keeping in mind the heap issues)?
Ok so I solved it, apparently my request was getting a 404 for adding the query param in the path... .path("download?delete=true")
I'm trying to download zip files from internet using following code:
public void getFile(String updateURL) throws Exception {
URL url = new URL(updateURL);
HttpURLConnection httpsConn = (HttpURLConnection) url.openConnection();
httpsConn.setRequestMethod("GET");
TrustModifier.relaxHostChecking(httpsConn);
int responseCode = httpsConn.getResponseCode();
if (responseCode == HttpsURLConnection.HTTP_OK) {
String fileName = "fileFromNet";
try (FileOutputStream outputStream = new FileOutputStream(fileName)) {
ReadableByteChannel rbc = Channels.newChannel(httpsConn.getInputStream());
outputStream.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
}
}
httpsConn.disconnect();
}
TrustModifier is a class used to solve the "trust issue": http://www.obsidianscheduler.com/blog/ignoring-self-signed-certificates-in-java/
The code above works well for zip files available via plain http or for non compressed files exposed via https but but if I try to download a zip file exposed via https endpoint only a small fragment of original file will be downloaded. I have tested with different download links from internet and always got the same result.
Does anybody has an idea what I've been doing wrong here?
Thank you.
transferFrom() must be called in a loop until the transfer is complete, and in this case the only way you can know that is by adding up the return values of transferFrom() until they equal the Content-length of the HTTP response.
Actually the problem was in the TrustModifier Class I was using to switch off the servier certificate check. Once I removed it because I didn't need it any longer (I took the certificate from server and put it in a local trust store), my problem was solved.
I am trying to download a PDF file from a response of Java REST call after custom authentication check.
I can see downloaded file but it is empty file.
Below is my code snippet.
//Custom HTTPClient
HTTPAuthClient client = new HTTPAuthClient(url,username,password)
Request request = new Request(downloadURL); //I'm downloading file content of an URL.
Response response = client.executeGet(request);
String response1 = response.getResponseBody();
InputStream is = new ByteArrayInputStream(response.getBytes());
response.setContentType("Content-type",application/pdf); //here response is //javax.servlet.HttpServletResponse
response.setHeader("Content-Disposition","attachment;filename="myfile.pdf");
IOUtils.copy(is,response.getOutPutStream());
response.flushBuffer();
With this code I could download the file but when I open the file and verified there is no data.
As part of response body also I can see some data.
Could you please help me out where I'm doing mistake I tried many options but did not find solution.
How can you use setContentType like this
response.setContentType("Content-type",application/pdf);
If only one avalible param in this method is String void setContentType(String type) so your method should be:
response.setContentType("application/pdf");
Java Doc to be sure.
I have a requirement in which I have to write to files kept in FTP server using java ,I cant write to a file in the local server and then transfer it to ftp due to sensitivity of the data,can anyone share some thoughts/links on this.
Any Help will be greatly appreciated.
Apology for not posting my code snippet earlier below is the code I wrote
Student stu=new Student();
stu.setName("xyz");
stu.setRoll("12");
ftpClient.changeWorkingDirectory("/mydirectory/release/");
//abc.txt is the file on the server
FileOutputStream fos=(FileOutputStream)ftpClient.appendFileStream("abc.txt");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(stu);
Iam not getting any exception ,but also not able to write into the file..
yes I want to upload bytes via ftp from memory..
Thanks
I got the solution,I had to use url instead of ftpClient
URL url = new URL("ftp://user:pass#myftp.abc.com/myFile.txt;type=i");
URLConnection urlc = url.openConnection();
OutputStream os = urlc.getOutputStream(); // To upload
OutputStream buffer = new BufferedOutputStream(os);
ObjectOutput output = new ObjectOutputStream(buffer);
output.writeObject(myObject);
buffer.close();
os.close();
output.close();
It looks like Apache's FTPClient class, documented here, will do what you want.