Develop Reference

Read a file using Java from an S3 bucket and HTTP PUT file to presigned AWS S3 URL of another bucket in a way that simulates an actual file upload

New to Java and HTTP requests.
Why this question is not a duplicate: I'm not using AWS SDK to generate any presigned URL. I get it from an external API.
Here is what I'm trying to accomplish:
Step 1: Read the source S3 bucket for a file (for now .xlsx)
Step 2: Parse this file by converting it to an InputStreamReader (I need help here)
Step 3: Do a HTTP PUT of this file by transferring the contents of the InputStreamReader to an OutputStreamWriter, on a pre-signed S3 URL that I already have obtained from an external team. The file must sit in the destination S3 bucket, in the exact way a file is uploaded manually by dragging and dropping. (Also need help here)
Here is what I've tried:
Step 1: Read the S3 bucket for the file
public class LambdaMain implements RequestHandler<S3Event, String> {
#Override
public String handleRequest(final S3Event event, final Context context) {
System.out.println("Create object was called on the S3 bucket");
S3EventNotification.S3EventNotificationRecord record = event.getRecords().get(0);
String srcBucket = record.getS3().getBucket().getName();
String srcKey = record.getS3().getObject().getUrlDecodedKey();
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(DefaultAWSCredentialsProviderChain.getInstance())
.build();
S3Object s3Object = s3Client.getObject(new GetObjectRequest(
srcBucket, srcKey));
String presignedS3Url = //Assume that I have this by making an external API call
InputStreamReader inputStreamReader = parseFileFromS3(s3Object); #Step 2
int responseCode = putContentIntoS3URL(inputStreamReader, presignedS3Url); #Step 3
}
Step 2: Parse the file into an InputStreamReader to copy it to an OutputStreamWriter:
private InputStreamReader parseFileFromS3(S3Object s3Object) {
return new InputStreamReader(s3Object.getObjectContent(), StandardCharsets.UTF_8);
}
Step 3: Make a HTTP PUT call by copying the contents from InputStreamReader to OutputStreamWriter:
private int putContentIntoS3URL(InputStreamReader inputStreamReader, String presignedS3Url) {
URL url = null;
try {
url = new URL(presignedS3Url);
} catch (MalformedURLException e) {
e.printStackTrace();
}
HttpURLConnection httpCon = null;
try {
assert url != null;
httpCon = (HttpURLConnection) url.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
httpCon.setDoOutput(true);
try {
httpCon.setRequestMethod("PUT");
} catch (ProtocolException e) {
e.printStackTrace();
}
OutputStreamWriter outputStreamWriter = null;
try {
outputStreamWriter = new OutputStreamWriter(
httpCon.getOutputStream());
} catch (IOException e) {
e.printStackTrace();
}
try {
IOUtils.copy(inputStreamReader, outputStreamWriter);
} catch (IOException e) {
e.printStackTrace();
}
try {
outputStreamWriter.close();
} catch (IOException e) {
e.printStackTrace();
}
try {
httpCon.getInputStream();
} catch (IOException e) {
e.printStackTrace();
}
int responseCode = 0;
try {
responseCode = httpCon.getResponseCode();
} catch (IOException e) {
e.printStackTrace();
}
return responseCode;
}
The issue with the among approach is that when I read an .xlsx file via an S3 insert trigger and PUT into the URL, when I download the uploaded file - it gets downloaded as some gibberish.
When I try reading in a .png file and PUT into the URL, when I download the uploaded file - it gets downloaded as some text file with some gibberish (I did see the word PNG in it though)
It feels like I'm making mistakes with:
Incorrectly creating an OutputStreamWriter since I don't understand how to send a file via a HTTP request
Assuming that every file type can be handled in a generic way.
Not setting the content-type in the HTTP request
Expecting S3 to magically understand my file type after the PUT operation
I would like to know if my above 4 assumptions are correct or incorrect.
The intention is that, I do the PUT on the file data correctly so it sits in the S3 bucket along with the correct file type/extension. I hope my effort is worthy to garner some help. I've done a lot of searching into HTTP PUT and File/IO, but I'm unable to LINK them together for my use-case, since I perform a File I/O followed by a HTTP PUT.
UPDATE 1:
I've added the setRequestProperty("Content-Type", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"), but the file doesn't sit in the S3 bucket with the file extension. It simply sits there as an object.
UPDATE 2:
I think this also has something to do with setContentDisposition() header, although I'm not sure how I go about setting these headers for Excel files.
UPDATE 3:
This may simply have to do with how the Presigned S3 URL itself is vended out to us. As mentioned in the question, I said that we get the Presigned S3 URL from some other team. The question itself has multiple parts that need answering.
Does the default Presigned S3 URL ALLOW clients to set the content-type and content-disposition in the HTTP header?: I've set up another separate question here since it's quite unclear: Can a client set file name and extension programmatically when he PUTs file content to a presigned S3 URL that the service vends out?
If the answer to above question is TRUE, then and only then must we go into how to set the file contents and write it to the OutputStream

You are using InputStreamReader and OutputStreamWriter, which are both bridges between a byte stream and a character stream. However, you are using these with byte data, which means you first convert your bytes to characters, and then back to bytes. Since your data is not character data, this conversion might explain why you get gibberish as a result.
I'd start trying to get rid of the reader and writer, instead directly using the InputStream (which you already got from s3Object.getObjectContent()), and the OutputStream (which you got from httpCon.getOutputStream()). IOUtils.copy should also support this.
Also as a side note, when you construct the InputStreamReader you set StandardCharsets.UTF_8 as the charset to use, but when you construct the OutputStreamWriter you don't set the charset. Should the default charset not be UTF-8, this conversion would probably also result in gibberish.

How to upload multiple files in Play Framework using Java

Hi i have been trying to upload image file in Play Framework. I have been trying out with Java File Upload since morning but unable to do so. I have seen [JavaFileUpload][1] tutorial available on framework website. But i am still not successful. Here is my code which i am trying to run:
Http.MultipartFormData body = request().body().asMultipartFormData();
List<Http.MultipartFormData.FilePart> fileParts = body.getFiles();
for (Http.MultipartFormData.FilePart filePart : fileParts) {
String filename = filePart.getFilename();
File file = filePart.getFile(); //error comes on this line
if (filePart.getFilename().toLowerCase().endsWith(".png")) {
//saving here but how?
} else {
return badRequest("Invalid request, only PNGs are allowed.");
}
}
but problem is that whenever i try to get the file i am having this conversion error:
java.lang.Object cannot be converted to java.io.File
Anyone can guide me in the direction? if we see the official document there is no proper documentation on how to upload multiple files. If anyone can show me some website which can helps me in that direction that will be also helpful

I'm using Play 2.4 and
FilePart filePart = request().body().asMultipartFormData()
.getFile("myFileKey");
File file = filePart.getFile();
With Play 2.2 I used for multiple file uploads:
MultipartFormData mfd = request().body().asMultipartFormData();
List<FilePart> filePartList = mfd.getFiles();
FilePart filePart = filePartList.get(0);

So after lots of trouble i was able to figure out the answer to my question. Here i am going to post the answer so it helps other people searching the answer to the same problem i faced
The controller function call which will upload the files looks like this:
Http.MultipartFormData body = request().body().asMultipartFormData();
List<Http.MultipartFormData.FilePart> fileParts = body.getFiles();
for (Http.MultipartFormData.FilePart filePart : fileParts) {
if (filePart.getFilename().toLowerCase().endsWith(".png")) {
String filename = filePart.getFilename();
Files.write(Paths.get(filename + ".png"), readContentIntoByteArray((File) filePart.getFile()));
} else {
return badRequest("Invalid request, only PNGs are allowed.");
}
}
I am using a function call to read the content of the file into byte array and save them inside the file:
private static byte[] readContentIntoByteArray(File file) {
FileInputStream fileInputStream = null;
byte[] bFile = new byte[(int) file.length()];
try {
//convert file into array of bytes
fileInputStream = new FileInputStream(file);
fileInputStream.read(bFile);
fileInputStream.close();
} catch (Exception e) {
e.printStackTrace();
}
return bFile;
}
Remember you can choose whatever the path you want to save the file at Paths.get(filename + ".png")

Create and download CSV file in a Java servlet

I am working on Java ExtJS application in which I need to create and download a CSV file.
On clicking a button I want a CSV file to be downloaded to a client's
machine.
On buttons listener I am calling a servlet using AJAX. There I am
creating a CSV file.
I don't want the CSV file to be saved in the server. I want the file should be created dynamically with a download option. I want the contents of a file to be created as a string and then I will serve the content as file in which it will open as download mode in browser (this I have achieved in other language, but not sure how to achieve it in Java).
Here is my code only to create a CSV file, but I really don't want to create or save CSV file if I can only download the file as CSV.
public String createCSV() {
try {
String filename = "c:\\test.csv";
FileWriter fw = new FileWriter(filename);
fw.append("XXXX");
fw.append(',');
fw.append("YYYY");
fw.append(',');
fw.append("ZZZZ");
fw.append(',');
fw.append("AAAA");
fw.append(',');
fw.append("BBBB");
fw.append('\n');
CSVResult.close();
return "Csv file Successfully created";
} catch(Exception e) {
return e.toString();
}
}
Can any one help me on this.
Thanks

I got the solution and I am posting it below.
public void doGet(HttpServletRequest request, HttpServletResponse response)
{
response.setContentType("text/csv");
response.setHeader("Content-Disposition", "attachment; filename=\"userDirectory.csv\"");
try
{
OutputStream outputStream = response.getOutputStream();
String outputResult = "xxxx, yyyy, zzzz, aaaa, bbbb, ccccc, dddd, eeee, ffff, gggg\n";
outputStream.write(outputResult.getBytes());
outputStream.flush();
outputStream.close();
}
catch(Exception e)
{
System.out.println(e.toString());
}
}
Here we don't need to save / store the file in the server.
Thanks

First of all you need to get the HttpServletResponse object so that you can stream a file into it.
Note : This example is something I Wrote for one of my projects and it works.Works on Java 7.
Assuming you got the HttpServletResponse you can do something like this to stream a file. This way the file will be saved into clients' machine.
public void downloadFile(HttpServletResponse response){
String sourceFile = "c:\\source.csv";
try {
FileInputStream inputStream = new FileInputStream(sourceFile);
String disposition = "attachment; fileName=outputfile.csv";
response.setContentType("text/csv");
response.setHeader("Content-Disposition", disposition);
response.setHeader("content-Length", String.valueOf(stream(inputStream, response.getOutputStream())));
} catch (IOException e) {
logger.error("Error occurred while downloading file {}",e);
}
}
And the stream method should be like this.
private long stream(InputStream input, OutputStream output) throws IOException {
try (ReadableByteChannel inputChannel = Channels.newChannel(input); WritableByteChannel outputChannel = Channels.newChannel(output)) {
ByteBuffer buffer = ByteBuffer.allocate(10240);
long size = 0;
while (inputChannel.read(buffer) != -1) {
buffer.flip();
size += outputChannel.write(buffer);
buffer.clear();
}
return size;
}
}
What this does is, get an inputstream from your source file and write that stream into the outputstream of the HttpServletResponse. This should work since it works perfectly for me. Hope this helps. Sorry for my bad English.

I would like add something to the answer by gaurav. I recently had to implment this functionality in a project of mine and using javascript was out of the question becuase we had to support IE 9. What is the problem with IE 9?
(Export to CSV using jQuery and html), see the second answer in the link.
I needed an easy way to convert a ResultSet of a database query to a string which represent the the same data in CSV format. For that I used http://opencsv.sourceforge.net/ which provided an easy way to get a String ot of the ResultSet, and the rest is as above answer did it.
THe examples in the project soruce folder give good examples.

upload image in java and save it on server. Without Html5

I want to upload image file on server using servlet and i am not using HTML5. I have seen many questions of uploading image on stackoverflow but most of the answers are using PHP. I have tried it by reading image file at client side in java script using FileReader.readAsDataURL() method and then send this to server side and again make a .jpg file by decrypting it by BASE64. But the file made is not readable by computer. So please help me to solve this problem. Some other method for uploading are also appreciated. Please Answer me regarding JAVA JAVA JAVA. I also want to use AJAX :) Thank you.
I have done uploading Image file using servlet. It is working fine. I am trying to upload image using AJAX. I am calling same servlet through the AJAX request. But its not working. I am using common- fileupload.jar and common-io.jar for the same. Snippet looks like:
List items = new FileUpload(new DiskFileItemFactory())
.parseRequest(request);
This code not working with Ajax I am sending dataForm object as data.
data = new FormData();
jQuery.each($('#file')[0].files, function(i, file) {
data.append('file-'+i, file);
});
The ajax request looks like:
$.ajax({
type : "POST",
data: data,
cache: false,
url : "/uploadImage/upload",
contentType: false,
processData: false,
success : function() {
alert("Done..!!");
}
});

1) Create a html form first to sent multiparty request.you can see the blow link to create a html form to upload file http://www.caucho.com/resin-3.0/jsp/tutorial/multipart.xtp
2) Create a java dynamic web project and add a servert into it.
take a look this sample http://helloworldprograms.blogspot.com/2010/12/dynamic-web-project-eclipse.html
3) create a servlet and use this code for uploading i am use com.oreilly.servlet.MultipartRequest library
private void uploadImage(HttpServletResponse response,HttpServletRequest request) throws IOException
{
String dirName = request.getSession().getServletContext().getRealPath("/temp/");
String fileName = null;
MultipartRequest multi=null;
try{
multi= new MultipartRequest(request, dirName,1024*1024, "ISO-8859-1");
}catch (IOException ex){
ex.printStackTrace();
}
try{
Enumeration files = multi.getFileNames();
while (files.hasMoreElements()){
String name = (String) files.nextElement();
fileName = multi.getFilesystemName(name);
}
String filePath = dirName + System.getProperty("file.separator") + fileName;
File clientImage = new File(filePath);
} catch (Exception ioe){
ioe.printStackTrace();
logger.info("\nFile does not exist or cant delete\n");
}
}

JasperReport: how to download generated report?

I'm developing a web application on which user able to generate reports of some data. I'm using JasperReport do it. It's my first experience with JasperReport. So I've followed this tutorial.
Method described there can save reports on server, but I need that report would be downloadable after generation.
Could appreciate some help, thanks.

Your code should look something like this :
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
try {
String path = getServletContext().getRealPath("/reports/report1.jrxml");
jasReport = JasperCompileManager.compileReport(path);
System.out.println("Jasper Report : " + jasReport);
Connection con = MyConnFactory.getConnection();
System.out.println(con);
jasPrint = JasperFillManager.fillReport(jasReport,null,con);//, mapParam, con);
System.out.println("Jasper Print : " + jasPrint);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
// JasperExportManager.exportReportToPdfFile(jasPrint, "f:/nn.pdf");
// JasperExportManager.exportReportToPdfStream(jasPrint, baos);
// ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
// inputStream = bais;
ServletOutputStream sos=resp.getOutputStream();
JasperExportManager.exportReportToPdfStream(jasPrint, sos);
try {
MyConnFactory.getConnection().close();
sos.close();
} catch (SQLException ex) {
Logger.getLogger(MyReport.class.getName()).log(Level.SEVERE, null, ex);
}
} catch (JRException ex) {
Logger.getLogger(MyReport.class.getName()).log(Level.SEVERE, null, ex);
}
}

You will need to set these in your servlet code when trying to download the generated file. May need to change some parameters depending on the file format for your specific case.
response.setContentType("APPLICATION/OCTET-STREAM");
String disHeader = "Attachment;Filename=\"ReportFile.csv" + "\"";
response.setHeader("Content-Disposition", disHeader);

Either:
Save the file to a location accessible directly by the client, or
Stream the bytes back from a file (or other byte stream).
To do #1, you'll either have to save it "inside" the web app, which isn't possible when deploying a war, and tenuous if deploying an exploded war. Symlinks and container games can work around that.
To do #2, save it anywhere, and create a servlet that can take a request parameter that maps to the generated file (or whatever) and streams it back--searching for "download servlet" gives a ton of implementations.
Most frameworks offer similar functionality without writing a pure servlet.