I have a REST API with Restlet 2.3 and need to implement a file-uploading functionality to it.
The problem is that, when someone uploads a file using a POST (with a multipart/form-data Content-Type), the file reaches the server with another encoding.
To test this, I printed the contents of the original file in a Unix Terminal and then printed it again before parsing the requet with Apache Commons FileUpload (with almost the same code of this example http://restlet.com/technical-resources/restlet-framework/guide/2.2/extensions/fileupload). Both printed contents are very similar, but the original file has less characters, so i assume that my Java server is using the wrong encoding to interpret the file.
The file I sent is a PNG image. With text files the server works perfectly, but when I send photos or any binary file, the problem appears.
I don't know how you exactly did to check the received content. First you should check the content type that is used for your file part within the content of your multipart request. You should have something like that for a JPG image:
-----------------------------75956101888331271337088331
Content-Disposition: form-data; name="fileToUpload"; filename="myimage.jpg"
Content-Type: image/jpeg
Secondly, I don't know how you actually write the content you received. Apache Commons IO brings an utility method IOUtils.copy that provides a simple solution to write in an OutputStream the content received from an InputStream. See how ti can be used in your context:
while (fileIterator.hasNext()) {
FileItemStream fi = fileIterator.next();
if (fi.getFieldName().equals("fileToUpload")) {
FileOutputStream fos = new FileOutputStream(
"output"+File.separator+fi.getFieldName());
IOUtils.copy(fi.openStream(), fos);
fos.close();
}
}
IMO, the encoding aspect only applies for text not for binary content.
Hope it helps,
Thierry
I actually solved it by using Google's ByteStreams class:
while (fileIterator.hasNext()) {
FileItemStream fi = fileIterator.next();
if (fi.getFieldName().equals(FILE_TO_UPLOAD)) {
byte[] byteArray = ByteStreams.toByteArray(fi.openStream());
result = new String(byteArray,Charset.forName("ISO-8859-1"));
}
}
I had the similar problem when uploading the image file. This is how I fixed. The problem was in my case the data read from the inputstream. As it is reading from a socket no guarantee that you will have the full buffer of your array filled. Therefore you should check your data size before writing it to the outputbuffer/file. Here is my code hope it helps. Also available in repository https://github.com/esabilbulbul/java-servlet-fileupload/blob/master/README.md
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
upload.setHeaderEncoding("UTF-8");
// Parse the request
FileItemIterator iter = upload.getItemIterator(request);
while (iter.hasNext())
{
FileItemStream item = iter.next();
String name = item.getFieldName();
//InputStream attachmentStream = item.openStream();
//byte[] attachmentBytes = ByteStreams.toByteArray(attachmentStream);
//InputStream stream = item.getInputStream();
InputStream stream = item.openStream();
if (item.isFormField())
{
//System.out.println("Form field " + name + " with value " + Streams.asString(stream) + " detected.");
}
else
{
System.out.println("File field " + name + " with file name "+ item.getName() + " detected.");
// Process the input stream
FileOutputStream fout= new FileOutputStream ("c:\\" + item.getName());
BufferedOutputStream bout= new BufferedOutputStream (fout);
BufferedInputStream bin= new BufferedInputStream(stream);
byte buf[] = new byte[2048];
int len=0;
while ((len = bin.read(buf)) > 0)//((bin.read(buf)) != -1)
{
bout.write(buf, 0, len);
if (len<2048)
len = len;
}
bout.close();
bin.close();
}
}
Related
I am using the following code to get the content from an object in s3 bucket. I am able to copy the data into a file locally, but the file needs to be 'downloaded' and it has to be shown in the browser's downloads list.
I searched a bit about this and cam to know this has something to do with response.setHeader( "Content-Disposition", "attachment;filename=\"" + filename + "\"" );
I tried to add that too but somehow couldn't get it to work. My understanding is that the source has to be a file on a server which is converted into a stream. but I get the s3 contents in the form of a input stream. How to i download this as a file in the browser?
Below is the code i have tried so far
AmazonS3 s3 = new AmazonS3Client(new ProfileCredentialsProvider());
S3Object fetchFile = s3.getObject(new GetObjectRequest(bucketname, fileloc));
final BufferedInputStream i = new BufferedInputStream(fetchFile.getObjectContent());
response.setContentType("application/json");
response.setHeader( "Content-Disposition", "attachment;filename=\"" + filename + "\"" );
ServletOutputStream sos = response.getOutputStream();
int bytesread = i.read();
while(bytesread!=-1)
{
sos.write(bytesread);
bytesread = i.read();
}
if(i!= null)i.close();
if(sos!= null)sos.close();
Everything is correct except the file reading part.
//Set the size of buffer to stream the data.
byte []buffer=new byte[1024*8];
instead of
int byte=i.read();
Now read the file.
while( ( length = yourInputStream.read(buffer))!=-1)
{ yourOutputStream.write(buffer);
}
System.out.println("File is downloaded.");
Additionally,puting your whole code within try/catch block will help you to know the exact reason of your problem.
<%
Document downloadFile = null;
String mimeType = null;
try{
downloadFile = new DocumentsDao().loadById(Long.parseLong(request.getParameter("id")));
// gets MIME type of the file
mimeType = downloadFile.getFileType();
if (mimeType == null) {
// set to binary type if MIME mapping not found
mimeType = "application/octet-stream";
}
System.out.println("MIME type: " + mimeType);
}catch (Exception e){
return;
}
// modifies response
response.reset();
response.resetBuffer();
response.setContentType(mimeType);
response.setContentLength((int) downloadFile.getDocumentData().length);
// forces download
String headerKey = "Content-Disposition";
String headerValue = String.format("attachment; filename=\"%s\"", downloadFile.getFileName());
response.setHeader(headerKey, headerValue);
// obtains response's output stream
OutputStream outStream = response.getOutputStream();
byte[] buffer = new byte[8192];
int bytesRead = -1;
System.out.println("### Length from db = "+downloadFile.getDocumentData().length);
ByteArrayInputStream inStream = new ByteArrayInputStream(downloadFile.getDocumentData());
while ((bytesRead = inStream.read(buffer)) != -1) {
outStream.write(buffer, 0, bytesRead);
}
inStream.close();
outStream.close();
response.flushBuffer();
return;
%>
The above code in a JSP produces a file to download which has an additional sequence of 0d0a x 4 at the end which causes the microsoft applications word and excel to complain and have to repair the file which has been downloaded.
I thought it might be the upload of the file but it was not, and retrieving from the database is fine. So the input stream is fine the problem occurs after the output stream is closed.
Errors you get are 'Word found unreadable content' 'Excel found unreadable content'
Has anyone seen this?
cheers
Charlie
The conversion of the JSP to servlet introduced the 0d0a X 4 characters. I used fiddle to find that the Apache Tomcat web server was altering the content length and sending the bytes. I looked at working apps at work and they all used servlets to do the job so I converted the above code to a servlet and it worked perfectly.
So don't use JSPs for this purpose.
I encode a pdf into a base64 byte[] stream and I want to send it as a http response to the browser. The problem is that the browser fails to load pdf.
I compared the base 64 string which I printed into the IDE console and the one from the browser console. The one from the IDE console is correct and the one from the browser has extra characters.
So, my base64 byte[]stream gets broken somehow when it's sent as a http response ? How do I solve this?
L.e. : The code
FileInputStream inputFileInputStream = null;
ServletOutputStream outputFileOutputStream = null;
File exportFile = new File(exportedReport);
int fileSize = (int) exportFile.length();
String fullPathToExport = exportFile.getAbsolutePath();
File fullPathFile = new File(fullPathToExport);
try {
// test to see if the path of the file is correct
System.out.println("The file is located at: "
+ exportFile.getAbsolutePath());
response.reset();
response.setContentType("application/pdf");
response.setContentLength(fileSize);
response.addHeader("Content-Transfer-Encoding", "base64");
response.setHeader( "Content-Disposition", "inline; filename=\"" + exportedReport +"\"");
inputFileInputStream = new FileInputStream(fullPathFile);
outputFileOutputStream = response.getOutputStream();
if (bytesToRead == -1) {
bytesToRead = (int)fullPathFile.length();
}
byte[] buffer = new byte[bytesToRead];
int bytesRead = -1;
while((inputFileInputStream != null) && ((bytesRead = inputFileInputStream.read(buffer)) != -1)){
if (codec.equals("base64")) {
//outputFileOutputStream.write(Base64.encodeBytes(buffer).getBytes("UTF-8"), 0, bytesToRead);
outputFileOutputStream.write(org.apache.commons.codec.binary.Base64.encodeBase64(buffer));
} else {
outputFileOutputStream.write(buffer, 0, bytesToRead);
}
}
inputFileInputStream.close();
outputFileOutputStream.flush();
outputFileOutputStream.close();
Your code has one major problem:
You are not sending one base64 encoded data part but many base64 encoded data parts (concatenated). But two or more base64 encoded data parts are not equal to one base64 encoded data part.
Example:
base64("This is a test") -> "VGhpcyBpcyBhIHRlc3Q="
base64("This ")+base64("is a ")+base64("test") -> "VGhpcyA=aXMgYSA=dGVzdA=="
You should use the org.apache.commons.codec.binary.Base64InputStream instead of the Base64.encodeBase64() utility method. Read the whole FileInputStream through it and you will get a valid base64 encoded data stream.
Anyway what you are doing is not necessary. You can transfer any 8 bit data via HTTP without further encoding.
I am sending images from my android client to java jersey restful service and I succeded in doing that.But my issue is when I try to send large images say > 1MB its consumes more time so I like to send image in CHUNKS can anyone help me in doing this.How to send(POST) image stream in CHUNKS to server
references used :
server code & client call
server function name
/*** SERVER SIDE CODE****/
#POST
#Path("/upload/{attachmentName}")
#Consumes(MediaType.APPLICATION_OCTET_STREAM)
public void uploadAttachment(
#PathParam("attachmentName") String attachmentName,
#FormParam("input") InputStream attachmentInputStream) {
InputStream content = request.getInputStream();
// do something better than this
OutputStream out = new FileOutputStream("content.txt");
byte[] buffer = new byte[1024];
int len;
while ((len = in.read(buffer)) != -1) {
// whatever processing you want here
out.write(buffer, 0, len);
}
out.close();
return Response.status(201).build();
}
/**********************************************/
/**
CLIENT SIDE CODE
**/
// .....
client.setChunkedEncodingSize(1024);
WebResource rootResource = client.resource("your-server-base-url");
File file = new File("your-file-path");
InputStream fileInStream = new FileInputStream(file);
String contentDisposition = "attachment; filename=\"" + file.getName() + "\"";
ClientResponse response = rootResource.path("attachment").path("upload").path("your-file-name")
.type(MediaType.APPLICATION_OCTET_STREAM).header("Content-Disposition", contentDisposition)
.post(ClientResponse.class, fileInStream);
You should split the file in the client and restore part of the file in the server.
and after that you should merge the files together. Take a look at split /merge file on coderanch
Enjoy ! :)
Another path is available, if you don't want to code too much consider using :
file upload apache that is great ! :)
hi i have tried the following java codes which works fine if i use them as a java application but when i use the same code in my servlet page they dont work means i am not able to download the files. Please suggest what changes should i do so that i can download the file using Servlets.
a.
java.io.BufferedInputStream in = new java.io.BufferedInputStream(new java.net.URL("http://169.254.174.150:8084/WebApplication1/files/check.txt").openStream());
File f1 = new File("D:\\a.txt");
java.io.FileOutputStream fos = new java.io.FileOutputStream(f1);
java.io.BufferedOutputStream bout = new BufferedOutputStream(fos, 1024);
byte data[] = new byte[1024];
while (in.read(data, 0, 1024) >= 0) {
bout.write(data);
}
bout.close();
in.close();
}
b. http://www.javabeat.net/examples/2012/04/13/download-file-from-http-https-server-using-java/
One of the older JavaBeat examples like the one you specified can be found here
I found other solutions too but this seems to be the most comprehensive.
Couple of things, insetad of writing it to a file try wrting the data directly to the responce. Before writing data you will have to set the following parameters to the responce
//byte[] filedata = ; intialize your file contents
String filename = "a.txt";
// set the header information in the response.
res.setHeader("Content-Disposition", "attachment; filename=\"" + filename + "\";");
res.setContentType("application/x-unknown");
ByteArrayInputStream byteStream = new ByteArrayInputStream(filedata);
BufferedInputStream bufStream = new BufferedInputStream(byteStream);
ServletOutputStream responseOutputStream = res.getOutputStream();
int data = bufStream.read();
while (data != -1)
{
responseOutputStream.write(data);
data = bufStream.read();
}
bufStream.close();
responseOutputStream.close();
where res is a HttpServletResponse object. After this you can write data to responseOutputStream.