How can I start a thread which will run in the background but the response will be sent to the user immediately?
I'm trying to upload a file from an android device. While the uploading process works fine, I'd like to tell the user that the process has finished once the file has been uploaded from his end.
I have checked that the call to ServletFileUpload.parseRequest() is what takes a long time.
ServletFileUpload uploadHandler = new ServletFileUpload(fileItemFactory);
List items = uploadHandler.parseRequest(request);
So basically I'm looking for a way where I can tell the user "hey you've done your part", now it's the servlet's turn to do all the processing.
but I can't because parseRequest blocks.
Am I wrong in that it blocks because that's when it's actually being sent by the device?
Uploading on android:
while (bytesRead > 0)
{
task.myPublishProgress(totalBytesRead, fileSize);
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
totalBytesRead+= bytesRead;
}
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// close streams
fileInputStream.close();
dos.flush();
Log.i(TAG ,"Database file with name " + this.fileName + " has been successfully written");
InputStream is = conn.getInputStream();
// retrieve the response from server
int ch;
StringBuilder b = new StringBuilder();
while( ( ch = is.read() ) != -1 ){
b.append( (char)ch );
}
dos.close();
handling on server:
DiskFileItemFactory fileItemFactory = new DiskFileItemFactory ();
ServletFileUpload uploadHandler = new ServletFileUpload(fileItemFactory);
/*
* Parse the request
*/
System.out.println("parsing the request....");
**List items = uploadHandler.parseRequest(request);**
FileItem item = (FileItem) items.get(0);
fileName = item.getName();
File file = new File(destinationDir, fileName);
item.write(file);
I'd like the servlet not too block on parseRequest, but instead handle that in a thread perhaps, and send the response to the client.
thanks
This is not possible in standard HTTP. The response can only be sent when the request body has fully been consumed. As long as you don't call uploadHandler.parseRequest(request);, the files are not been uploaded. You would be lying if you tell "You're done!" prior this point.
Only after calling uploadHandler.parseRequest(request) and having the items, you could tell the user in all honesty that he's done. The remnant can however perfectly be done in a separate thread. I'd suggest to create a fixed threadpool for that which is managed by a ServletContextListener. However, I think this is a bit exaggerated when you aren't doing any expensive tasks with the uploaded files.
As to the concrete problem (that the application seems to do nothing while it is busy to upload the files), I'd rather suggest to solve the problem at the client side. Get the size of the files, get hold of the count of the bytes sent to the outputstream and then show some progress bar in the client side so that the client get informed about the progress.
Related
I have to do a file transfer, in my case a pdf, through socket in java for my homework. Usually I requested text and got text back, but this time I have to send a file through socket. In my investigation I discovered that file transfers are made with Fileinput(output)streams. My problem is that the request to the server has to look something like this:
File file = new File(pathToFile);
Pirntwriter out = new PrintWriter(Socket s.getOutputStream());
Outputstream outFile = s.getOutputStream();
int count
out.write("user file\r\n"
+ file.getName()+"\r\n"
+ file.length()+"\r\n"
+ "body\r\n");
// send file but im not sure how
byte[] buffer = new buffer with size of file.length()
while ((count = in.read(buffer)) > 0){
outFile.write(buffer, 0, count);
}
out.flush
outFile.flush
Unfortunately this doesn't work for me. In this way the server counts the requests as two different outputs. Is there a way to combine both Outputstreams or write the request in one single Outputstream?
I have a REST API with Restlet 2.3 and need to implement a file-uploading functionality to it.
The problem is that, when someone uploads a file using a POST (with a multipart/form-data Content-Type), the file reaches the server with another encoding.
To test this, I printed the contents of the original file in a Unix Terminal and then printed it again before parsing the requet with Apache Commons FileUpload (with almost the same code of this example http://restlet.com/technical-resources/restlet-framework/guide/2.2/extensions/fileupload). Both printed contents are very similar, but the original file has less characters, so i assume that my Java server is using the wrong encoding to interpret the file.
The file I sent is a PNG image. With text files the server works perfectly, but when I send photos or any binary file, the problem appears.
I don't know how you exactly did to check the received content. First you should check the content type that is used for your file part within the content of your multipart request. You should have something like that for a JPG image:
-----------------------------75956101888331271337088331
Content-Disposition: form-data; name="fileToUpload"; filename="myimage.jpg"
Content-Type: image/jpeg
Secondly, I don't know how you actually write the content you received. Apache Commons IO brings an utility method IOUtils.copy that provides a simple solution to write in an OutputStream the content received from an InputStream. See how ti can be used in your context:
while (fileIterator.hasNext()) {
FileItemStream fi = fileIterator.next();
if (fi.getFieldName().equals("fileToUpload")) {
FileOutputStream fos = new FileOutputStream(
"output"+File.separator+fi.getFieldName());
IOUtils.copy(fi.openStream(), fos);
fos.close();
}
}
IMO, the encoding aspect only applies for text not for binary content.
Hope it helps,
Thierry
I actually solved it by using Google's ByteStreams class:
while (fileIterator.hasNext()) {
FileItemStream fi = fileIterator.next();
if (fi.getFieldName().equals(FILE_TO_UPLOAD)) {
byte[] byteArray = ByteStreams.toByteArray(fi.openStream());
result = new String(byteArray,Charset.forName("ISO-8859-1"));
}
}
I had the similar problem when uploading the image file. This is how I fixed. The problem was in my case the data read from the inputstream. As it is reading from a socket no guarantee that you will have the full buffer of your array filled. Therefore you should check your data size before writing it to the outputbuffer/file. Here is my code hope it helps. Also available in repository https://github.com/esabilbulbul/java-servlet-fileupload/blob/master/README.md
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
upload.setHeaderEncoding("UTF-8");
// Parse the request
FileItemIterator iter = upload.getItemIterator(request);
while (iter.hasNext())
{
FileItemStream item = iter.next();
String name = item.getFieldName();
//InputStream attachmentStream = item.openStream();
//byte[] attachmentBytes = ByteStreams.toByteArray(attachmentStream);
//InputStream stream = item.getInputStream();
InputStream stream = item.openStream();
if (item.isFormField())
{
//System.out.println("Form field " + name + " with value " + Streams.asString(stream) + " detected.");
}
else
{
System.out.println("File field " + name + " with file name "+ item.getName() + " detected.");
// Process the input stream
FileOutputStream fout= new FileOutputStream ("c:\\" + item.getName());
BufferedOutputStream bout= new BufferedOutputStream (fout);
BufferedInputStream bin= new BufferedInputStream(stream);
byte buf[] = new byte[2048];
int len=0;
while ((len = bin.read(buf)) > 0)//((bin.read(buf)) != -1)
{
bout.write(buf, 0, len);
if (len<2048)
len = len;
}
bout.close();
bin.close();
}
}
I am sending images from my android client to java jersey restful service and I succeded in doing that.But my issue is when I try to send large images say > 1MB its consumes more time so I like to send image in CHUNKS can anyone help me in doing this.How to send(POST) image stream in CHUNKS to server
references used :
server code & client call
server function name
/*** SERVER SIDE CODE****/
#POST
#Path("/upload/{attachmentName}")
#Consumes(MediaType.APPLICATION_OCTET_STREAM)
public void uploadAttachment(
#PathParam("attachmentName") String attachmentName,
#FormParam("input") InputStream attachmentInputStream) {
InputStream content = request.getInputStream();
// do something better than this
OutputStream out = new FileOutputStream("content.txt");
byte[] buffer = new byte[1024];
int len;
while ((len = in.read(buffer)) != -1) {
// whatever processing you want here
out.write(buffer, 0, len);
}
out.close();
return Response.status(201).build();
}
/**********************************************/
/**
CLIENT SIDE CODE
**/
// .....
client.setChunkedEncodingSize(1024);
WebResource rootResource = client.resource("your-server-base-url");
File file = new File("your-file-path");
InputStream fileInStream = new FileInputStream(file);
String contentDisposition = "attachment; filename=\"" + file.getName() + "\"";
ClientResponse response = rootResource.path("attachment").path("upload").path("your-file-name")
.type(MediaType.APPLICATION_OCTET_STREAM).header("Content-Disposition", contentDisposition)
.post(ClientResponse.class, fileInStream);
You should split the file in the client and restore part of the file in the server.
and after that you should merge the files together. Take a look at split /merge file on coderanch
Enjoy ! :)
Another path is available, if you don't want to code too much consider using :
file upload apache that is great ! :)
I've spent the last day or so trying to debug this issue and I'm out of ideas. Basically I have an Android app that is POSTing some data to a PHP/Apache web server. This code seems to work fine when I point it at my local test server. It also seems to work fine when I point it at my production server, but ONLY when I comment out the line conn.setChunkedStreamingMode(maxBufferSize);. Once that line is enabled, the post only works on my local test server, but when posting to the production server, the PHP $_FILES array is empty. I've tried passing numerous values to setChunkedStreamingMode (including 0 and 1024) but none of these seem to fix the problem.
At this point I'm assuming the issue has to do with the way the production server's PHP is configured, but as far as I can tell, all the important parameters on the server are the same as on my test instance. Additionally, they're both running the same version of Apache and PHP. My production server is run by Bluehost.
Here's the Java code I'm using to upload:
HttpURLConnection conn = null;
DataOutputStream dos = null;
DataInputStream inStream = null;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "***************************************************";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 212144; // 1024*1024 = 1MB. 212144 is a quarter MB.
FileInputStream fileInputStream = null;
try
{
// ------------------ CLIENT REQUEST
fileInputStream = new FileInputStream(new File(existingFileWithFullPath));
// open a URL connection to the Servlet
URL url = new URL(BACKUP_POST_URL);
// Open a HTTP connection to the URL
conn = (HttpURLConnection) url.openConnection();
// Allow Inputs
conn.setDoInput(true);
// Allow Outputs
conn.setDoOutput(true);
// Send in chunks (to avoid out of memory error)
conn.setChunkedStreamingMode(maxBufferSize);
// Don't use a cached copy.
conn.setUseCaches(false);
// Use a post method.
conn.setRequestMethod("POST");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("Content-Type", "multipart/form-data;boundary="
+ boundary);
conn.setReadTimeout(200000); // 200 seconds...
dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\""
+ fileName + "\"" + lineEnd);
dos.writeBytes(lineEnd);
// create a buffer of maximum size
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0)
{
try {
dos.write(buffer, 0, bufferSize);
} catch (OutOfMemoryError oome) {
Log.e(CommonStatic.LOG_NAME, "Out of memory error caught...");
oome.printStackTrace();
fileInputStream.close();
throw new Exception("Out Of Memory!");
}
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
fileInputStream.close();
dos.flush();
dos.close();
// close streams
Log.d(CommonStatic.LOG_NAME, "Backup file written to server successfully...");
}
catch (Exception ex)
{
Log.e(CommonStatic.LOG_NAME, "Backup File Upload Error: " + ex.getMessage(), ex);
throw new Exception (c.getString(R.string.SAVE_TO_CLOUD_ERROR));
}
And here's the PHP code I'm using on the other end to recieve:
$target = "userfiles/";
$target = $target . basename( $_FILES['uploadedfile']['name']);
if(move_uploaded_file($_FILES['uploadedfile']['tmp_name'], $target))
{
echo "SUCCESS";
}
else
{
echo "FAIL";
}
I stuck a print_r($_FILES); at the very beginning of the script to determine that $_FILES is empty on the production instance, but not on the test instance. Any ideas would be greatly appreciated.
Unfortunately I wasn't able to find a direct solution to this problem, but I was able to find a workaround. Instead of using conn.setChunkedStreamingMode I used conn.setFixedLengthStreamingMode, which I had tried before with no success. The key to getting conn.setFixedLengthStreamingMode to work is to pass it the full length of the data you are attempting to send (in this case the file), plus the length of the headers. Fortunately header length is usually fixed in code such as this, so once you figure out how big your header is (keeping in mind things like the filename, which gets sent in the header under Content-Disposition is also variable) you can just put it in as a fixed number. To figure out my header length, I ran the code first without specifying any length for the header. The error message I received back gave me an expected and an actual value for the number of bytes sent, which allowed me to calculate the header length.
I'm running a web app that provides a servlet. this servlet opens a pdf file from a Network File System and finally streams it to the requesting browser.
All the pdf files are linearized by adobe lifecycle pdf generator and ready for fast web view.
unfortunately, the fast web view does not work. I guess it's a problem of how to open and stream the file in java code and the setting of response header info.
if i deploy a test pdf within my webapp onto the jboss AS and open it directly from the browser by url, the incrementel loading works.
can anyone help me?
Here's the code of my servlet:
response.setContentType("application/pdf");
response.setHeader("Expires", "0");
response.setHeader("Cache-Control",
"must-revalidate, post-check=0, pre-check=0");
response.setHeader("Content-Disposition",
"inline;filename=" + documentReference);
response.setHeader("Accept-Ranges", "bytes");
File nfsPDF = new File(NFS_DIRECTORY_PATH + documentReference);
FileInputStream fis = new FileInputStream(nfsPDF);
BufferedInputStream bis = new BufferedInputStream(fis);
ServletOutputStream sos = response.getOutputStream();
byte[] buffer = new byte[(int) nfsPDF.length()];
while (true) {
int bytesRead = bis.read(buffer, 0, buffer.length);
if (bytesRead < 0) {
break;
}
sos.write(buffer, 0, bytesRead);
}
sos.flush();
//... closing...
Let's see. You want to send a file in parts, right? Then you should check Range header (HTTP Header) and send only bytes in this range. I'm correct?
I'm not familiar with "PDF fast web view" feature, but in you're code your're first reading the file completely into buffer and then you write it out. The client won't receive anything before the call to sos.flush(). In fact your while loop is obsolete because there will always be just one run.
Maybe you should try to read/write the stuff blockwise.
byte[] buffer = new byte[1024];
while (true) {
int bytesRead = bis.read(buffer, 0, buffer.length);
if (bytesRead < 0) {
break;
}
sos.write(buffer, 0, bytesRead);
sos.flush();
}
sos.flush();