In my servlet, I upload a file to a specific url, then call a method in a different class to grab that file online and alter it. The problem is that a lot of the time, there is a lot of data in the file and the dopost method finishes (and goes to the next page from the upload file submit form) before the file is fully altered. How do I prevent the do post method from going to the next page before my file is completely altered?
I want the Test.preFirstMethod() to finish before the page reloads and makes the download link. Test.preFirstMethod() takes in an excel file and modifies it using dynamic content from the internet.
public class Uploads extends HttpServlet {
private Object lock1 = new Object();
private static final long serialVersionUID = 1L;
private ServletFileUpload uploader = null;
public Thread tt = new Thread(new Runnable() {
public void run()
{
try {Test.preFirstMethod();
} catch (Exception e) {
e.printStackTrace();
}
}
});
int BUFFER_LENGTH = 4096;
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
PrintWriter out = response.getWriter();
String fileName="";
for (Part part : request.getParts()) {
InputStream is = request.getPart(part.getName()).getInputStream();
fileName = getFileName(part);
File f2 = new File(fileName);
FileOutputStream os = new FileOutputStream(System.getenv("OPENSHIFT_DATA_DIR") + "nn.xlsx");
byte[] bytes = new byte[BUFFER_LENGTH];
int read = 0;
while ((read = is.read(bytes, 0, BUFFER_LENGTH)) != -1) {
os.write(bytes, 0, read);
}
os.flush();
is.close();
fileName=fileName.substring(0,fileName.lastIndexOf("."))+"_ZillowAdded.xlsx";
try {
tt.start();
try { tt.join(); } catch (InterruptedException e) {
e.printStackTrace();
}
InputStream is2 = new FileInputStream(f2);
FileOutputStream os2 = new FileOutputStream(System.getenv("OPENSHIFT_DATA_DIR") + fileName);
while ((read = is2.read(bytes, 0, BUFFER_LENGTH)) != -1) {
os2.write(bytes, 0, read);
}
os2.flush();
is2.close();
os2.close();
} catch(Exception e) {e.printStackTrace();}
os.close();
}
if(!ServletFileUpload.isMultipartContent(request)){
throw new ServletException("Content type is not multipart/form-data");
}
response.setContentType("text/html");
out.write("<html><head></head><body>");
out.write("File "+fileName.substring(0,fileName.indexOf("_Zillow"))+ " uploaded successfully.");
out.write("<br>");
out.write("Download "+fileName+"");
out.write("</body></html>"); }
Related
I have been trying to stream|transfer a huge file available in the local file-system over restapi using streamingoutput. I keep running into heapSpace error. Can anyone help me figure out what I am doing wrong? As per my understanding, streamingoutput shouldn't keep the file in memory.
Please find the code below:
public Response getBulkBillDownload(#QueryParam("requestID") String requestID,
#QueryParam("zipFileName") String zipFileName) throws RestException {
StreamingOutput stream = null;
try {
File file = null;
Optional<File> document = getCorporatePaymentManager().getBulkBillDownloadResponse(requestID, zipFileName);
if (document.isPresent()) {
file = document.get();
} else {
throw new RestException("File not found");
}
final FileInputStream fStream = new FileInputStream(file);
// register stream to Response and it will callback with server OutputStream
stream = new StreamingOutput() {
#Override
public void write(OutputStream output) throws IOException, WebApplicationException {
pipe(fStream, output);
}
};
} catch (Exception e) {
handleException(e);
}
return Response.status(200).entity(stream).header("Content-Disposition", "attachment; filename=" + zipFileName)
.build();
}
private void pipe(InputStream is, OutputStream os) throws IOException {
byte[] buf=new byte[1024];
int bytesread = 0, bytesBuffered = 0;
while( (bytesread = is.read( buf )) > -1 ) {
os.write( buf, 0, bytesread );
bytesBuffered += bytesread;
if (bytesBuffered > 1024 * 1024) { //flush after 1MB
bytesBuffered = 0;
os.flush();
}
}
os.close();
}
I have referred many sites, none of them worked for me to check if download was completed 100%
Scenario- I am downloading a file, and I want my selenium/Java program to wait until the download is completed 100%.
(Downloading through HTTP would be the best, but I did not find anything appropriate that would help me out)
Thanks in Advance!!
The below code worked for me,
There are 2 ways:
1st Method:(You need to download AsyncHttpClient JAR)
try {
AsyncHttpClient client = Dsl.asyncHttpClient();
final FileOutputStream stream = new FileOutputStream(FILE_NAME);
client.prepareGet(FILE_URL).execute(new AsyncCompletionHandler<FileOutputStream>() {
#Override
public State onBodyPartReceived(HttpResponseBodyPart bodyPart)
throws Exception {
stream.getChannel().write(bodyPart.getBodyByteBuffer());
return State.CONTINUE;
}
#Override
public FileOutputStream onCompleted(Response response)
throws Exception {
return stream;
}
});
}
catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
2nd Method :
private static void startDownload(String FILE_URL, String FILE_NAME)
{
try (BufferedInputStream in = new BufferedInputStream(new URL(FILE_URL).openStream());
FileOutputStream fileOutputStream = new FileOutputStream(FILE_NAME)) {
byte dataBuffer[] = new byte[1024];
int bytesRead;
while ((bytesRead = in.read(dataBuffer, 0, 1024)) != -1) {
fileOutputStream.write(dataBuffer, 0, bytesRead);
}
} catch (IOException e) {
System.out.println(e);
}
}
I have my code generating two files with rewritable data. I need a code that continues generating the files with recursive file names and should keep all the previous files as well .
In the below code, every time i have to update my file, I have to hard code it and copy it into a new file.
I want a recursive function that saves the file, named numerically in an order(Ascending), while keeping the data in my previous file as well, everytime i run the code.
public static void main(String[] args) throws IOException
{
createFileUsingFileClass();
copyFileVersion();
fileChecker();
String data_2 = "This is the new data written in your file";
writeUsingFileWriter(data_2);
copyFileInCode(data_2);
}
private static void createFileUsingFileClass() throws IOException
{
File file = new File("C:\\Users\\esunrsa\\Documents\\file.txt");
//Create the file
if (file.createNewFile()){
System.out.println("File is created!");
}else{
System.out.println("File already exists.");
}
//Write Content
FileWriter writer = new FileWriter(file);
String data_1 = " Initial data";
writer.write(data_1);
writer.close();
}
private static void copyFileVersion() {
FileInputStream ins = null;
FileOutputStream outs = null;
try {
File infile =new File("C:\\Users\\esunrsa\\Documents\\file.txt");
File outfile =new File("C:\\Users\\esunrsa\\Documents\\file_01.txt");
ins = new FileInputStream(infile);
outs = new FileOutputStream(outfile);
byte[] buffer = new byte[1024];
int length;
while ((length = ins.read(buffer)) > 0) {
outs.write(buffer, 0, length);
}
ins.close();
outs.close();
System.out.println("File created successfully!!");
} catch(IOException ioe) {
ioe.printStackTrace();
}
}
private static void fileChecker() {
File f = new File("C:\\Users\\esunrsa\\Documents\\sunrita.txt");
if(f.exists()){
System.out.println("File existed");
}else{
System.out.println("File doesnt exist");
System.exit(0);
//System.out.println("File not found!");
}
}
private static void writeUsingFileWriter(String data_2) {
File file = new File("C:\\Users\\esunrsa\\Documents\\file.txt");
FileWriter fr = null;
try {
fr = new FileWriter(file);
fr.write(data_2);
} catch (IOException e) {
e.printStackTrace();
}finally{
//close resources
try {
fr.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
private static void copyFileInCode(String filename) {
FileInputStream ins = null;
FileOutputStream outs = null;
try {
File infile =new File("C:\\Users\\esunrsa\\Documents\\file.txt");
File outfile =new File("C:\\Users\\esunrsa\\Documents\\file_02.txt");
ins = new FileInputStream(infile);
outs = new FileOutputStream(outfile);
byte[] buffer = new byte[1024];
int length;
while ((length = ins.read(buffer)) > 0) {
outs.write(buffer, 0, length);
}
ins.close();
outs.close();
System.out.println("File created successfully!!");
} catch(IOException ioe) {
ioe.printStackTrace();
}
}
}
How do I upload a photo using a URL in the playframework?
I was thinking like this:
URL url = new URL("http://www.google.ru/intl/en_com/images/logo_plain.png");
BufferedImage img = ImageIO.read(url);
File newFile = new File("google.png");
ImageIO.write(img, "png", newFile);
But maybe there's another way. In the end I have to get the File and file name.
Example controller:
public static Result uploadPhoto(String urlPhoto){
Url url = new Url(urlPhoto); //doSomething
//get a picture and write to a temporary file
File tempPhoto = myUploadPhoto;
uploadFile(tempPhoto); // Here we make a copy of the file and save it to the file system.
return ok('something');
}
To get that photo you can use The play WS API, the code behind is an example extracted from the play docs in the section Processing large responses, I recommend you to read the full docs here
final Promise<File> filePromise = WS.url(url).get().map(
new Function<WSResponse, File>() {
public File apply(WSResponse response) throws Throwable {
InputStream inputStream = null;
OutputStream outputStream = null;
try {
inputStream = response.getBodyAsStream();
// write the inputStream to a File
final File file = new File("/tmp/response.txt");
outputStream = new FileOutputStream(file);
int read = 0;
byte[] buffer = new byte[1024];
while ((read = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, read);
}
return file;
} catch (IOException e) {
throw e;
} finally {
if (inputStream != null) {inputStream.close();}
if (outputStream != null) {outputStream.close();}
}
}
}
);
Where url is :
String url = "http://www.google.ru/intl/en_com/images/logo_plain.png"
This is as suggested in play documentation for large files:
*
When you are downloading a large file or document, WS allows you to
get the response body as an InputStream so you can process the data
without loading the entire content into memory at once.
*
Pretty much the same as the above answer then some...
Route: POST /testFile 'location of your controller goes here'
Request body content: {"url":"http://www.google.ru/intl/en_com/images/logo_plain.png"}
Controller(using code from JavaWS Processing large responses):
public static Promise<Result> saveFile() {
//you send the url in the request body in order to avoid complications with encoding
final JsonNode body = request().body().asJson();
// use new URL() to validate... not including it for brevity
final String url = body.get("url").asText();
//this one's copy/paste from Play Framework's docs
final Promise<File> filePromise = WS.url(url).get().map(response -> {
InputStream inputStream = null;
OutputStream outputStream = null;
try {
inputStream = response.getBodyAsStream();
final File file = new File("/temp/image");
outputStream = new FileOutputStream(file);
int read = 0;
byte[] buffer = new byte[1024];
while ((read = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, read);
}
return file;
} catch (IOException e) {
throw e;
} finally {
if (inputStream != null) {
inputStream.close();
}
if (outputStream != null) {
outputStream.close();
}
}
}); // copy/paste ended
return filePromise.map(file -> (Result) ok(file.getName() + " saved!")).recover(
t -> (Result) internalServerError("error -> " + t.getMessage()));
}
And that's it...
In order to serve the file after the upload phase you can use this answer(I swear I'm not promoting myself...): static asset serving from absolute path in play framework 2.3.x
I am writing a small file upload utility thing as part of a larger project. Originally I was handling this from a servlet using the Apache commons File utility classes. Here is a snippet from a quick test client I wrote for the service:
public static void main(String[] args) {
JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
factory.getInInterceptors().add(new LoggingInInterceptor());
factory.getOutInterceptors().add(new LoggingOutInterceptor());
factory.setServiceClass(FileUploadService.class);
factory.setAddress("http://localhost:8080/FileUploadService/FileUploadService");
FileUploadService client = (FileUploadService) factory.create();
FileType file = new FileType();
file.setName("statemo_1256144312279");
file.setType("xls");
DataSource source = new FileDataSource(new File("c:/development/statemo_1256144312279.xls"));
file.setHandler(new DataHandler(source));
Boolean ret = client.uploadFile(file);
System.out.println (ret);
System.exit(0);
}
This works absolutely fine. Now the problem comes when I am trying to replace the Apache commons utilities. In the above code I am creating a DataSource from a File with an absolute path name. In my servlet, I can't get an absolute path name however and the file I am sending over the wire is empty.
Here is the servlet code:
#SuppressWarnings("unchecked")
protected void doPost (final HttpServletRequest request, final HttpServletResponse response)
throws ServletException, IOException {
// form should have enctype="multipart/form-data" as an attribute
if (!ServletFileUpload.isMultipartContent (request)) {
LOG.info("Invalid form attribute");
return;
}
//DataInputStream in = new DataInputStream(request.getInputStream());
final DiskFileItemFactory factory = new DiskFileItemFactory ();
factory.setSizeThreshold(FILE_THRESHOLD_SIZE);
final ServletFileUpload sfu = new ServletFileUpload (factory);
sfu.setSizeMax(MAX_FILE_SIZE);
final HttpSession session = request.getSession();
final List<FileItem> files = new ArrayList<FileItem>();
final List<String> filesToProcess = new ArrayList<String>();
try {
final List<FileItem> items = sfu.parseRequest(request);
for (final FileItem f : items) {
if (!f.isFormField())
files.add(f);
}
/*for (final FileItem f : files) {
final String absoluteFileName = UPLOAD_DESTINATION + FilenameUtils.getName(f.getName());
//f.write(new File (absoluteFileName));
filesToProcess.add(absoluteFileName);
}*/
FileItem f = files.get(0);
LOG.info("File: " + FilenameUtils.getName(f.getName()));
LOG.info("FileBaseName: " + FilenameUtils.getBaseName(f.getName()));
LOG.info("FileExtension: " + FilenameUtils.getExtension(f.getName()));
FileUploadServiceClient client = new FileUploadServiceClient();
DataSource source = new FileDataSource(new File(f.getName()));
FileType file = new FileType();
file.setHandler(new DataHandler(source));
file.setName(FilenameUtils.getBaseName(f.getName()));
file.setType(FilenameUtils.getExtension(f.getName()));
Boolean ret = client.uploadFile(file);
LOG.info("File uploaded - " + ret);
filesToProcess.add(UPLOAD_DESTINATION + FilenameUtils.getName(f.getName()));
session.setAttribute("filesToProcess", filesToProcess);
final RequestDispatcher dispatcher = request.getRequestDispatcher("Validate");
if (null != dispatcher) {
dispatcher.forward(request, response);
}
} catch (FileUploadException e) {
LOG.info("Exception " + e.getMessage());
e.printStackTrace();
} catch (Exception e) {
LOG.info("Exception " + e.getMessage());
e.printStackTrace();
}
}
I've been working on this for the better part of this morning and am not getting anywhere. Even if I get rid of the Apache commons file stuff completely and handle the parsing of the request myself, I still can't construct the DataSource appropriately.
Thanks!
This was rather simple actually, I just copied over the bytes from the InputStream to the DataSource:
FileItem f = files.get(0);
// there is a problem here where the file being created is empty, since we only have a
// partial path:
DataSource source = new FileDataSource(new File(f.getName()));
// because of the above problem, we are going to copy over the data ourselves:
byte[] sourceBytes = f.get();
OutputStream sourceOS = source.getOutputStream();
sourceOS.write(sourceBytes);
This is the code of commons-email ByteArrayDataSource
it sounds odd to try to replace apache commons - don't, unless you have a really good reason
you can get absolute paths in a servlet. You can call getServletContext().getRealPath("/") which will return the absolute path of your application, and then you can get files relative to it.
In our application there are objects that have properties InputStream and Name. We are using next class to construct DataSource with those properties.
public class InputStreamDataSource implements DataSource {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
private final String name;
public InputStreamDataSource(InputStream inputStream, String name) {
this.name = name;
try {
int nRead;
byte[] data = new byte[16384];
while ((nRead = inputStream.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
inputStream.close();
buffer.flush();
} catch (IOException e) {
e.printStackTrace();
}
}
#Override
public String getContentType() {
return new MimetypesFileTypeMap().getContentType(name);
}
#Override
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(buffer.toByteArray());
}
#Override
public String getName() {
return name;
}
#Override
public OutputStream getOutputStream() throws IOException {
throw new IOException("Read-only data");
}
}
Most of the solutions shown here require that the InpustStream be closed (read into memory). It is possible to wrap the InputStream in a DataSource object without closing the InputStream though:
private record PipedDataSource(InputStream in, String contentType, String encoding)
implements DataSource, EncodingAware {
public String getContentType() {
return contentType;
}
public InputStream getInputStream() {
return in;
}
public String getName() {
return "PipedDataSource";
}
public OutputStream getOutputStream() throws IOException {
throw new IOException("No OutputStream");
}
#Override
public String getEncoding() {
return encoding;
}
}
The example above also implements EncodingAware. This can prevent the InputStream from being closed by third part libraries (for example java.mail.internet.MimeUtility) when they get the data source encoding.