I try to upload a big file on Google cloud with wicket. I use FileUploadField and UploadFile methods. Nevertheless I can only upload small files (less than 10kb). If I upload a bigger file I obtain an exception (java.security.AccessControlException: access denied). I do not have the permission to create a buffer file and write.
final FileUploadField FiletoUpload = new FileUploadField("uploadfile", new Model());
form.add(FiletoUpload);
form.add(new Button("upload") {
#Override
public void onSubmit() {
//here we upload
getRequestCycle().scheduleRequestHandlerAfterCurrent(new IRequestHandler() {
#Override
public void respond(IRequestCycle irc) {
FileUpload uploadedFile = FiletoUpload.getFileUpload();
HttpServletResponse httpResponse = (HttpServletResponse) irc.getResponse().getContainerResponse();
InputStream CORPUS = null;
try {
CORPUS = uploadedFile.getInputStream();
} catch (IOException ex) {
Logger.getLogger(Upload.class.getName()).log(Level.SEVERE, null, ex);
}
try {
doGet(null, httpResponse);
} catch (IOException ex) {
Logger.getLogger(Upload.class.getName()).log(Level.SEVERE, null, ex);
}
uploadedFile.closeStreams();
}
#Override
public void detach(IRequestCycle irc) {
}
});
}
});
I do not use blobstore, I use com.google.appengine.tools.cloudstorage.*. I open a channel to write on the cloud.
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
resp.setContentType("text/plain");
GcsService gcsservice = GcsServiceFactory.createGcsService();
GcsFilename uploadfile = new GcsFilename(BUCKETNAME, FILENAME);
GcsFileOptions optionsBuilder = new GcsFileOptions.Builder()
.mimeType("text/plain")
.acl("bucket-owner-full-control")
.build();
GcsOutputChannel writechannel = gcsservice.createOrReplace(uploadfile, optionsBuilder );
buffer = new StringBuffer();
int char_read = 0;
long i;
for(i=0; i< CORPUSsize; i++){
char_read = CORPUS.read(); //I read the corpus of the file
char mychar = (char) char_read;
buffer.append(mychar);
}
writechannel.write(ByteBuffer.wrap(buffer.toString().getBytes("UTF-8")));
writechannel.close();
}
Is there a solution to upload a big file without obtaining a permission exception?
Wicket uses a DiskFileItemFactory, which stores files on disk after a 10240 bytes threshold.
You have to change the sizeThreshold or uses an alternative implementation of FileItemFactory. See the MultipartServletWebRequestImpl constructor.
Related
I have a file txt on the server (previously generated). When user clicks on button it generates the file, now I want (additionally) download the file inside my function. But I can't make it work(I'm new on JAVA EE), cause I don't know how to get HttpServletResponse.
From web I call function with this:
#Path("getreport")
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public JSONObject getreport(CommonInput input) {
JSONObject j = objectmapper.conertValue(reportBean.getreport(),JSONObject.class);
return j;
}
reprotBean has function:
public void getreport() {
//...doing many things
//generating my file
List<String> lines = new ArrayList<>();
lines.add("star file");
//..adding many lines
Path file = Paths.get("C:\\Users\\myuser\\file.txt");
Files.write(file, lines, StandardCharsets.UTF_8);
downloadFile();
//...doing many things
}
I found this way to download my file:
public void downloadFile(HttpServletResponse response){
String sourceFile = ""C:\\Users\\myuser\\file.txt"";
try {
FileInputStream inputStream = new FileInputStream(sourceFile);
String disposition = "attachment; fileName=outputfile.txt";
response.setContentType("text/txt");
response.setHeader("Content-Disposition", disposition);
response.setHeader("content-Length", String.valueOf(stream(inputStream, response.getOutputStream())));
} catch (IOException e) {
logger.error("Error occurred while downloading file {}",e);
}
}
private long stream(InputStream input, OutputStream output) throws IOException {
try (ReadableByteChannel inputChannel = Channels.newChannel(input); WritableByteChannel outputChannel = Channels.newChannel(output)) {
ByteBuffer buffer = ByteBuffer.allocate(10240);
long size = 0;
while (inputChannel.read(buffer) != -1) {
buffer.flip();
size += outputChannel.write(buffer);
buffer.clear();
}
return size;
}
}
When I try to use downloadFile(), it requires HttpServletResponse, and I don't have that parameter. I can't understand how to get that (how it works), or do I have to use another method for download my file?
All solutions I found requires HttpServletResponse (download files from browsers)
If you have that file generated already. Just need write it to HttpServletResponse
resp.setContentType("text/plain");
resp.setHeader("Content-disposition", "attachment; filename=sample.txt");
try(InputStream in = req.getServletContext().getResourceAsStream("sample.txt");
OutputStream out = resp.getOutputStream()) {
byte[] buffer = new byte[ARBITARY_SIZE];
int numBytesRead;
while ((numBytesRead = in.read(buffer)) > 0) {
out.write(buffer, 0, numBytesRead);
}
}
Be sure to make your file to be accessed by ServeletContext
If you are using Spring Rest framework. Can refer to below
#GetMapping("/download")
public ResponseEntity<byte[]> downloadErrorData() throws Exception {
List<Employee> employees = employeeService.getEmployees();
ObjectMapper objectMapper = new ObjectMapper();
String json = objectMapper.writeValueAsString(employees);
byte[] isr = json.getBytes();
String fileName = "employees.json";
HttpHeaders respHeaders = new HttpHeaders();
respHeaders.setContentLength(isr.length);
respHeaders.setContentType(new MediaType("text", "json"));
respHeaders.setCacheControl("must-revalidate, post-check=0, pre-check=0");
respHeaders.set(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=" + fileName);
return new ResponseEntity<byte[]>(isr, respHeaders, HttpStatus.OK);
}
credit to: https://www.jeejava.com/file-download-example-using-spring-rest-controller/
I am trying to download 800 MB file from google drive in a streamed fashion. Like I fetch bytes from google drive & write to my response output stream & flush it. Here is the code for it
public void downloadFileAsStream(String accessToken, String fileId
, HttpServletResponse response) throws Exception {
Credential credential = new GoogleCredential().setAccessToken(accessToken);
Drive service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential).build();
File file = null;
try {
file = service.files().get(fileId).setFields("name, size").execute();
} catch (Exception ex) {
logger.error("Exception occurred while getting file from google drive", ex);
throw ex;
}
long fileSize = file.getSize();
OutputStream ros = response.getOutputStream();
for (long i = 0; i<= fileSize; i=i+10000000) {
byte[] fileRangeBytes = getBytes(service, accessToken, fileId, i, directDownloadThreshold);
ros.write(fileRangeBytes);
ros.flush();
}
ros.close();
}
private byte[] getBytes(Drive drive, String accessToken, String fileId, long position, long byteCount) throws Exception {
byte[] receivedByteArray = null;
String downloadUrl = "https://www.googleapis.com/drive/v3/files/" + fileId + "?alt=media&access_token="
+ accessToken;
try {
com.google.api.client.http.HttpRequest httpRequestGet = drive.getRequestFactory().buildGetRequest(new GenericUrl(downloadUrl));
httpRequestGet.getHeaders().setRange("bytes=" + position + "-" + (position + byteCount - 1));
com.google.api.client.http.HttpResponse response = httpRequestGet.execute();
InputStream is = response.getContent();
receivedByteArray = IOUtils.toByteArray(is);
response.disconnect();
} catch (IOException e) {
e.printStackTrace();
throw e;
}
return receivedByteArray;
}
The problem here is that the files are not getting downloaded in the browser immediately in chunks
Rather my application just keeps waiting till the whole file is written to response's outputstream.
So why is the flushing to the browser not happening in my case though I have responseOutputStream.flush() inside the for loop like in this question Java file download hangs
this is my file path
public final static String BOOKINGPDFFILE= "D:/Hotels/pdf/";
This below code is what I have written to download pdf from the above resource folder
Pdf="column name in database i used for storing in database"
#RequestMapping(value = "/getpdf/{pdf}", method = RequestMethod.GET)
public void getPdf(#PathVariable("pdf") String fileName, HttpServletResponse response,HttpServletRequest request) throws IOException {
try {
File file = new File(FileConstant.BOOKINGPDFFILE + fileName+ ".pdf");
Files.copy(file.toPath(),response.getOutputStream());
} catch (IOException ex) {
System.out.println("Contract Not Found");
System.out.println(ex.getMessage());
}
}
Here is the way, hope it help.
#RequestMapping(value = "/getpdf/{pdf}", method = RequestMethod.GET)
public void getPdf(#PathVariable("pdf") String fileName, HttpServletResponse response) throws IOException {
try {
File file = new File(FileConstant.BOOKINGPDFFILE + fileName+ ".pdf");
if (file.exists()) {
// here I use Commons IO API to copy this file to the response output stream, I don't know which API you use.
FileUtils.copyFile(file, response.getOutputStream());
// here we define the content of this file to tell the browser how to handle it
response.setContentType("application/pdf");
response.setHeader("Content-disposition", "attachment;filename=" + fileName + ".pdf");
response.flushBuffer();
} else {
System.out.println("Contract Not Found");
}
} catch (IOException exception) {
System.out.println("Contract Not Found");
System.out.println(exception.getMessage());
}
}
You may try something like this:
#RequestMapping(method = { RequestMethod.GET }, value = { "/downloadPdf" })
public ResponseEntity<InputStreamResource> downloadPdf()
{
try
{
File file = new File(BOOKINGPDFFILE);
HttpHeaders respHeaders = new HttpHeaders();
MediaType mediaType = MediaType.parseMediaType("application/pdf");
respHeaders.setContentType(mediaType);
respHeaders.setContentLength(file.length());
respHeaders.setContentDispositionFormData("attachment", file.getName());
InputStreamResource isr = new InputStreamResource(new FileInputStream(file));
return new ResponseEntity<InputStreamResource>(isr, respHeaders, HttpStatus.OK);
}
catch (Exception e)
{
String message = "Errore nel download del file "+idForm+".csv; "+e.getMessage();
logger.error(message, e);
return new ResponseEntity<InputStreamResource>(HttpStatus.INTERNAL_SERVER_ERROR);
}
}
And in your web page you can write the link in this way:
download PDF
You need to create an implementation of AbstractPdfView to achieve this.. You can refer this link https://www.mkyong.com/spring-mvc/spring-mvc-export-data-to-pdf-file-via-abstractpdfview/
Here is the Detailed answer for your question.
let me start with the server side code:
Below class is used to create pdf with some random content and return the equivalent byte array outputstream.
public class pdfgen extends AbstractPdfView{
private static ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
public ByteArrayOutputStream showHelp() throws Exception {
Document document = new Document();
// System.IO.MemoryStream ms = new System.IO.MemoryStream();
PdfWriter.getInstance(document,byteArrayOutputStream);
document.open();
document.add(new Paragraph("table"));
document.add(new Paragraph(new Date().toString()));
PdfPTable table=new PdfPTable(2);
PdfPCell cell = new PdfPCell (new Paragraph ("table"));
cell.setColspan (2);
cell.setHorizontalAlignment (Element.ALIGN_CENTER);
cell.setPadding (10.0f);
//cell.setBackgroundColor (new BaseColor (140, 221, 8));
table.addCell(cell);
ArrayList<String[]> row=new ArrayList<String[]>();
String[] data=new String[2];
data[0]="1";
data[1]="2";
String[] data1=new String[2];
data1[0]="3";
data1[1]="4";
row.add(data);
row.add(data1);
for(int i=0;i<row.size();i++) {
String[] cols=row.get(i);
for(int j=0;j<cols.length;j++){
table.addCell(cols[j]);
}
}
document.add(table);
document.close();
return byteArrayOutputStream;
}
}
Then comes the controller code : here the bytearrayoutputstream is converted to bytearray and sent to the client side using the response-entity with appropriate headers.
#RequestMapping(path="/home")
public ResponseEntity<byte[]> render(HttpServletRequest request , HttpServletResponse response) throws IOException
{
pdfgen pg=new pdfgen();
response.setContentType("application/pdf");
response.setHeader("Content-Disposition", "attachment:filename=report.pdf");
byte[] contents = null;
try {
contents = pg.showHelp().toByteArray();
}
catch (Exception e) {
e.printStackTrace();
}
//These 3 lines are used to write the byte array to pdf file
/*FileOutputStream fos = new FileOutputStream("/Users/naveen-pt2724/desktop/nama.pdf");
fos.write(contents);
fos.close();*/
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.parseMediaType("application/pdf"));
//Here you have to set the actual filename of your pdf
String filename = "output.pdf";
headers.setContentDispositionFormData(filename, filename);
headers.setCacheControl("must-revalidate, post-check=0, pre-check=0");
ResponseEntity<byte[]> respons = new ResponseEntity<byte[]>(contents, headers, HttpStatus.OK);
return respons;
}
The header should be set to "application/pdf"
Then comes the client side code :
Where you can make ajax request to server to open the pdf file in new tab of the browser
$.ajax({
url:'/PDFgen/home',
method:'POST',
cache:false,
xhrFields: {
responseType: 'blob'
},
success: function(data) {
//alert(data);
let blob = new Blob([data], {type: 'application/pdf'}); //mime type is important here
let link = document.createElement('a'); //create hidden a tag element
let objectURL = window.URL.createObjectURL(blob); //obtain the url for the pdf file
link.href = objectURL; // setting the href property for a tag
link.target = '_blank'; //opens the pdf file in new tab
link.download = "fileName.pdf"; //makes the pdf file download
(document.body || document.documentElement).appendChild(link); //to work in firefox
link.click(); //imitating the click event for opening in new tab
},
error:function(xhr,stats,error){
alert(error);
}
});
Try this
#Controller
#RequestMapping("/download")
public class FileDownloadController
{
#RequestMapping("/pdf/{fileName}")
public void downloadPDFResource( HttpServletRequest request,
HttpServletResponse response,
#PathVariable("fileName") String fileName)
{
//If user is not authorized - he should be thrown out from here itself
//Authorized user will download the file
String dataDirectory = request.getServletContext().getRealPath("/WEB-INF/downloads/pdf/");
Path file = Paths.get(dataDirectory, fileName);
if (Files.exists(file))
{
response.setContentType("application/pdf");
response.addHeader("Content-Disposition", "attachment; filename="+fileName);
try
{
Files.copy(file, response.getOutputStream());
response.getOutputStream().flush();
}
catch (IOException ex) {
ex.printStackTrace();
}
}
}
}
In my servlet, I upload a file to a specific url, then call a method in a different class to grab that file online and alter it. The problem is that a lot of the time, there is a lot of data in the file and the dopost method finishes (and goes to the next page from the upload file submit form) before the file is fully altered. How do I prevent the do post method from going to the next page before my file is completely altered?
I want the Test.preFirstMethod() to finish before the page reloads and makes the download link. Test.preFirstMethod() takes in an excel file and modifies it using dynamic content from the internet.
public class Uploads extends HttpServlet {
private Object lock1 = new Object();
private static final long serialVersionUID = 1L;
private ServletFileUpload uploader = null;
public Thread tt = new Thread(new Runnable() {
public void run()
{
try {Test.preFirstMethod();
} catch (Exception e) {
e.printStackTrace();
}
}
});
int BUFFER_LENGTH = 4096;
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
PrintWriter out = response.getWriter();
String fileName="";
for (Part part : request.getParts()) {
InputStream is = request.getPart(part.getName()).getInputStream();
fileName = getFileName(part);
File f2 = new File(fileName);
FileOutputStream os = new FileOutputStream(System.getenv("OPENSHIFT_DATA_DIR") + "nn.xlsx");
byte[] bytes = new byte[BUFFER_LENGTH];
int read = 0;
while ((read = is.read(bytes, 0, BUFFER_LENGTH)) != -1) {
os.write(bytes, 0, read);
}
os.flush();
is.close();
fileName=fileName.substring(0,fileName.lastIndexOf("."))+"_ZillowAdded.xlsx";
try {
tt.start();
try { tt.join(); } catch (InterruptedException e) {
e.printStackTrace();
}
InputStream is2 = new FileInputStream(f2);
FileOutputStream os2 = new FileOutputStream(System.getenv("OPENSHIFT_DATA_DIR") + fileName);
while ((read = is2.read(bytes, 0, BUFFER_LENGTH)) != -1) {
os2.write(bytes, 0, read);
}
os2.flush();
is2.close();
os2.close();
} catch(Exception e) {e.printStackTrace();}
os.close();
}
if(!ServletFileUpload.isMultipartContent(request)){
throw new ServletException("Content type is not multipart/form-data");
}
response.setContentType("text/html");
out.write("<html><head></head><body>");
out.write("File "+fileName.substring(0,fileName.indexOf("_Zillow"))+ " uploaded successfully.");
out.write("<br>");
out.write("Download "+fileName+"");
out.write("</body></html>"); }
I am writing a small file upload utility thing as part of a larger project. Originally I was handling this from a servlet using the Apache commons File utility classes. Here is a snippet from a quick test client I wrote for the service:
public static void main(String[] args) {
JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
factory.getInInterceptors().add(new LoggingInInterceptor());
factory.getOutInterceptors().add(new LoggingOutInterceptor());
factory.setServiceClass(FileUploadService.class);
factory.setAddress("http://localhost:8080/FileUploadService/FileUploadService");
FileUploadService client = (FileUploadService) factory.create();
FileType file = new FileType();
file.setName("statemo_1256144312279");
file.setType("xls");
DataSource source = new FileDataSource(new File("c:/development/statemo_1256144312279.xls"));
file.setHandler(new DataHandler(source));
Boolean ret = client.uploadFile(file);
System.out.println (ret);
System.exit(0);
}
This works absolutely fine. Now the problem comes when I am trying to replace the Apache commons utilities. In the above code I am creating a DataSource from a File with an absolute path name. In my servlet, I can't get an absolute path name however and the file I am sending over the wire is empty.
Here is the servlet code:
#SuppressWarnings("unchecked")
protected void doPost (final HttpServletRequest request, final HttpServletResponse response)
throws ServletException, IOException {
// form should have enctype="multipart/form-data" as an attribute
if (!ServletFileUpload.isMultipartContent (request)) {
LOG.info("Invalid form attribute");
return;
}
//DataInputStream in = new DataInputStream(request.getInputStream());
final DiskFileItemFactory factory = new DiskFileItemFactory ();
factory.setSizeThreshold(FILE_THRESHOLD_SIZE);
final ServletFileUpload sfu = new ServletFileUpload (factory);
sfu.setSizeMax(MAX_FILE_SIZE);
final HttpSession session = request.getSession();
final List<FileItem> files = new ArrayList<FileItem>();
final List<String> filesToProcess = new ArrayList<String>();
try {
final List<FileItem> items = sfu.parseRequest(request);
for (final FileItem f : items) {
if (!f.isFormField())
files.add(f);
}
/*for (final FileItem f : files) {
final String absoluteFileName = UPLOAD_DESTINATION + FilenameUtils.getName(f.getName());
//f.write(new File (absoluteFileName));
filesToProcess.add(absoluteFileName);
}*/
FileItem f = files.get(0);
LOG.info("File: " + FilenameUtils.getName(f.getName()));
LOG.info("FileBaseName: " + FilenameUtils.getBaseName(f.getName()));
LOG.info("FileExtension: " + FilenameUtils.getExtension(f.getName()));
FileUploadServiceClient client = new FileUploadServiceClient();
DataSource source = new FileDataSource(new File(f.getName()));
FileType file = new FileType();
file.setHandler(new DataHandler(source));
file.setName(FilenameUtils.getBaseName(f.getName()));
file.setType(FilenameUtils.getExtension(f.getName()));
Boolean ret = client.uploadFile(file);
LOG.info("File uploaded - " + ret);
filesToProcess.add(UPLOAD_DESTINATION + FilenameUtils.getName(f.getName()));
session.setAttribute("filesToProcess", filesToProcess);
final RequestDispatcher dispatcher = request.getRequestDispatcher("Validate");
if (null != dispatcher) {
dispatcher.forward(request, response);
}
} catch (FileUploadException e) {
LOG.info("Exception " + e.getMessage());
e.printStackTrace();
} catch (Exception e) {
LOG.info("Exception " + e.getMessage());
e.printStackTrace();
}
}
I've been working on this for the better part of this morning and am not getting anywhere. Even if I get rid of the Apache commons file stuff completely and handle the parsing of the request myself, I still can't construct the DataSource appropriately.
Thanks!
This was rather simple actually, I just copied over the bytes from the InputStream to the DataSource:
FileItem f = files.get(0);
// there is a problem here where the file being created is empty, since we only have a
// partial path:
DataSource source = new FileDataSource(new File(f.getName()));
// because of the above problem, we are going to copy over the data ourselves:
byte[] sourceBytes = f.get();
OutputStream sourceOS = source.getOutputStream();
sourceOS.write(sourceBytes);
This is the code of commons-email ByteArrayDataSource
it sounds odd to try to replace apache commons - don't, unless you have a really good reason
you can get absolute paths in a servlet. You can call getServletContext().getRealPath("/") which will return the absolute path of your application, and then you can get files relative to it.
In our application there are objects that have properties InputStream and Name. We are using next class to construct DataSource with those properties.
public class InputStreamDataSource implements DataSource {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
private final String name;
public InputStreamDataSource(InputStream inputStream, String name) {
this.name = name;
try {
int nRead;
byte[] data = new byte[16384];
while ((nRead = inputStream.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
inputStream.close();
buffer.flush();
} catch (IOException e) {
e.printStackTrace();
}
}
#Override
public String getContentType() {
return new MimetypesFileTypeMap().getContentType(name);
}
#Override
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(buffer.toByteArray());
}
#Override
public String getName() {
return name;
}
#Override
public OutputStream getOutputStream() throws IOException {
throw new IOException("Read-only data");
}
}
Most of the solutions shown here require that the InpustStream be closed (read into memory). It is possible to wrap the InputStream in a DataSource object without closing the InputStream though:
private record PipedDataSource(InputStream in, String contentType, String encoding)
implements DataSource, EncodingAware {
public String getContentType() {
return contentType;
}
public InputStream getInputStream() {
return in;
}
public String getName() {
return "PipedDataSource";
}
public OutputStream getOutputStream() throws IOException {
throw new IOException("No OutputStream");
}
#Override
public String getEncoding() {
return encoding;
}
}
The example above also implements EncodingAware. This can prevent the InputStream from being closed by third part libraries (for example java.mail.internet.MimeUtility) when they get the data source encoding.