I have been trying to stream|transfer a huge file available in the local file-system over restapi using streamingoutput. I keep running into heapSpace error. Can anyone help me figure out what I am doing wrong? As per my understanding, streamingoutput shouldn't keep the file in memory.
Please find the code below:
public Response getBulkBillDownload(#QueryParam("requestID") String requestID,
#QueryParam("zipFileName") String zipFileName) throws RestException {
StreamingOutput stream = null;
try {
File file = null;
Optional<File> document = getCorporatePaymentManager().getBulkBillDownloadResponse(requestID, zipFileName);
if (document.isPresent()) {
file = document.get();
} else {
throw new RestException("File not found");
}
final FileInputStream fStream = new FileInputStream(file);
// register stream to Response and it will callback with server OutputStream
stream = new StreamingOutput() {
#Override
public void write(OutputStream output) throws IOException, WebApplicationException {
pipe(fStream, output);
}
};
} catch (Exception e) {
handleException(e);
}
return Response.status(200).entity(stream).header("Content-Disposition", "attachment; filename=" + zipFileName)
.build();
}
private void pipe(InputStream is, OutputStream os) throws IOException {
byte[] buf=new byte[1024];
int bytesread = 0, bytesBuffered = 0;
while( (bytesread = is.read( buf )) > -1 ) {
os.write( buf, 0, bytesread );
bytesBuffered += bytesread;
if (bytesBuffered > 1024 * 1024) { //flush after 1MB
bytesBuffered = 0;
os.flush();
}
}
os.close();
}
Related
I've taken over a project where the file upload functionality is broken. Currently when a file is uploaded it is converted to a byteArray like so and then stored in a SQL table
public static byte[] saveAttachment(String filePath) throws IOException{
InputStream inputStream = new FileInputStream(filePath);
byte[] buffer = new byte[1048576];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
while((bytesRead = inputStream.read(buffer)) != -1){
output.write(buffer, 0 , bytesRead);
}
inputStream.close();
return output.toByteArray();
}
I can't say I agree with the approach that has been taken but alas I must work with it. My question becomes how do I go about retrieving this file to display?
I have read
https://wiki.apache.org/tapestry/Tapestry5HowToStreamAnExistingBinaryFile
And tried (which didn't work)
#OnEvent(component="viewAttachment")
private Object viewAttachment(){
final File getFile();
final OutputStreamResponse response = new OutputStreamResponse() {
public String getContentType() {
return "image/jpg";
}
public void prepareResponse(Response response) {
response.setHeader ("Content-Disposition", "attachment; filename=\"" + file.getName() + "\"");
}
#Override
public void writeToStream(OutputStream out) throws IOException {
try {
InputStream in = new FileInputStream(file);
IOUtils.copy(in,out);
in.close();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
};
return response;
}
However I am not sure this is the proper/optimal solution.
Assuming that row.getBytes() returns your image as byte array, and row.getName() is image name:
return new StreamResponse() {
#Override
public String getContentType() {
return "image/jpeg";
}
#Override
public InputStream getStream() throws IOException {
return new ByteArrayInputStream(row.getBytes());
}
#Override
public void prepareResponse(Response response) {
response.setHeader("Content-Disposition", "attachment; filename=\"" + row.getName() + "\"");
}
};
Its better to save the file in a location and save the location in the database. This will help to have size of database.
Also file is available and can retrieve easily without heavy database object.
Or you can add the BLOB column in database and store the file in database.
Convert file into File object
File image = new File("D:\\a.gif");
FileInputStream fis = new FileInputStream(image);
stmt.setBinaryStream(1, fis, (int) image.length());
Add retrieve it using
File image = new File("D:\\java.gif");
FileOutputStream fos = new FileOutputStream(image);
byte[] buffer = new byte[1];
InputStream is = resultSet.getBinaryStream(3);
while (is.read(buffer) > 0) {
fos.write(buffer);
}
fos.close();
String fileName = "/CSVLogs/test";
String fileType = "csv";
resp.setContentType(fileType);
resp.setHeader("Content-disposition","attachment; filename=test.csv");
File my_file = new File(fileName);
OutputStream out = resp.getOutputStream();
FileInputStream in = new FileInputStream(my_file);
byte[] buffer = new byte[4096];
int length;
while ((length = in.read(buffer)) > 0){
out.write(buffer, 0, length);
}
in.close();
out.flush();
I need to download a csv file but it seems to return "java.lang.IllegalStateException: WRITER"
<form enctype="multipart/form-data" action="/TestServlet/ConfigServlet?do=downloadLogs" method="post" style="height:68px;">
UPDATE
resp.setContentType("application/octet-stream");
try
{
OutputStream outputStream = resp.getOutputStream();
InputStream in = StorageUtil.getInstance().getFile("/CSVLogs/test.csv").getInputStream();
/* InputStream in = StorageUtil.getInstance().getCSVLogsZip().getInputStream();*/
byte[] buffer = new byte[4096];
int length;
while ((length = in.read(buffer)) > 0){
outputStream.write(buffer, 0, length);
in.close();
outputStream.flush();
}
}
catch(Exception e) {
System.out.println(e.toString());
}
I still get the same error.
java.lang.IllegalStateException: WRITER
(drunk) Why am I getting this error >_<
Try this:
public void doGet(HttpServletRequest request, HttpServletResponse response)
{
response.setContentType("text/csv");
response.setHeader("Content-Disposition", "attachment; filename=\"test.csv\"");
try
{
OutputStream outputStream = response.getOutputStream();
FileInputStream in = new FileInputStream(my_file);
byte[] buffer = new byte[4096];
int length;
while ((length = in.read(buffer)) > 0){
outputStream.write(buffer, 0, length);
in.close();
outputStream.flush();
}
}
catch(Exception e)
{
model.closeConnection();
System.out.println(e.toString());
}
}
public void downloadFile(HttpServletResponse response){
String sourceFile = "c:\\source.csv";
try {
FileInputStream inputStream = new FileInputStream(sourceFile);
String disposition = "attachment; fileName=outputfile.csv";
response.setContentType("text/csv");
response.setHeader("Content-Disposition", disposition);
response.setHeader("content-Length", String.valueOf(stream(inputStream, response.getOutputStream())));
} catch (IOException e) {
logger.error("Error occurred while downloading file {}",e);
}
}
And the stream method should be like this.
private long stream(InputStream input, OutputStream output) throws IOException {
try (ReadableByteChannel inputChannel = Channels.newChannel(input); WritableByteChannel outputChannel = Channels.newChannel(output)) {
ByteBuffer buffer = ByteBuffer.allocate(10240);
long size = 0;
while (inputChannel.read(buffer) != -1) {
buffer.flip();
size += outputChannel.write(buffer);
buffer.clear();
}
return size;
}
}
java/servlet code you supplied works perfectly fine.
i call the servlet CSVD as below:
< form enctype="multipart/form-data" action="CSVD" method="post" style="height:68px;">
<input type="submit" value="submit" />
< /form>
or through anchor this way < a href="/CSVDownloadApp/CSVD">click here to download csv< /a>
possibly your error is coming for a different reason.
Try this:
response.setContentType("application/x-rar-compressed");
response.setHeader("Content-Disposition", "attachment; filename=\"test.csv\"");
For writing the file in OutputStream, try following
FileInputStream fis = new FileInputStream("your_csv_file.csv");
byte[] b = new byte[fis.available()];
outputStream.write(b);
outputStream.flush();
outputStream.close();
I want to write simple rest api for file download.
I cant find docs about it as I understood I need to set mimetype='application/zip' for response, but not clear how to return stream.
http://sparkjava.com/
update:
resolved here example code:
public static void main(String[] args) {
//setPort(8080);
get("/hello", (request, responce) -> getFile(request,responce));
}
private static Object getFile(Request request, Response responce) {
File file = new File("MYFILE");
responce.raw().setContentType("application/octet-stream");
responce.raw().setHeader("Content-Disposition","attachment; filename="+file.getName()+".zip");
try {
try(ZipOutputStream zipOutputStream = new ZipOutputStream(new BufferedOutputStream(responce.raw().getOutputStream()));
BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(file)))
{
ZipEntry zipEntry = new ZipEntry(file.getName());
zipOutputStream.putNextEntry(zipEntry);
byte[] buffer = new byte[1024];
int len;
while ((len = bufferedInputStream.read(buffer)) > 0) {
zipOutputStream.write(buffer,0,len);
}
}
} catch (Exception e) {
halt(405,"server error");
}
return null;
What you need is similar to this thread. You only need to close the OutputStream and return the raw HTTPServletResponse:
try {
...
zipOutputStream.flush();
zipOutputStream.close();
} catch (Exception e) {
halt(405,"server error");
}
return responce.raw();
I am using the code in this link http://www.mkyong.com/webservices/jax-rs/file-upload-example-in-jersey/ to upload a file.In this example I have to pass from a html page to specify the file to upload but I want to to acceed to it when I call the webservice by its path ( s.thing like that : http://*****:8080/RESTfulExample/file/upload/C://image.png)
Are there any suggestions to this issue? Please help!
That is what i did till now to solve it
#Path(value="/files")
public class upload {
#POST
#Path(value = "upload/{path}")
#Consumes("image/jpg")
public Response uploadPng(#PathParam("path") String path, File file) throws IOException {
file = new File("path");
String uploadedFileLocation = "C:/Users/Desktop/" + file.getName();
DataInputStream diStream =new DataInputStream(new FileInputStream(file));
long len = (int) file.length();
byte[] fileBytes = new byte[(int) len];
int read = 0;
int numRead = 0;
while (read < fileBytes.length && (numRead =
diStream.read(fileBytes, read,fileBytes.length - read)) >= 0) {
read = read + numRead;
}
writeToFile(diStream, uploadedFileLocation);
System.out.println("File uploaded to : " + uploadedFileLocation);
return Response.status(200).entity(file).build();
}
private void writeToFile(InputStream uploadedInputStream,
String uploadedFileLocation) {
try {
OutputStream out =new FileOutputStream(new File(uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[1024];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}}}
But I have an 405 error now !!
EDIT
#Path(value= "/up")
public class upload {
private static final String SERVER_UPLOAD_LOCATION_FOLDER = "C://Users/Marwa/Desktop/mafile.png";
#POST
#Path(value="upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public String uploadFile(#FormDataParam("file") InputStream fileInputStream) {
String filePath = SERVER_UPLOAD_LOCATION_FOLDER ;
System.out.println("*****serverpath********");
saveFile(fileInputStream, filePath);
String output = "File saved to server location : " + filePath;
return output;
}
private void saveFile(InputStream uploadedInputStream,String serverLocation) {
try {
OutputStream outpuStream = new FileOutputStream(new File(serverLocation));
int read = 0;
byte[] bytes = new byte[1024];
outpuStream = new FileOutputStream(new File(serverLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
outpuStream.write(bytes, 0, read);}
outpuStream.flush();
outpuStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
I think you'd just call http://example.com/file/upload and post the file there with the browser (with JavaScript) or some other client. For example, you could test it with curl
curl -i -F "file=#/home/user1/Desktop/test.jpg" http://example.com/file/upload
Do you need the file path for something on the server side? If you need the path on the server side for some reason, you could just add a #PathParam.
#POST
#Path("/upload/{path}")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(
#PathParam("path") String path,
#FormDataParam("file") InputStream uploadedInputStream,
#FormDataParam("file") FormDataContentDisposition fileDetail) {
...
}
You could also try leaving off #consumes or using a specific type like #consumes("image/jpg"). For example:
#POST
#Path("/upload/{path}")
#Consumes("image/jpg")
public Response uploadFile(
#PathParam("path") String path,
InputStream uploadedInputStream) {
...
}
I tried to upload a stream (restlet memoryfile) to gcs. But the file has another filesize and is a little different so that the file is marked as "broken".
I tried is local and on google app engine. While debugging to this part the stream looks good in size InputStream inputStream = item.getInputStream();
But the result in the store isn't that size. There are 4 Bits at the beginning: ’[NUL][ENQ]
Where are they from?
List<FileItem> items;
try {
MemoryFileItemFactory factory = new MemoryFileItemFactory();
RestletFileUpload restletFileUpload = new RestletFileUpload(factory);
items = restletFileUpload.parseRequest(req);
//items = restletFileUpload.parseRepresentation(entity);
for (FileItem item : items) {
if (!item.isFormField()) {
MediaType type = MediaType.valueOf(item.getContentType());
GcsFileOptions options = new GcsFileOptions.Builder().mimeType(type.getName()).acl("public-read").build();
GcsOutputChannel outputChannel = gcsService.createOrReplace(fileName, options);
ObjectOutputStream oout = new ObjectOutputStream(Channels.newOutputStream(outputChannel));
InputStream inputStream = item.getInputStream();
copy(inputStream, oout);
//oout.close();
}
}
} catch (FileUploadException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
private void copy(InputStream input, OutputStream output) throws IOException {
try {
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = input.read(buffer);
while (bytesRead != -1) {
output.write(buffer, 0, bytesRead);
bytesRead = input.read(buffer);
}
} finally {
input.close();
output.close();
}
}
private final GcsService gcsService = GcsServiceFactory.createGcsService(new RetryParams.Builder()
.initialRetryDelayMillis(10)
.retryMaxAttempts(10)
.totalRetryPeriodMillis(15000)
.build());
Remove the finally close statements from the copy-function and close the GcsOutputChannel instead. Further you don't need to do this: ObjectOutputStream oout = new ObjectOutputStream(Channels.newOutputStream(outputChannel));
Maybe that adds the extra-bits
Something like that:
GcsOutputChannel outputChannel = gcsService.createOrReplace(fileName, options);
InputStream inputStream = item.getInputStream();
try {
copy(inputStream, Channels.newOutputStream(outputChannel));
} finally {
outputChannel.close();
inputStream.close();
}
private void copy(InputStream input, OutputStream output) throws IOException {
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = input.read(buffer);
while (bytesRead != -1) {
output.write(buffer, 0, bytesRead);
bytesRead = input.read(buffer);
}
}