I'm currently working on a project that is done in Java, on google appengine. i have above 2000 records
Appengine does not allow files to be stored so any on-disk representation objects cannot be used. Some of these include the File class.
I want to write data and export it to a few csv files, the user to download it.
How may I do this without using any File classes? I'm not very experienced in file handling so I hope you guys can advise me.
Thanks.
Just generate the csv in memory using a StringBuffer and then use StringBuffer.toString().getBytes() to get a byte array which can then be sent to your output stream.
For instance if using a servlet in GAE:
protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
StringBuffer buffer = new StringBuffer();
buffer.append("header1, header2, header3\n");
buffer.append("row1column1, row1column2, row1column3\n");
buffer.append("row2column1, row2column2, row2column3\n");
// Add more CSV data to the buffer
byte[] bytes = buffer.toString().getBytes();
// This will suggest a filename for the browser to use
resp.addHeader("Content-Disposition", "attachment; filename=\"myFile.csv\"");
resp.getOutputStream().write(bytes, 0, bytes.length);
}
More information about GAE Servlets
More information about Content-Disposition
You can store data in memory using byte arrays, stings and streams. For example,
ByteArrayOutputStream csv = new ByteArrayOutputStream();
PrintStream printer = new PrintStream(csv);
printer.println("a;b;c");
printer.println("1;2;3");
printer.close();
csv.close();
Then in your Servlet you can serve your csv.toByteArray() as a stream. Some example is given here: Implementing a simple file download servlet.
You can use OpenCSV library in Google App Engine.
Related
For example, i would like to download one zip file and one csv file in one response. Is there any way other than compressing these two files in one zip file.
Although ServletResponse is not meant to do this, we could programmatically tweak it to send multiple files, which all client browsers except IE seems to handle properly. A sample code snippet is given below.
response.setContentType("multipart/x-mixed-replace;boundary=END");
ServletOutputStream out = response.getOutputStream();
out.println("--END");
for(File f:files){
FileInputStream fis = new FileInputStream(file);
BufferedInputStream fif = new BufferedInputStream(fis);
int data = 0;
out.println("--END");
while ((data = fif.read()) != -1) {
out.write(data);
}
fif.close();
out.println("--END");
out.flush();
}
out.flush();
out.println("--END--");
out.close();
This will not work in IE browsers.
N.B - Try Catch blocks not included
Code developed by Jason Hunter to handle servlet request and response having multiple parts has been the defacto since years. You can find it at servlets.com
No you can not do that. The reason is that whenever you want to sent any data in request you use steam available in request and retrive this data using request.getRequestParameter("streamParamName").getInputStream(), also please make a note if you have already consumed this stream once you will not be able to get it again.
The example mentioned above is a tweak that google also uses in sending multipart email with multiple attachments. To achieve that they define boundaries for each attachment and client have to take care of these boundaries while retrieving this information and rendering it.
I've developed a Web Service in Java, and my client will upload an .ini file to the service using the API which definition is something like
public Response PostStreamInfo(#Context HttpServletRequest request,
#PathParam("UniqueId") String uniqueId, InputStream inputStream)
{
/* my code here */
}
Now I want to convert the inputStream back into .ini format, and retreive some value. However, all methods I found to get .ini content is read from real file, none of them talk about how to transfer from inputstream.
Is my goal possible? Because I don't want to Store the .ini file, Read it, and Delete it only for 1 value I need, I think that's a waste of time. Or should I just use Json format instead of InputStream to transfer the content? Thanks!
Are you dealing with actual INI files or Java property files?
It looks like there's an Apache class that you could use via an InputStreamReader and the read method for INI files.
If you're just dealing with Java property files, there are read methods that take InputStreams and also Readers.
I'm using Blobstore to upload a simple text file using this doc: https://cloud.google.com/appengine/docs/java/blobstore/#Java_Uploading_a_blob . I understand from the docs how to save and serve the blob to users, but I don't understand how can my servlet that handle the file upload actually read the contents of the text file?
I found the answers. This is the code:
Map<String, List<FileInfo>> infos = blobstoreService.getFileInfos(request);
Long fileSize = infos.get("myFile").get(0).getSize();
Map<String, List<BlobKey>> blobKeys = blobstoreService.getUploads(request);
byte[] fileBytes =
blobstoreService.fetchData(blobKeys.get("myFile").get(0), 0, fileSize);
String input = new String(fileBytes);
In python there is the BlobReader class to help you do this. (https://cloud.google.com/appengine/docs/python/blobstore/blobreaderclass)
It seems like you are using Java though. There does not seem to be an equivalent class in Java. What I would do is to use GCS as the backing for you blobstore (https://cloud.google.com/appengine/docs/java/blobstore/#Java_Using_the_Blobstore_API_with_Google_Cloud_Storage). This way the files uploaded to the blobstore will be accessibly in GCS.
You can then read the file using the GCS client library for Java.
For example, i would like to download one zip file and one csv file in one response. Is there any way other than compressing these two files in one zip file.
Although ServletResponse is not meant to do this, we could programmatically tweak it to send multiple files, which all client browsers except IE seems to handle properly. A sample code snippet is given below.
response.setContentType("multipart/x-mixed-replace;boundary=END");
ServletOutputStream out = response.getOutputStream();
out.println("--END");
for(File f:files){
FileInputStream fis = new FileInputStream(file);
BufferedInputStream fif = new BufferedInputStream(fis);
int data = 0;
out.println("--END");
while ((data = fif.read()) != -1) {
out.write(data);
}
fif.close();
out.println("--END");
out.flush();
}
out.flush();
out.println("--END--");
out.close();
This will not work in IE browsers.
N.B - Try Catch blocks not included
Code developed by Jason Hunter to handle servlet request and response having multiple parts has been the defacto since years. You can find it at servlets.com
No you can not do that. The reason is that whenever you want to sent any data in request you use steam available in request and retrive this data using request.getRequestParameter("streamParamName").getInputStream(), also please make a note if you have already consumed this stream once you will not be able to get it again.
The example mentioned above is a tweak that google also uses in sending multipart email with multiple attachments. To achieve that they define boundaries for each attachment and client have to take care of these boundaries while retrieving this information and rendering it.
I need to provide a feature where user can download reports in excel/csv format in my web application. Once i made a module in web application which creates excel and then read it and sent to browser. It was working correctly. This time i don't want to generate excel file, as i don't have that level of control over file systems. I guess one way is to generate appropriate code in StringBuffer and set correct contenttype(I am not sure about this approach). Other team also has this feature but they are struggling when data is very large. What is the best way to provide this feature considering size of data could be very huge. Is it possible to send data in chunk without client noticing(except delay in downloading).
One issue i forgot to add is when there is very large data, it also creates problem in server side (cpu utilization and memory consumption). Is it possible that i read fixed amount of records like 500, send it to client, then read another 500 till completed.
You can also generate HTML instead of CSV and still set the content type to Excel. This is nice for colouring and styled text.
You can also use gzip compression when the client accepts that compression. Normally there are standard means, like a servlet filter.
Never a StringBuffer or the better StringBuilder. Better streaming it out. If you do not (cannot) call setContentength, the output goes chunked (without predictive progress).
URL url = new URL("http://localhost:8080/Works/images/address.csv");
response.setHeader("Content-Type", "text/csv");
response.setHeader("Content-disposition", "attachment;filename=myFile.csv");
URLConnection connection = url.openConnection();
InputStream stream = connection.getInputStream();
BufferedOutputStream outs = new BufferedOutputStream(response.getOutputStream());
int len;
byte[] buf = new byte[1024];
while ((len = stream.read(buf)) > 0) {
outs.write(buf, 0, len);
}
outs.close();