Hello Stack Overflow community,
I am doing multistep processing on some data I am receiving with a java Servlet. The current process I have is that I input the files to a server using Apache File Upload and convert them to a File. Then once input1 is populated with data, I run through a flow similar to this (where the process functions are xsl transforms):
File input1 = new File(FILE_NAME); // <---this is populated with data
File output1 = new File(TEMP_FILE); // <---this is the temporary file
InputStream read = new FileInputStream(input1);
OuputStream out = new FileOutputStream(output1);
process1ThatReadsProcessesOutputs( read, out);
out.close();
read.close();
//this is basically a repeat of the above process!
File output2 = new File(RESULT_FILE); // <--- This is the result file
InputStream read1 = new FileInputStream(output1);
OutputStream out1 = new FileOutputStream(output2);
Process2ThatReadsProcessesOutputs( read1, out1);
read1.close();
out1.close();
…
So my question is if there is a better way to do this so I do not have to create those temporary Files and recreate streams to those Files? (I am assuming I am incurring a decent performace penatly)
I saw this Most Efficient Way to create InputStream from OutputStream but I am not sure if this is the best route to go...
Just replace FileOutputStream to ByteArrayInputStream vice/versa.
Example:
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayInputStream in = new ByteArrayInputStream(out.toByteArray());
I don't know why are you converting the FileItem retrieved with Apache Commons if you don't really needed. You can use the same InputStream that each FileItem has to using and read the content of the uploaded file:
// create/retrieve a new file upload handler
ServletFileUpload upload = ...;
// parse the request
List<FileItem> items = (List<FileItem>) upload.parseRequest(request);
/* get the FileItem from the List. Yes, it's not a best practice because you must verify
how many you receive, and check everything is ok, etc.
Let's suppose you've done it */
//...
FileItem item = items.get(0);
// get the InputStrem to read the contents of the file
InputStream is = item.getInputStream();
So finally, you can use the InputStream object to read the uploaded stream sent by the client avoiding unnecessary instantiations.
And yes, it's really recommended to use Buffered clases like BufferedInputStream and BufferedOutputStream.
The other idea could be to avoid FileOutputStream (the middle one) and replace it with ByteArrayOutputStream if you don't need to be written in disk (always is slower than working in memory).
Java 9 brings a new answer to the question:
// All bytes from an InputStream at once
byte[] result = new ByteArrayInputStream(buf)
.readAllBytes();
// Directly redirect an InputStream to an OutputStream
new ByteArrayInputStream(buf)
.transferTo(System.out);
Related
I am writing a csv file in a very old java application so i can not use all the new Java 8 streams.
Writer writer = new OutputStreamWriter(new FileOutputStream("file.csv"));
writer.append("data,");
writer.append("data,");
...
Then I need to transform the writer object into a ByteArrayInputStream.
How can i do it ?
Thanks in advance.
Best regards.
This depends on what you are trying to do.
If you are writing a bunch of data to the file and THEN reading the file you will want to use a FileInputStream in place of your ByteArrayInputStream.
If you want to write a bunch of data to a byte array then you should take a look at using a ByteArrayOutputStream. If you then need to read the byte array as a ByteArrayInputStream you can pass the ByteArrayOutputStream into the input stream like what is shown below. Keep in mind this only works for writing and THEN reading. You can not use this like a buffer.
//Create output stream
ByteArrayOutputStream out = new ByteArrayOutputStream();
//Create Writer
Writer writer = new OutputStreamWriter(out);
//Write stuff
...
//Close writer
writer.close();
//Create input stream using the byte array from out as input.
ByteArrayInputStream in = new ByteArrayInputStream(out.toByteArray());
Short answer: you can't.
A ByteArrayInputStream is just not assignable from a OutputStreamWriter.
Since you're probably after write, you can just read the file back to a byte[] and then construct a ByteArrayInputStream with it:
File file = new File("S:\\Test.java");
FileInputStream fis = new FileInputStream(file);
byte[] content = new byte[(int) file.length()];
fis.read(content,0,content.length);
ByteArrayInputStream bais = new ByteArrayInputStream(content);
I'm a bit confused. I got this block of code :
public void serializeStock(){
FTPClient client = new FTPClient();
try{
client.connect("ftp.**.***");
client.login("***", "*****");
FileInputStream fin = new FileInputStream(someVariableContainingObject); // <-- The object ( StockVault stocks = new StockVault();
ObjectInputStream oos = new ObjectInputStream(fin);
client.storeFile("theObject", oos);
System.out.println("Wrote Stocks data to file");
client.logout();
client.disconnect();
}catch(Exception ex){
ex.printStackTrace();
}
}
I open a connection to my remote FTP server, I now want to write a object named "stocks" to the server. I know I can save the object file first, and then upload it. But is it possible to store the object in a "file" variable or something, and feed it directly to the FileInputStream? This way I wont have to save the object to an actual system file, and then upload it. But make some kind of variable to hold the file, and then upload the "file-variable" to the ftp-server.
It is possible to stream directly to the FTP server, but FileInputStream is not the right class to do so -- it always reads from a local file.
The simplest way to do this in your case may be to use
client.storeFileStream(remoteName)
This method returns an output stream you can write to (instead of sucking up all the data from a given input stream), which seems to be a more natural fit for your case.
Example:
OutputStream os = client.storeFileStream("theObject");
ObjectOutputStream oos = new ObjectOutputStream(os);
// write your object to oos here,
// then close everything.
An alternative that uses the same client method as in your example but no file would be to first write the object to a ByteArrayOutputStream, then create a ByteArrayInputStream from getBytes(), and hand that over to the ftp client.
BTW: The Object stream in your code can be omitted -- you can use the file stream directly:
FileInputStream fin = new FileInputStream(someVariableContainingObject);
client.storeFile("theObject", fin);
// ...
To process some images in my android application I currently use code like this:
FileOutputStream fileOuputStream = new FileOutputStream(imgpath);
[..DO SOME STUFF..]
Bitmap data = BitmapFactory.decodeByteArray(bFile, 0, bFile.length, options);
data.compress(Bitmap.CompressFormat.JPEG, 90, fileOuputStream);
[..DO SOME STUFF..]
File file = new File(imgpath);
FileInputStream imageInFile = new FileInputStream(file);
byte imageData[] = new byte[(int) file.length()];
imageInFile.read(imageData);
[..DO SOME STUFF..]
file.delete();
//NOTE: The code is all in the same method
the problem is that passing my image from one part of the code to another using this method creates a temporary file.
I was looking for a way to read / write the file data using a memory variable, something like "generic stream" in which store data in order to replace use of "FileInputStream " and "FileOutputStream " and do not write temporary file.
If you are able to use an InputStream or OutputStream you can use ByteArrayInputStream or ByteArrayOutputStream for in memory handling of the data.
If you have two thread you can also use PipedInputStream and PipedOutputStream together to communicate between the threads.
You could write your data to a ByteArrayOutputStream and use the byte array of that stream:
ByteArrayOutputStream out = new ByteArrayOutputStream();
data.compress(Bitmap.CompressFormat.JPEG, 90, out);
// now take the bytes out of your Stream
byte[] imgData = out.toByteArray();
I need help on my homework, any help will be much appreciated. I can send small files without a problem. But when i try to send let’s say a 1GB file byte array sends OutOfMemoryError so i need a better solution to send file from server to client. How can i improve this code and send big files, please help me.
Server Code:
FileInputStream fis = new FileInputStream(file);
byte[] fileByte = new byte[fis.available()]; //This causes the problem.
bytesRead = fis.read(fileByte);
oos = new ObjectOutputStream(sock.getOutputStream());
oos.writeObject(fileByte);
Client Code:
ois = new ObjectInputStream(sock.getInputStream());
byte[] file = (byte[]) ois.readObject();
fos = new FileOutputStream(file);
fos.write(file);
Don't read the whole file into memory, use a small buffer and write while you are reading the file:
BufferedOutputStream bos = new BufferedOutputStream(sock.getOutputStream())
File file = new File("asd");
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
byte[] buffer = new byte[1024*1024*10];
int n = -1;
while((n = bis.read(buffer))!=-1) {
bos.write(buffer,0,n):
}
Use Buffered* to optimize the writing and reading from Streams
Just split the array to smaller chunks so that you don't need to allocate any big array.
For example you could split the array into 16Kb chunks, eg new byte[16384] and send them one by one. On the receiving side you would have to wait until a chunk can be fully read and then store them somewhere and start with next chunk.
But if you are not able to allocate a whole array of the size you need on server side you won't be able to store all the data that you are going to receive anyway.
You could also compress the data before sending it to save bandwidth (and time), take a look at ZipOutputStream and ZipInputStream.
Here's how I solved it:
Client Code:
bis=new BufferedInputStream(sock.getInputStream());
fos = new FileOutputStream(file);
int n;
byte[] buffer = new byte[8192];
while ((n = bis.read(buffer)) > 0){
fos.write(buffer, 0, n);}
Server Code:
bos= new BufferedOutputStream(sock.getOutputStream());
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
int n=-1;
byte[] buffer = new byte[8192];
while((n = bis.read(buffer))>-1)
bos.write(buffer,0,n);
Depending on whether or not you have to write the code yourself, there are existing libraries which solve this problem, e.g. rmiio. If you are not using RMI, just plain java serialization, you can use the DirectRemoteInputStream, which is kind of like a Serializable InputStream. (this library also has support for things like auto-magically compressing the data).
Actually, if you are only sending file data, you would be better off ditching the Object streams and use DataInput/DataOutput streams. first write an integer indicating the file length, then copy the bytes directly to the stream. on the receiving side, read the integer file length, then read exactly that many bytes.
when you copy the data between streams, use a small, fixed size byte[] to move chunks of data between the input and output streams in a loop. there are numerous examples of how to do this correctly available online (e.g. #ErikFWinter's answer).
I`m hoping can help me out with a file creation/response question.
I know how to create and save a file. I know how to send that file back to the user via a ServletOutputStream.
But what I need is to create a file, without saving it on the disk, and then send that file via the ServletOutputStream.
The code above explains the parts that I have. Any help appreciated. Thanks in Advance.
// This Creates a file
//
String text = "These days run away like horses over the hill";
File file = new File("MyFile.txt");
Writer writer = new BufferedWriter(new FileWriter(file));
writer.write(text);
writer.close();
// Missing link goes here
//
// This sends file to browser
//
InputStream inputStream = null;
inputStream = new FileInputStream("C:\\MyFile.txt");
byte[] buffer = new byte[8192];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int bytesRead;
while ( (bytesRead = inputStream.read(buffer)) != -1)
baos.write(buffer, 0, bytesRead);
response.setContentType("text/html");
response.addHeader("Content-Disposition", "attachment; filename=Invoice.txt");
byte[] outBuf = baos.toByteArray();
stream = response.getOutputStream();
stream.write(outBuf);
You don't need to save off a file, just use a ByteArray stream, try something like this:
inputStream = new ByteArrayInputStream(text.getBytes());
Or, even simpler, just do:
stream.write(text.getBytes());
As cHao suggests, use text.getBytes("UTF-8") or something similar to specify a charset other than the system default. The list of available charsets is available in the API docs for Charset.