Someone explain to me what InputStream and OutputStream are?
I am confused about the use cases for both InputStream and OutputStream.
If you could also include a snippet of code to go along with your explanation, that would be great. Thanks!
The goal of InputStream and OutputStream is to abstract different ways to input and output: whether the stream is a file, a web page, or the screen shouldn't matter. All that matters is that you receive information from the stream (or send information into that stream.)
InputStream is used for many things that you read from.
OutputStream is used for many things that you write to.
Here's some sample code. It assumes the InputStream instr and OutputStream osstr have already been created:
int i;
while ((i = instr.read()) != -1) {
osstr.write(i);
}
instr.close();
osstr.close();
InputStream is used for reading, OutputStream for writing. They are connected as decorators to one another such that you can read/write all different types of data from all different types of sources.
For example, you can write primitive data to a file:
File file = new File("C:/text.bin");
file.createNewFile();
DataOutputStream stream = new DataOutputStream(new FileOutputStream(file));
stream.writeBoolean(true);
stream.writeInt(1234);
stream.close();
To read the written contents:
File file = new File("C:/text.bin");
DataInputStream stream = new DataInputStream(new FileInputStream(file));
boolean isTrue = stream.readBoolean();
int value = stream.readInt();
stream.close();
System.out.printlin(isTrue + " " + value);
You can use other types of streams to enhance the reading/writing. For example, you can introduce a buffer for efficiency:
DataInputStream stream = new DataInputStream(
new BufferedInputStream(new FileInputStream(file)));
You can write other data such as objects:
MyClass myObject = new MyClass(); // MyClass have to implement Serializable
ObjectOutputStream stream = new ObjectOutputStream(
new FileOutputStream("C:/text.obj"));
stream.writeObject(myObject);
stream.close();
You can read from other different input sources:
byte[] test = new byte[] {0, 0, 1, 0, 0, 0, 1, 1, 8, 9};
DataInputStream stream = new DataInputStream(new ByteArrayInputStream(test));
int value0 = stream.readInt();
int value1 = stream.readInt();
byte value2 = stream.readByte();
byte value3 = stream.readByte();
stream.close();
System.out.println(value0 + " " + value1 + " " + value2 + " " + value3);
For most input streams there is an output stream, also. You can define your own streams to reading/writing special things and there are complex streams for reading complex things (for example there are Streams for reading/writing ZIP format).
From the Java Tutorial:
A stream is a sequence of data.
A program uses an input stream to read data from a source, one item at a time:
A program uses an output stream to write data to a destination, one item at time:
The data source and data destination pictured above can be anything that holds, generates, or consumes data. Obviously this includes disk files, but a source or destination can also be another program, a peripheral device, a network socket, or an array.
Sample code from oracle tutorial:
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
public class CopyBytes {
public static void main(String[] args) throws IOException {
FileInputStream in = null;
FileOutputStream out = null;
try {
in = new FileInputStream("xanadu.txt");
out = new FileOutputStream("outagain.txt");
int c;
while ((c = in.read()) != -1) {
out.write(c);
}
} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}
}
}
This program uses byte streams to copy xanadu.txt file to outagain.txt , by writing one byte at a time
Have a look at this SE question to know more details about advanced Character streams, which are wrappers on top of Byte Streams :
byte stream and character stream
you read from an InputStream and write to an OutputStream.
for example, say you want to copy a file. You would create a FileInputStream to read from the source file and a FileOutputStream to write to the new file.
If your data is a character stream, you could use a FileReader instead of an InputStream and a FileWriter instead of an OutputStream if you prefer.
InputStream input = ... // many different types
OutputStream output = ... // many different types
byte[] buffer = new byte[1024];
int n = 0;
while ((n = input.read(buffer)) != -1)
output.write(buffer, 0, n);
input.close();
output.close();
OutputStream is an abstract class that represents writing output. There are many different OutputStream classes, and they write out to certain things (like the screen, or Files, or byte arrays, or network connections, or etc). InputStream classes access the same things, but they read data in from them.
Here is a good basic example of using FileOutputStream and FileInputStream to write data to a file, then read it back in.
A stream is a continuous flow of liquid, air, or gas.
Java stream is a flow of data from a source into a destination. The source or destination can be a disk, memory, socket, or other programs. The data can be bytes, characters, or objects. The same applies for C# or C++ streams. A good metaphor for Java streams is water flowing from a tap into a bathtub and later into a drainage.
The data represents the static part of the stream; the read and write methods the dynamic part of the stream.
InputStream represents a flow of data from the source, the OutputStream represents a flow of data into the destination.
Finally, InputStream and OutputStream are abstractions over low-level access to data, such as C file pointers.
Stream: In laymen terms stream is data , most generic stream is binary representation of data.
Input Stream : If you are reading data from a file or any other source , stream used is input stream. In a simpler terms input stream acts as a channel to read data.
Output Stream : If you want to read and process data from a source (file etc) you first need to save the data , the mean to store data is output stream .
An output stream is generally related to some data destination like a file or a network etc.In java output stream is a destination where data is eventually written and it ends
import java.io.printstream;
class PPrint {
static PPrintStream oout = new PPrintStream();
}
class PPrintStream {
void print(String str) {
System.out.println(str)
}
}
class outputstreamDemo {
public static void main(String args[]) {
System.out.println("hello world");
System.out.prinln("this is output stream demo");
}
}
For one kind of InputStream, you can think of it as a "representation" of a data source, like a file.
For example:
FileInputStream fileInputStream = new FileInputStream("/path/to/file/abc.txt");
fileInputStream represents the data in this path, which you can use read method to read bytes from the file.
For the other kind of InputStream, they take in another inputStream and do further processing, like decompression.
For example:
GZIPInputStream gzipInputStream = new GZIPInputStream(fileInputStream);
gzipInputStream will treat the fileInputStream as a compressed data source. When you use the read(buffer, 0, buffer.length) method, it will decompress part of the gzip file into the buffer you provide.
The reason why we use InputStream because as the data in the source becomes larger and larger, say we have 500GB data in the source file, we don't want to hold everything in the memory (expensive machine; not friendly for GC allocation), and we want to get some result faster (reading the whole file may take a long time).
The same thing for OutputStream. We can start moving some result to the destination without waiting for the whole thing to finish, plus less memory consumption.
If you want more explanations and examples, you have check these summaries: InputStream, OutputStream, How To Use InputStream, How To Use OutputStream
In continue to the great other answers, in my simple words:
Stream - like mentioned #Sher Mohammad is data.
Input stream - for example is to get input – data – from the file. The case is when I have a file (the user upload a file – input) – and I want to read what we have there.
Output Stream – is the vice versa. For example – you are generating an excel file, and output it to some place.
The “how to write” to the file, is defined at the sender (the excel workbook class) not at the file output stream.
See here example in this context.
try (OutputStream fileOut = new FileOutputStream("xssf-align.xlsx")) {
wb.write(fileOut);
}
wb.close();
Related
I am trying to understand what is the correct way to read a file in Java Servlet program. I need to read a file from a fixed path on my machine using my servlet code. Now I can read the file in multiple ways and one of the way which I am planning to use is to read the information in bytes as shown in below code:
private static void readFile(HttpServletRequest req, HttpServletResponse resp, String path)
throws IOException
{
File file = new File("C:\\temp\", path);
if (!file.isFile()) {
resp.sendError(404, "File not found: " + file);
return;
}
InputStream in = null;
ServletOutputStream out = null;
try {
resp.setContentLength(Long.valueOf(file.length()).intValue());
resp.resetBuffer();
out = resp.getOutputStream();
in = new BufferedInputStream(new FileInputStream(file));
readFile(in, out);
}
finally {
//Code for closing the input & output steams
}
}
}
public static void readFile(InputStream in, OutputStream out) throws IOException
{
byte[] buf = new byte[4096];
int data;
while ((data = in.read(buf, 0, buf.length)) != -1)
out.write(buf, 0, data);
}
I don't have issues with this logic and it is working fine.
Now I came across the post How To Read File In Java – BufferedReader in mykyong site and here the example uses BufferedReader.
Can someone please tell me which is the efficient way of reading a file in servlet code? when we need to prefer using BufferedReader in comparison to reading data in bytes.
There's almost no reason to read a file manually anymore since Java 7's NIO.
Just use Files.readAllBytes(Path) to read the full byte[]. Or if you want to stream directly to an OutputStream, Files.copy(Path, OutputStream).
Can someone please tell me which is the efficient way of reading a
file in servlet code? when we need to prefer using BufferedReader in
comparison to reading data in bytes.
Any buffered method will work. Here, BufferedReader allows you to read streams as String values. As the javadoc says
Reads text from a character-input stream, buffering characters so as
to provide for the efficient reading of characters, arrays, and lines.
I use the "get" method from java drive api, and I can get the inputstream. but I cannt open the file when I use the inputstream to creat it. It likes the file is broken.
private static String fileurl = "C:\\googletest\\drive\\";
public static void newFile(String filetitle, InputStream stream) throws IOException {
String filepath = fileurl + filetitle;
BufferedInputStream bufferedInputStream=new BufferedInputStream(stream);
byte[] buffer = new byte[bufferedInputStream.available()];
File file = new File(filepath);
if (!file.exists()) {
file.getParentFile().mkdirs();
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(new FileOutputStream(filepath));
while( bufferedInputStream.read(buffer) != -1) {
bufferedOutputStream.write(buffer);
}
bufferedOutputStream.flush();
bufferedOutputStream.close();
}
}
Firstly, C:\googletest\drive\ is not a URL. It is a file system pathname.
Next, the following probably does not do what you think it does:
byte[] buffer = new byte[bufferedInputStream.available()];
The problem is that the available() call can return zero ... for a non-empty stream. The value returned by available() is an estimate of how many bytes that are currently available to read ... right now. That is not necessarily the stream length ... or anything related to it. And indeed the device drivers for some devices consistently return zero, even when there is data to be read.
Finally, this is wrong:
while( bufferedInputStream.read(buffer) != -1) {
bufferedOutputStream.write(buffer);
You are assuming that read returning -1 means that it filled the buffer. That is not so. Any one of the read calls could return with a partly full buffer. But then you write the entire buffer contents to the output stream ... including "junk" from previous reads.
Either or both of the 2nd and 3rd problems could lead to file corruption. In fact, the third one is likely to.
I get a java.lang.ArrayIndexOutOfBoundsException when using ByteArrayInputStream.
First, I use a ZipInputStream to read through a zip file,
and while looping through the zipEntries,
I use a ByteArrayInputStream to capture the data of each zipEntry
using the
ZipInputStream.read(byte[] b) and ByteArrayInputStream(byte[] b) methods.
At the end, I have a total of 6 different ByteArrayInputStream objects containing data from 6 different zipEntries.
I then use OpenCSV to read through each of the ByteArrayInputStream.
I have no problem reading 4 of the 6 ByteArrayInputStream objects, of which have byte sizes of less than 2000.
The other 2 ByteArrayInputStream objects have byte sizes of 2155 and 4010 respectively and the CSVreader was only able to read part of these 2 objects, then give an java.lang.ArrayIndexOutOfBoundsException.
This is the code I used to loop through the ZipInputStream
InputStream fileStream = attachment.getInputStream();
try {
ZipInputStream zippy = new ZipInputStream(fileStream);
ZipEntry entry = zippy.getNextEntry();
ByteArrayInputStream courseData = null;
while (entry!= null) {
String name = entry.getName();
long size = entry.getSize();
if (name.equals("course.csv")) {
courseData = copyInputStream(zippy, (int)size);
}
//similar IF statements for 5 other ByteArrayInputStream objects
entry = zippy.getNextEntry();
}
CourseDataManager.load(courseData);
}catch(Exception e){
e.printStackTrace();
}
The following is the code with which I use to copy the data from the ZipInputStream to the ByteArrayInputStream.
public ByteArrayInputStream copyInputStream(InputStream in, int size)
throws IOException {
byte[] buffer = new byte[size];
in.read(buffer);
ByteArrayInputStream b = new ByteArrayInputStream(buffer);
return b;
}
The 2 sets of openCSV codes are able to read a few lines of data, before throwing that exception, which leads me to believe that it is the byteArray that is causing the problem. Is there anything I can do or work around this problem?
I am trying to make an application that accepts a zip file, while not storing any temporary files in the web app, as I am deploying to both google app engine and tomcat server.
Fixed!!! Thanks to stephen C, i realized that read(byte[]) does not read everything so I adjusted the code to make the copyInputStream fully functional.
Since this looks like homework, here's a hint:
The read(byte[]) method returns the number bytes read.
On what line do you get the error? And have you checked the value of size? I suspect it's 0
I have a file that contains some amount of plain text at the start followed by binary content at the end. The size of the binary content is determined by some one of the plain text lines I read.
I was using a BufferedReader to read the individual lines, however it exposes no methods to refer to read a byte array. The readUTF for a DataInputStream doesnt read all the way to the end of the line, and the readLine method is deprecated.
Using the underlying FileInputStream to read returns empty byte arrays. Any suggestions on how to go about this?
private DOTDataInfo parseFile(InputStream stream) throws IOException{
DOTDataInfo info = new DOTDataInfo();
BufferedReader reader = new BufferedReader(new InputStreamReader(stream));
int binSize = 0;
String line;
while((line = reader.readLine()) != null){
if(line.length() == 0)
break;
DOTProperty prop = parseProperty(line);
info.getProperties().add(prop);
if(prop.getName().equals("ContentSize"))
binSize = Integer.parseInt(prop.getValue());
}
byte[] content = new byte[binSize];
stream.read(content); //Its all empty now. If I use a DataInputStream instead, its got the values from the file
return info;
}
You could use RandomAccessFile. Use readLine() to read the plain text at the start (note the limitations of this, as described in the API), and then readByte() or readFully() to read the subsequent binary data.
Using the underlying FileInputStream
to read returns empty byte arrays.
That's because you have wrapped the stream in a BufferedReader, which has probably consumed all the bytes from the stream when filling up its buffer.
If you genuinely have a file (rather than something harder to seek in, e.g. a network stream) then I suggest something like this:
Open the file as a FileInputStream
Wrap it in InputStreamReader and a BufferedReader
Read the text, so you can find out how much content there is
Close the BufferedReader (which will close the InputStreamReader which will close the FileInputStream)
Reopen the file
Skip to (total file length - binary content length)
Read the rest of the data as normal
You could just call mark() at the start of the FileInputStream and then reset() and skip() to get to the right place if you want to avoid reopening the file. (I was looking for an InputStream.seek() but I can't see one - I can't remember wanting it before in Java, but does it really not have one? Ick.)
You need to use an InputStream. Readers are for character data. Look into wrapping your input stream with a DataInputStream, like:
stream=new DataInputStream(new BufferedInputStream(new FileInputStream(...)));
The data input stream will give you many useful methods to read various types of data, and of course, the base InputStream methods for reading bytes.
(This is actually exactly what a HTTP server must do to read a request with content.)
The readUTF doesn't read a line, it reads a string that was written in (modified) UTF8 format - refer to the JavaDoc.
Alas, DataInputStream is deprecated and does not handle UTF. But this should help (it reads a line from a binary stream, without any lookahead).
public static String lineFrom(InputStream in) throws IOException {
byte[] buf = new byte[128];
int pos = 0;
for (;;) {
int ch = in.read();
if (ch == '\n' || ch < 0) break;
buf[pos++] = (byte) ch;
if (pos == buf.length) buf = Arrays.copyOf(buf, pos + 128);
}
return new String(Arrays.copyOf(buf, pos), "UTF-8");
}
The correct way is to use an InputStream of some form, probably a FileInputStream unless this becomes a performance barrier.
What do you mean "Using the underlying FileInputStream to read returns empty byte arrays."? This seems very unlikely and is probably where your mistake is. Can you show us the example code you've tried?
You can read the text with BufferedReader. When you know where the binary starts you can close the file and open it with RandomAccessFile and read binary from any point in the file.
Or you can read the file as binary and convert to text the sections you identify as text. {Using new String(bytes, encoding)}
I recommend using DataInputStream. You have the following options:
Read both text and binary content with DataInputStream
Open a BufferedReader, read text and close the stream. Then open a DataInputStream, skip bytes equal to the size of the text and read binary data.
I am trying to read a single file from a java.util.zip.ZipInputStream, and copy it into a java.io.ByteArrayOutputStream (so that I can then create a java.io.ByteArrayInputStream and hand that to a 3rd party library that will end up closing the stream, and I don't want my ZipInputStream getting closed).
I'm probably missing something basic here, but I never enter the while loop here:
ByteArrayOutputStream streamBuilder = new ByteArrayOutputStream();
int bytesRead;
byte[] tempBuffer = new byte[8192*2];
try {
while ((bytesRead = zipStream.read(tempBuffer)) != -1) {
streamBuilder.write(tempBuffer, 0, bytesRead);
}
} catch (IOException e) {
// ...
}
What am I missing that will allow me to copy the stream?
Edit:
I should have mentioned earlier that this ZipInputStream is not coming from a file, so I don't think I can use a ZipFile. It is coming from a file uploaded through a servlet.
Also, I have already called getNextEntry() on the ZipInputStream before getting to this snippet of code. If I don't try copying the file into another InputStream (via the OutputStream mentioned above), and just pass the ZipInputStream to my 3rd party library, the library closes the stream, and I can't do anything more, like dealing with the remaining files in the stream.
Your loop looks valid - what does the following code (just on it's own) return?
zipStream.read(tempBuffer)
if it's returning -1, then the zipStream is closed before you get it, and all bets are off. It's time to use your debugger and make sure what's being passed to you is actually valid.
When you call getNextEntry(), does it return a value, and is the data in the entry meaningful (i.e. does getCompressedSize() return a valid value)? IF you are just reading a Zip file that doesn't have read-ahead zip entries embedded, then ZipInputStream isn't going to work for you.
Some useful tidbits about the Zip format:
Each file embedded in a zip file has a header. This header can contain useful information (such as the compressed length of the stream, it's offset in the file, CRC) - or it can contain some magic values that basically say 'The information isn't in the stream header, you have to check the Zip post-amble'.
Each zip file then has a table that is attached to the end of the file that contains all of the zip entries, along with the real data. The table at the end is mandatory, and the values in it must be correct. In contrast, the values embedded in the stream do not have to be provided.
If you use ZipFile, it reads the table at the end of the zip. If you use ZipInputStream, I suspect that getNextEntry() attempts to use the entries embedded in the stream. If those values aren't specified, then ZipInputStream has no idea how long the stream might be. The inflate algorithm is self terminating (you actually don't need to know the uncompressed length of the output stream in order to fully recover the output), but it's possible that the Java version of this reader doesn't handle this situation very well.
I will say that it's fairly unusual to have a servlet returning a ZipInputStream (it's much more common to receive an inflatorInputStream if you are going to be receiving compressed content.
You probably tried reading from a FileInputStream like this:
ZipInputStream in = new ZipInputStream(new FileInputStream(...));
This won’t work since a zip archive can contain multiple files and you need to specify which file to read.
You could use java.util.zip.ZipFile and a library such as IOUtils from Apache Commons IO or ByteStreams from Guava that assist you in copying the stream.
Example:
ByteArrayOutputStream out = new ByteArrayOutputStream();
try (ZipFile zipFile = new ZipFile("foo.zip")) {
ZipEntry zipEntry = zipFile.getEntry("fileInTheZip.txt");
try (InputStream in = zipFile.getInputStream(zipEntry)) {
IOUtils.copy(in, out);
}
}
I'd use IOUtils from the commons io project.
IOUtils.copy(zipStream, byteArrayOutputStream);
You're missing call
ZipEntry entry = (ZipEntry) zipStream.getNextEntry();
to position the first byte decompressed of the first entry.
ByteArrayOutputStream streamBuilder = new ByteArrayOutputStream();
int bytesRead;
byte[] tempBuffer = new byte[8192*2];
ZipEntry entry = (ZipEntry) zipStream.getNextEntry();
try {
while ( (bytesRead = zipStream.read(tempBuffer)) != -1 ){
streamBuilder.write(tempBuffer, 0, bytesRead);
}
} catch (IOException e) {
...
}
You could implement your own wrapper around the ZipInputStream that ignores close() and hand that off to the third-party library.
thirdPartyLib.handleZipData(new CloseIgnoringInputStream(zipStream));
class CloseIgnoringInputStream extends InputStream
{
private ZipInputStream stream;
public CloseIgnoringInputStream(ZipInputStream inStream)
{
stream = inStream;
}
public int read() throws IOException {
return stream.read();
}
public void close()
{
//ignore
}
public void reallyClose() throws IOException
{
stream.close();
}
}
I would call getNextEntry() on the ZipInputStream until it is at the entry you want (use ZipEntry.getName() etc.). Calling getNextEntry() will advance the "cursor" to the beginning of the entry that it returns. Then, use ZipEntry.getSize() to determine how many bytes you should read using zipInputStream.read().
It is unclear how you got the zipStream. It should work when you get it like this:
zipStream = zipFile.getInputStream(zipEntry)
t is unclear how you got the zipStream. It should work when you get it like this:
zipStream = zipFile.getInputStream(zipEntry)
If you are obtaining the ZipInputStream from a ZipFile you can get one stream for the 3d party library, let it use it, and you obtain another input stream using the code before.
Remember, an inputstream is a cursor. If you have the entire data (like a ZipFile) you can ask for N cursors over it.
A diferent case is if you only have an "GZip" inputstream, only an zipped byte stream. In that case you ByteArrayOutputStream buffer makes all sense.
Please try code bellow
private static byte[] getZipArchiveContent(File zipName) throws WorkflowServiceBusinessException {
BufferedInputStream buffer = null;
FileInputStream fileStream = null;
ByteArrayOutputStream byteOut = null;
byte data[] = new byte[BUFFER];
try {
try {
fileStream = new FileInputStream(zipName);
buffer = new BufferedInputStream(fileStream);
byteOut = new ByteArrayOutputStream();
int count;
while((count = buffer.read(data, 0, BUFFER)) != -1) {
byteOut.write(data, 0, count);
}
} catch(Exception e) {
throw new WorkflowServiceBusinessException(e.getMessage(), e);
} finally {
if(null != fileStream) {
fileStream.close();
}
if(null != buffer) {
buffer.close();
}
if(null != byteOut) {
byteOut.close();
}
}
} catch(Exception e) {
throw new WorkflowServiceBusinessException(e.getMessage(), e);
}
return byteOut.toByteArray();
}
Check if the input stream is positioned in the begging.
Otherwise, as implementation: I do not think that you need to write to the result stream while you are reading, unless you process this exact stream in another thread.
Just create a byte array, read the input stream, then create the output stream.