Develop Reference

Need to improve performance in file unzipping from DB

I am getting zipped blob from db and using that blob in below way,
Ex:-
byte[] inputBlob = blobfile.getBytes(1, (int) blobfile.length());
After getting the blob, the way i got the zippedStream and passed it into another Class method(unzipper).
Ex:-
ByteArrayOutputStream zippedStream = null;
InputStream byteInputStream = null;
IParser parser = null;
byte[] buffer = null;
try {
zippedStream = new ByteArrayOutputStream();
byteInputStream = new ByteArrayInputStream(blob);
blob = null;
int bytes_read;
buffer = new byte[byteInputStream.available()];
while ((bytes_read = byteInputStream.read(buffer)) > 0) {
zippedStream.write(buffer, 0, bytes_read);
}
buffer = null;
byteInputStream.close();
byteInputStream = null;
}catch(Exception e){
e.printStackTrace();
}
unzipper method:
Ex:-
byte[] buffer = new byte[1024];
try {
InputStream decodedInput = new ByteArrayInputStream(zippedStream.toByteArray());
zippedStream.close();
zippedStream = null;
GZIPInputStream unzippedStream = new GZIPInputStream(decodedInput);
decodedInput.close();
decodedInput = null;
int bytes_read;
unzippedOutputstream = new ByteArrayOutputStream();
while ((bytes_read = unzippedStream.read(buffer)) > 0) {
unzippedOutputstream.write(buffer, 0, bytes_read);
}
buffer = null;
unzippedStream.close();
unzippedStream = null;
} catch (Exception ex) {
logger.setException(ex);
logger.error("unzipper", generateMsg("Exception occurred"));
}
Using this way my application got stucked some time, and performance was so bad.
Is there any optimize way to get the zippedstream file and unzipping that easily?

Is all this buffering really needed. Can you IParser parse a Stream?
InputStream zippedStream = ...
IParser parser = ...
parser.parse(new GZIPInputStream(zippedStream));
This will read compressed data, uncompressing as it goes which is much more efficient.

avoiding garbage data while reading data using a byte buffer

I am trying to write a program to transfer a file between client and server using java tcp sockets I am using buffer size of 64K but The problem I am facing is that when when the tcp sometimes fail to send the whole 64K it sends the remaing part for example 32K in anther go
There for A garbage data of some Spaces or so is being taken by the buffer at reading side to make 64K complete and thus unnecessary data is making the file useless at receiving side.
Is there any solution to overcome this problem ???
I am using TCP protocol this code is using to send data to client
Server-side code
File transferFile = new File ("Document.txt");
byte [] bytearray = new byte [1024];
int byRead=0;
FileInputStream fin = new FileInputStream(transferFile);
BufferedInputStream bin = new BufferedInputStream(fin);
OutputStream os = socket.getOutputStream();
while(byRead>-1) {
byRead=bin.read(bytearray,0,bytearray.length);
os.write(bytearray,0,bytearray.length);
os.flush();
}
Client-side code
byte [] bytearray = new byte [1024];
InputStream is = socket.getInputStream();
FileOutputStream fos = new FileOutputStream("C:\\Users\\NetBeansProjects\\"+filename);
BufferedOutputStream bos = new BufferedOutputStream(fos);
bytesRead = is.read(bytearray,0,bytearray.length);
currentTot = bytesRead; System.out.println("Data is being read ...");
do {
bytesRead = is.read(bytearray, 0, (bytearray.length));
if(bytesRead == 0) continue;
if(bytesRead >= 0) currentTot += bytesRead;
bos.write(bytearray,0,bytearray.length);
} while(bytesRead > -1);
here I tried to skip the loop if the byte is empty by continue; statement but it is not
working.

bos.write(bytearray,0,bytearray.length);
This should be
bos.write(bytearray,0,bytesRead);
The region after 'bytesRead' in the buffer is undisturbed by the read. It isn't 'garbage'. It's just whatever was there before.

use CLIENT Side Code as below to get the total write bytes without garbage
int availableByte = socket.available();
if (availableByte > 0) {
byte[] buffer = new byte[availableByte];
int bytesRead = socketInputStream.read(buffer);
FileOutputStream fileOutputStream = new FileOutputStream(FilePath, true);
OutputStreamWriter outputStreamWriter = new OutputStreamWriter(fileOutputStream);
BufferedWriter bufferedWriter = new BufferedWriter(outputStreamWriter);
bufferedWriter.write(buffer.toString());
bufferedWriter.close();
}

How to get a binary file from a remote php script response?

I'm calling a script that gives me a binary file (12345.cl), with binary data. The script is done, and it's working, if I paste it on the navigator I get the binary file.
Now I have a problem: How I transform the response of the script into a binary resource to use it in my app?
For the moment, i have this code:
public void decodeStream( String mURL ){
BufferedInputStream bis = new BufferedInputStream(new URL(mURL).openStream(), BUFFER_IO_SIZE);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(baos, BUFFER_IO_SIZE);
copy(bis, bos);
bos.flush();
Then, I have a BufferedOutputStream with the response, but I don't know how to transform it into a binary resource to use it
I need to obtain a datainputstream with the file but I don't know how to achieve it

You can use following code:
public void decodeStream( String mURL, String ofile ) throws Exception {
InputStream in = null;
FileOutputStream out = null;
try {
URL url = new URL(mURL);
URLConnection urlConn = url.openConnection();
in = urlConn.getInputStream();
out = new FileOutputStream(ofile);
int c;
byte[] b = new byte[1024];
while ((c = in.read(b)) != -1)
out.write(b, 0, c);
} finally {
if (in != null)
in.close();
if (out != null)
out.close();
}
}

Socket Programming file upload issues?Complete file not uploading

Hi i am using the following code for uploding my file from android phone to the server bt the file does not upload completely..e.g i uploded a 11kb file and got only 8kb file at the server.What am i doing wrong?
Client side
Socket skt = new Socket"112.***.*.**", 3000);
String FileName=fil.getName();
PrintWriter out2 = new PrintWriter(new BufferedWriter(new OutputStreamWriter(skt.getOutputStream())),true);
out2.println("Upload");
out2.println(FileName);
out2.println(spinindx);
out2.println(singleton.arrylst_setngs.get(0).toString());
out2.println(singleton.arrylst_setngs.get(1).toString());
out2.println(singleton.arrylst_setngs.get(2).toString());
out2.println(singleton.arrylst_setngs.get(3).toString());
out2.println(singleton.arrylst_setngs.get(4).toString());
out2.flush();
//Create a file input stream and a buffered input stream.
FileInputStream fis = new FileInputStream(fil);
BufferedInputStream in = new BufferedInputStream(fis);
BufferedOutputStream out = new BufferedOutputStream(skt.getOutputStream());
//Write the file to the server socket
int i;
byte[] buf = new byte[512];
while ((i = in.read(buf)) != -1) {
out.write(buf,0,i);
publishProgress(in.available());
System.out.println(i);
}
//Close the writers,readers and the socket.
in.close();
out.flush();
out.close();
out2.close();
skt.close();
}
catch( Exception e ) {
System.out.println(e);
}
The server side
InputStream inStream = socket.getInputStream();
BufferedReader inm = new BufferedReader(new InputStreamReader(inStream));
String Request=inm.readLine();
if(Request.equals("Upload")){
fileName = inm.readLine();
chosn = inm.readLine();
lt=inm.readLine();
cs = inm.readLine();
om = inm.readLine();
o = inm.readLine();
check=inm.readLine();
//Read, and write the file to the socket
BufferedInputStream in = new BufferedInputStream(inStream);
int i=0;
File f=new File("D:/data/"+filePrefx+fileName);
if(!f.exists()){
f.createNewFile();
}
FileOutputStream fos = new FileOutputStream("D:/data/"+filePrefx+fileName);
BufferedOutputStream out = new BufferedOutputStream(fos);
byte[] buf = new byte[512];
while ((i = in.read(buf)) != -1) {
System.out.println(i);
out.write(buf,0,i);
System.out.println("Receiving data...");
}
in.close();
inStream.close();
out.close();
fos.close();
socket.close();

Looks like you are using both a BufferedReader and a BufferedInputStream on the same underlying socket at the server side, and two kinds of output stream/writer at the client. So your BufferedReader is buffering, which is what it's supposed to do, and thus 'stealing' some of the data you're expecting to read with the BufferedInputStream. Moral: you can't do that. Use DataInputStream & DataOutputStream only, and writeUTF()/readUTF() for the 8 lines you are reading from the client before the file.

You shared the same underlying InputStream between your BufferedReader and bufferedInputStream.
What happened is, when you do the reading through BufferedReader, it reads more than the a few lines you requested from the underlying InputStream into its own internal buffer. And when you create the BufferedInputStream, the data has already been read by the BufferedReader. So Apart from what EJP suggested not to use any buffered class, you can create the BufferedInputStream, and then create the Reader on Top of it. The code is something like this:
BufferedInputStream in = new BufferedInputStream(inStream);
Reader inm = new InputStreamReader(in);
Add it to the beginning of your server code and remove this line:
BufferedInputStream in = new BufferedInputStream(inStream);

See this, i never tried though
void read() throws IOException {
log("Reading from file.");
StringBuilder text = new StringBuilder();
String NL = System.getProperty("line.separator");
Scanner scanner = new Scanner(new FileInputStream(fFileName), fEncoding);
try {
while (scanner.hasNextLine()){
text.append(scanner.nextLine() + NL);
}
}
finally{
scanner.close();
}
log("Text read in: " + text);
}
Shamelessly copied from
http://www.javapractices.com/topic/TopicAction.do?Id=42

How do I get the InputStream of decompressed data from an InputStream of GZIPed data?

I call a service which returns a gzipped file. I have the data as an InputStream (courtesy of javax.activation.DataHandler.getInputStream();) from the response.
What I would like to do is, without writing anything to disk, get an InputStream of the decompressed data in the file that is in the archive. The compressed file in this case is an xml document that I am trying to unmarshal using javax.xml.bind.Unmarshaller which takes an InputStream.
I'm currently trying to write the InputStream to an OutputStream (decompressing the data) and then I'll need to write it back to an InputStream. It's not working yet so I thought I would see if there was a better (I would hope so) approach.
I can write the initial InputStream to disk and get a gz file, and then read that file, get the compressed file out of it and go from there but I'd rather keep it all in memory is possible.
Update 1: Here is my current (not working - get a "Not in GZIP format" exception):
ByteArrayInputStream xmlInput = null;
try {
InputStream in = dh.getInputStream(); //dh is a javax.activation.DataHandler
BufferedInputStream bis = new BufferedInputStream(in);
ByteArrayOutputStream bo = new ByteArrayOutputStream();
int bytes_read = 0;
byte[] dataBuf = new byte[4096];
while ((bytes_read = bis.read(dataBuf)) != -1) {
bo.write(dataBuf, 0, bytes_read);
}
ByteArrayInputStream bin = new ByteArrayInputStream(bo.toByteArray());
GZIPInputStream gzipInput = new GZIPInputStream(bin);
ByteArrayOutputStream out = new ByteArrayOutputStream();
dataBuf = new byte[4096];;
bytes_read = 0;
while ((bytes_read = gzipInput.read(dataBuf)) > 0) {
out.write(dataBuf, 0, bytes_read);
}
xmlInput = new ByteArrayInputStream(out.toByteArray());
If instead of writing to a ByteArrayOutputStream I write to a FileOutputStream the first time around I get a compressed file (which I can manually open to get the xml file within) and the service (eBay) says it should be a gzip file so I'm not sure why I get a "Not in GZIP format" error.
Update 2: I tried something a little different - same error ("Not in GZIP format"). Wow, I just tried to end that parenthesis with a semi-colon. Anyways, here is my second attempt, which still does not work:
ByteArrayInputStream xmlInput = null;
try {
GZIPInputStream gzipInput = new GZIPInputStream(dh.getInputStream());
ByteArrayOutputStream bo = new ByteArrayOutputStream();
int bytes_read = 0;
byte[] dataBuf = new byte[4096];
while ((bytes_read = gzipInput.read(dataBuf)) != -1) {
bo.write(dataBuf, 0, bytes_read);
}
xmlInput = new ByteArrayInputStream(bo.toByteArray());

Decorate the input stream with a GZIPInputStream.
InputStream decompressed = new GZIPInputStream(compressed);

The following code should work. Keep in mind you'll have to handle exceptions properly.
OutputStream out = null;
InputStream in = null;
try {
out = /* some output stream */;
in = new java.util.GZIPInputStream(/*some stream*/);
byte[] buffer = new byte[4096];
int c = 0;
while (( c = in.read(buffer, 0, 4096)) > 0) {
out.write(buffer, 0, c);
}
} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}

Take a look at GZIPInputStream. Here's an example; the class handles this very transparently, it's almost no work to use.