Java InputStream Downloading Slowly - java

I am using this code to download an encrypted file:
InputStream frame = new URL(url).openStream();
ByteArrayOutputStream data = new ByteArrayOutputStream();
int count;
byte[] buff = new byte[10000];
while((count = frame.read(buff)) > 0){
data.write(buff, 0, count);
System.out.println(count);
}
This code works great sometimes, but others it can take up to a minute for a ~36 kilobyte file. Is there a better way of doing this or is the problem simply the connection to the download server?

Related

only first few hundred samples are read from input stream into byte array(the rest are zeros)

i'm trying to convert audio(mp3/wav etc.) to byte array. i did it using inputStream to byte array conversion.
the problem is that after few hundred samples i recieve only zeroes.
at first i thought the problem was with the file so i tried debugging with another file and had the same problem.
I thought the problem was with the code so i tried using IOUtils and got the exact same resualts.
can anyone tell me what i'm doing wrong?
the code i used:
File file = new File(path);
final InputStream inputStream = new FileInputStream(file);
byte[] byteSamples = inputStreamToByteArray(inputStream);
public byte[] inputStreamToByteArray(InputStream inStream) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = inStream.read(buffer)) > 0) {
baos.write(buffer, 0, bytesRead);
}
return baos.toByteArray();
}
the alternate version using IOUtils:
byte[] byteSamples = IOUtils.toByteArray(inputStream);
update : i tried doing it using BufferedInputStream, still the exact same results.
byte[] byteSamples = new byte[(int)file.length()];
try {
BufferedInputStream buf = new BufferedInputStream(new FileInputStream(file));
buf.read(byteSamples, 0, byteSamples.length);
buf.close();
} catch (FileNotFoundException e) {
e.printStackTrace();}
You need to close the streams when done.

How to download a dataset using java

I want to make an app that will download all the paper mentioned in the image by taking the domain as input. i have been able to download a single file from some other website but was unable to download a paper form acm digital library.
what i need to do is download the entire data set.
here is the code that i used to download a single file.
String fileName = "1.txt";
URL link = new URL("http://shayconcepts.com");
InputStream in = new BufferedInputStream(link.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buf = new byte[2048];
int n = 0;
while (-1!=(n=in.read(buf)))
{
out.write(buf, 0, n);
}
out.close();
in.close();
byte[] response = out.toByteArray();
FileOutputStream fos = new FileOutputStream(fileName);
fos.write(response);
fos.close();
How can I modify it to download entire data set

avoiding garbage data while reading data using a byte buffer

I am trying to write a program to transfer a file between client and server using java tcp sockets I am using buffer size of 64K but The problem I am facing is that when when the tcp sometimes fail to send the whole 64K it sends the remaing part for example 32K in anther go
There for A garbage data of some Spaces or so is being taken by the buffer at reading side to make 64K complete and thus unnecessary data is making the file useless at receiving side.
Is there any solution to overcome this problem ???
I am using TCP protocol this code is using to send data to client
Server-side code
File transferFile = new File ("Document.txt");
byte [] bytearray = new byte [1024];
int byRead=0;
FileInputStream fin = new FileInputStream(transferFile);
BufferedInputStream bin = new BufferedInputStream(fin);
OutputStream os = socket.getOutputStream();
while(byRead>-1) {
byRead=bin.read(bytearray,0,bytearray.length);
os.write(bytearray,0,bytearray.length);
os.flush();
}
Client-side code
byte [] bytearray = new byte [1024];
InputStream is = socket.getInputStream();
FileOutputStream fos = new FileOutputStream("C:\\Users\\NetBeansProjects\\"+filename);
BufferedOutputStream bos = new BufferedOutputStream(fos);
bytesRead = is.read(bytearray,0,bytearray.length);
currentTot = bytesRead; System.out.println("Data is being read ...");
do {
bytesRead = is.read(bytearray, 0, (bytearray.length));
if(bytesRead == 0) continue;
if(bytesRead >= 0) currentTot += bytesRead;
bos.write(bytearray,0,bytearray.length);
} while(bytesRead > -1);
here I tried to skip the loop if the byte is empty by continue; statement but it is not
working.
bos.write(bytearray,0,bytearray.length);
This should be
bos.write(bytearray,0,bytesRead);
The region after 'bytesRead' in the buffer is undisturbed by the read. It isn't 'garbage'. It's just whatever was there before.
use CLIENT Side Code as below to get the total write bytes without garbage
int availableByte = socket.available();
if (availableByte > 0) {
byte[] buffer = new byte[availableByte];
int bytesRead = socketInputStream.read(buffer);
FileOutputStream fileOutputStream = new FileOutputStream(FilePath, true);
OutputStreamWriter outputStreamWriter = new OutputStreamWriter(fileOutputStream);
BufferedWriter bufferedWriter = new BufferedWriter(outputStreamWriter);
bufferedWriter.write(buffer.toString());
bufferedWriter.close();
}

binary data socket over android and java

I am using this code to get my image from facebook server.
String imageURL = "http://graph.facebook.com/"+id+"/picture?type=large";
InputStream inputStream = new URL(imageURL).openConnection().getInputStream();
DataInputStream dataStream = new DataInputStream(inputStream);
byte[] dataBuffer = new byte[4096];
ByteArrayOutputStream dataHolder = new ByteArrayOutputStream();
int n;
while ( (n=dataStream.read(dataBuffer))!=-1 ){
dataHolder.write(dataBuffer, 0, n);
}
and using the same way to get my image from my server:
FileInputStream inputStream = new FileInputSTream("/image.jpg");
DataOutputStream dataStream = new DataOutputStream(socket.getOutpustStream());
byte[] dataBuffer = new byte[4096];
int n;
while ( (n=inpuStream.read(dataBuffer))!=-1 ){
dataStream.write(dataBuffer, 0, n);
}//get -1 perfectly
and getting them on android app like this:
DataInputStream dataStream = new DataInputStream(socket.getInputStream());
byte[] dataBuffer = new byte[4096];
ByteArrayOutputStream dataHolder = new ByteArrayOutputStream();
int n;
while ( (n=dataStream.read(dataBuffer))!=-1 ){
dataHolder.write(dataBuffer, 0, n);
}//-1 ??????????????
but the android app cant get the right file and it cant go out form while loop,exactly can not get -1 but yesterday I accidently got -1 from some changes I did,really can not understand why I can not get -1 in my android app however I can get it from the servers while loop
You can directly convert the inputstream to a bitmap instead of reading bytes.
Bitmap myBitmap = BitmapFactory.decodeStream(inputStream);
Here is just some glassball guessing: Did you close the stream on the server side?
Put a dataStream.close(); after your writing loop.

How do I get the InputStream of decompressed data from an InputStream of GZIPed data?

I call a service which returns a gzipped file. I have the data as an InputStream (courtesy of javax.activation.DataHandler.getInputStream();) from the response.
What I would like to do is, without writing anything to disk, get an InputStream of the decompressed data in the file that is in the archive. The compressed file in this case is an xml document that I am trying to unmarshal using javax.xml.bind.Unmarshaller which takes an InputStream.
I'm currently trying to write the InputStream to an OutputStream (decompressing the data) and then I'll need to write it back to an InputStream. It's not working yet so I thought I would see if there was a better (I would hope so) approach.
I can write the initial InputStream to disk and get a gz file, and then read that file, get the compressed file out of it and go from there but I'd rather keep it all in memory is possible.
Update 1: Here is my current (not working - get a "Not in GZIP format" exception):
ByteArrayInputStream xmlInput = null;
try {
InputStream in = dh.getInputStream(); //dh is a javax.activation.DataHandler
BufferedInputStream bis = new BufferedInputStream(in);
ByteArrayOutputStream bo = new ByteArrayOutputStream();
int bytes_read = 0;
byte[] dataBuf = new byte[4096];
while ((bytes_read = bis.read(dataBuf)) != -1) {
bo.write(dataBuf, 0, bytes_read);
}
ByteArrayInputStream bin = new ByteArrayInputStream(bo.toByteArray());
GZIPInputStream gzipInput = new GZIPInputStream(bin);
ByteArrayOutputStream out = new ByteArrayOutputStream();
dataBuf = new byte[4096];;
bytes_read = 0;
while ((bytes_read = gzipInput.read(dataBuf)) > 0) {
out.write(dataBuf, 0, bytes_read);
}
xmlInput = new ByteArrayInputStream(out.toByteArray());
If instead of writing to a ByteArrayOutputStream I write to a FileOutputStream the first time around I get a compressed file (which I can manually open to get the xml file within) and the service (eBay) says it should be a gzip file so I'm not sure why I get a "Not in GZIP format" error.
Update 2: I tried something a little different - same error ("Not in GZIP format"). Wow, I just tried to end that parenthesis with a semi-colon. Anyways, here is my second attempt, which still does not work:
ByteArrayInputStream xmlInput = null;
try {
GZIPInputStream gzipInput = new GZIPInputStream(dh.getInputStream());
ByteArrayOutputStream bo = new ByteArrayOutputStream();
int bytes_read = 0;
byte[] dataBuf = new byte[4096];
while ((bytes_read = gzipInput.read(dataBuf)) != -1) {
bo.write(dataBuf, 0, bytes_read);
}
xmlInput = new ByteArrayInputStream(bo.toByteArray());
Decorate the input stream with a GZIPInputStream.
InputStream decompressed = new GZIPInputStream(compressed);
The following code should work. Keep in mind you'll have to handle exceptions properly.
OutputStream out = null;
InputStream in = null;
try {
out = /* some output stream */;
in = new java.util.GZIPInputStream(/*some stream*/);
byte[] buffer = new byte[4096];
int c = 0;
while (( c = in.read(buffer, 0, 4096)) > 0) {
out.write(buffer, 0, c);
}
} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}
Take a look at GZIPInputStream. Here's an example; the class handles this very transparently, it's almost no work to use.

Categories

Resources