I'm trying to read a binary file from a URLConnection. When I test it with a text file it seems to work fine but for binary files it doesn't. I'm using the following mime-type on the server when the file is send out:
application/octet-stream
But so far nothing seems to work. This is the code that I use to receive the file:
file = File.createTempFile( "tempfile", ".bin");
file.deleteOnExit();
URL url = new URL( "http://somedomain.com/image.gif" );
URLConnection connection = url.openConnection();
BufferedReader input = new BufferedReader( new InputStreamReader( connection.getInputStream() ) );
Writer writer = new OutputStreamWriter( new FileOutputStream( file ) );
int c;
while( ( c = input.read() ) != -1 ) {
writer.write( (char)c );
}
writer.close();
input.close();
This is how I do it,
input = connection.getInputStream();
byte[] buffer = new byte[4096];
int n;
OutputStream output = new FileOutputStream( file );
while ((n = input.read(buffer)) != -1)
{
output.write(buffer, 0, n);
}
output.close();
If you are trying to read a binary stream, you should NOT wrap the InputStream in a Reader of any kind. Read the data into a byte array buffer using the InputStream.read(byte[], int, int) method. Then write from the buffer to a FileOutputStream.
The way you are currently reading/writing the file will convert it into "characters" and back to bytes using your platform's default character encoding. This is liable to mangle binary data.
(There is a charset (LATIN-1) that provides a 1-to-1 lossless mapping between bytes and a subset of the char value-space. However this is a bad idea even when the mapping works. You will be translating / copying the binary data from byte[] to char[] and back again ... which achieves nothing in this context.)
Related
what i did so far :
I read a file1 with text, XORed the bytes with a key and wrote it back to another file2.
My problem: I read for example 'H' from file1 , the byte value is 72;
72 XOR -32 = -88
Now i wrote -88 in to the file2.
when i read file2 i should get -88 as first byte, but i get -3.
public byte[] readInput(String File) throws IOException {
Path path = Paths.get(File);
byte[] data = Files.readAllBytes(path);
byte[]x=new byte[data.length ];
FileInputStream fis = new FileInputStream(File);
InputStreamReader isr = new InputStreamReader(fis);//utf8
Reader in = new BufferedReader(isr);
int ch;
int s = 0;
while ((ch = in.read()) > -1) {// read till EOF
x[s] = (byte) (ch);
}
in.close();
return x;
}
public void writeOutput(byte encrypted [],String file) {
try {
FileOutputStream fos = new FileOutputStream(file);
Writer out = new OutputStreamWriter(fos,"UTF-8");//utf8
String s = new String(encrypted, "UTF-8");
out.write(s);
out.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
public byte[]DNcryption(byte[]key,byte[] mssg){
if(mssg.length==key.length)
{
byte[] encryptedBytes= new byte[key.length];
for(int i=0;i<key.length;i++)
{
encryptedBytes[i]=Byte.valueOf((byte)(mssg[i]^key[i]));//XOR
}
return encryptedBytes;
}
else
{
return null;
}
}
You're not reading the file as bytes - you're reading it as characters. The encrypted data isn't valid UTF-8-encoded text, so you shouldn't try to read it as such.
Likewise, you shouldn't be writing arbitrary byte arrays as if they're UTF-8-encoded text.
Basically, your methods have signatures accepting or returning arbitrary binary data - don't use Writer or Reader classes at all. Just write the data straight to the stream. (And don't swallow the exception, either - do you really want to continue if you've failed to write important data?)
I would actually remove both your readInput and writeOutput methods entirely. Instead, use Files.readAllBytes and Files.write.
In writeOutput method you convert encrypted byte array into UTF-8 String which changes the actual bytes you are writing later to the file. Try this code snippet to see what is happening when you try to convert byte array with negative values to UTF-8 String:
final String s = new String(new byte[]{-1}, "UTF-8");
System.out.println(Arrays.toString(s.getBytes("UTF-8")));
It will print something like [-17, -65, -67]. Try using OutputStream to write bytes to the file.
new FileOutputStream(file).write(encrypted);
I am trying to read a UTF-8 file from a zipFile and its turning out to be a major challenge.
Here I zip the String to a bytes array to persist to my db.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ZipOutputStream zo = new ZipOutputStream( bos );
zo.setLevel(9);
BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(bos, Charset.forName("utf-8"))
);
ZipEntry ze = new ZipEntry("data");
zo.putNextEntry(ze);
zo.write( s.getBytes() );
zo.close();
writer.close();
return bos.toByteArray();
And this is how I read the String back:
ZipInputStream zis = new ZipInputStream( new ByteArrayInputStream(bytes) );
ZipEntry entry = zis.getNextEntry();
byte[] buffer = new byte[2048];
ByteArrayOutputStream bos = new ByteArrayOutputStream();
int size;
while ((size = zis.read(buffer, 0, buffer.length)) != -1) {
bos.write(buffer, 0, size);
}
BufferedReader r = new BufferedReader( new InputStreamReader( new ByteArrayInputStream( bos.toByteArray() ), Charset.forName("utf-8") ) );
StringBuilder b = new StringBuilder();
while (r.ready()) {
b.append( r.readLine() ).append(" ");
}
The String that I get back here has lost the UTF8 charecters!
UPDATE 1:
I changed the code around so that I compared the byte array of the original String with the byte array I read back from the zipfile and they freaking match! So its probably how I'm building the string after i have the bytes.
Arrays.equals(converted, orgi)
Your problem is in the writing, presuming s is a String, you have:
zo.write( s.getBytes() );
But that will convert s to bytes using whatever the default encoding is. You'll want to use UTF-8 for that conversion:
zo.write( s.getBytes("utf-8") );
Your observation that the original bytes are the same as the uncompressed bytes make sense because the original written data is the source of the problem.
Note that you have the writer stream declared but you never actually use it for anything (nor should you, in this context, since writing to it will just write uncompressed string data to the same stream bos that your ZipOutputStream writes to). It looks like you may have confused yourself trying a few different things at once here, you should just get rid of writer.
For one, BufferedReader#ready() is not a good indicator for reading input. Here's a number of reasons why
Does BufferedReader.ready() method ensure that readLine() method does not return NULL?
BufferedReader not stating 'ready' when it should
Second, you are using
b.append( r.readLine() ).append(" ");
which is always adding a " " on every iteration. The resulting String value is bound to be different than the original just because of this.
Third, shout out to Jason C about your BufferedWriter not doing anything.
I want to download a file from a URL and store it into the file system. However I have memory limitation and I don't want to store it in the memory before. I am not a java expert and I am a bit lost with all the class InputStream, BufferedReader, FileOutputStream, etc. Could you help me please ?
For now I have:
URLConnection ucon = url.openConnection();
ucon.connect();
InputStream is = ucon.getInputStream();
// Create a reader for the input stream.
BufferedReader br = new BufferedReader(isr);
// ?
FileOutputStream fos = context.openFileOutput(FILENAME, Context.MODE_PRIVATE);
// Here the content can be too big for the memory...
fos.write(content.getBytes());
fos.close();
Please, could you give me some clue ? I was thinking also to read it chunk by chunk, but I am not sure what would be the easiest with java...
you can use apache commons
org.apache.commons.io.FileUtils.copyURLToFile(URL, File)
I guess it may not work on android
I use this code
InputStream input = connection.getInputStream();
byte[] buffer = new byte[4096];
int cnt = - 1;
OutputStream output = new FileOutputStream(file);
while ( (cnt = input.read(buffer)) != -1)
{
output.write(buffer, 0, cnt);
}
output.close();
Getting data onto inputStream object from web url
inputStream = AWSFileUtil.getInputStream(
AWSConnectionUtil.getS3Object(null),
"cdn.generalsentiment.com", filePath);
If they are mutliple files then i want to zip them and sent the filetype as "zip" to struts.xml which does the download.
actually am converting the inputstream into byteArrayInputStream
ByteArrayInputStream byteArrayInputStream = new
ByteArrayInputStream(inputStream.toString().getBytes());
while (byteArrayInputStream.read(inputStream.toString().getBytes()) > 0) {
zipOutputStream.write(inputStream.toString().getBytes());
}
and then
zipOutputStream.close();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
fileInputStream = new FileInputStream(file);
while (fileInputStream.read(buffer) > 0) {
byteArrayOutputStream.write(buffer);
}
byteArrayOutputStream.close();
inputStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
reportName = "GS_MediaValue_Reports.zip";
fileType = "zip";
}
return fileType;
But the downloaded zip when extracted gives corrupt files.
Please suggest me a solution for this issue.
The short answer is that it's not how ZipOutputStream works. Since it was designed to store multiple files, along with their file names, directory structures and so on, you need to tell the stream about that explicitly.
Furthermore, converting a stream to a string is a bad idea in general, plus it's slow, especially when you're doing it in a loop.
So your solution will be something like:
ZipEntry entry = new ZipEntry( fileName ); // You have to give each entry a different filename
zipOutputStream.putNextEntry( entry );
byte buffer[] = new byte[ 1024 ]; // 1024 is the buffer size here, but it could be anything really
int count;
while( (count = inputStream.read( buffer, 0, 1024 ) ) != -1 ) {
zipOutputStream.write( buffer, 0, count );
}
I have a simple question. I'm trying to upload a file to my ftp server in Java.
I have a file on my computer, and I want to make a copy of that file and upload it. I tried manually writing each byte of the file to the output stream, but that doesn't work for complicated files, like zip files or pdf files.
File file = some file on my computer;
String name = file.getName();
URL url = new URL("ftp://user:password#domain.com/" + name +";type=i");
URLConnection urlc = url.openConnection();
OutputStream os = urlc.getOutputStream();
//then what do I do?
Just for kicks, here is what I tried to do:
OutputStream os = urlc.getOutputStream();
BufferedReader br = new BufferedReader(new FileReader(file));
String line = br.readLine();
while(line != null && (!line.equals(""))) {
os.write(line.getBytes());
os.write("\n".getBytes());
line = br.readLine();
}
os.close();
For example, when I do this with a pdf and then try and open the pdf that I run with this program, it says an error occurred when trying to open the pdf. I'm guessing because I am writing a "\n" to the file? How do I copy the file without doing this?
Do not use any of the Reader or Writer classes when you're trying to copy the byte-for-byte exact contents of a binary file. Use these only for plain text! Instead, use the InputStream and OutputStream classes; they do not interpret the data at all, while the Reader and Writer classes interpret the data as characters. For example
OutputStream os = urlc.getOutputStream();
FileInputStreamReader fis = new FileInputStream(file);
byte[] buffer = new byte[1000];
int count = 0;
while((count = fis.read(buffer)) > 0) {
os.write(buffer, 0, count);
}
Whether your URLConnection usage is correct here, I don't know; using Apache Commons FTP (as suggested elsewhere) would be an excellent idea. Regardless, this would be the way to read the file.
Use a BufferedInputStream to read and BufferedOutputStream to write. Take a look at this post: http://www.ajaxapp.com/2009/02/21/a-simple-java-ftp-connection-file-download-and-upload/
InputStream is = new FileInputStream(localfilename);
BufferedInputStream bis = new BufferedInputStream(is);
OutputStream os =m_client.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
byte[] buffer = new byte[1024];
int readCount;
while( (readCount = bis.read(buffer)) > 0) {
bos.write(buffer, 0, readCount);
}
bos.close();
FTP usually opens another connection for data transfer.
So I am not convinced that this approach with URLConnection is going
to work.
I highly recommend that you use specialized ftp client. Apache commons
may have one.
Check this out
http://commons.apache.org/net/api/org/apache/commons/net/ftp/FTPClient.html