Develop Reference

The server is sending data through the socket, but the client is not receving it (Java)

I am writing a file storage and transfer system using Java. Here's the code on the client side to receive a file:
public static void receiveFile(Socket socket) throws IOException{
String fileLocation="/home/limafoxtrottango/Downloads/receivedFile";
int bytesRead=0;
int current = 0;
FileOutputStream fileOutputStream = null;
BufferedOutputStream bufferedOutputStream = null;
try {
// receive file
byte [] byteArray = new byte [60022386];
System.out.println("Waiting to receive a file...");
//reading file from socket
InputStream inputStream = socket.getInputStream();
fileOutputStream = new FileOutputStream(fileLocation);
bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
bytesRead = inputStream.read(byteArray,0,byteArray.length); //copying file from socket to byteArray
current = bytesRead;
do {
bytesRead =inputStream.read(byteArray, current, (byteArray.length-current));
if(bytesRead >= 0) current += bytesRead;
} while(bytesRead > -1);
bufferedOutputStream.write(byteArray, 0 , current); //writing byteArray to file
bufferedOutputStream.flush(); //flushing buffers
System.out.println("File " + fileLocation + " downloaded ( size: " + current + " bytes read)");
} catch(SocketException e){
System.out.println("Some error occured");
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
finally {
if (fileOutputStream != null) fileOutputStream.close();
if (bufferedOutputStream != null) bufferedOutputStream.close();
if (socket != null) socket.close();
}
}
While receiving a file, I get the following error:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:128)
at Test.receiveFile(Test.java:211)
at Test.main(Test.java:70)
Note: The error is in the following line of the code:
bufferedOutputStream.write(byteArray, 0 , current);
After debugging, I found-out that the client does not have any data in it's input stream, and hence, the read() method always returns -1 (eof). But the server is sending the file successfully.
Here is the code for the server:
public static void sendFile(Socket socket, String fileLocation)
{
FileInputStream fileInputStream = null;
BufferedInputStream bufferedInputStream = null;
OutputStream outputStream = null;
File file = new File (fileLocation);
byte [] byteArray = new byte [(int)file.length()];
try {
socket=new Socket(socket.getInetAddress(),port_no);
fileInputStream = new FileInputStream(file);
bufferedInputStream = new BufferedInputStream(fileInputStream);
bufferedInputStream.read(byteArray,0,byteArray.length); // copied file into byteArray
//sending file through socket
outputStream = socket.getOutputStream();
System.out.println("Sending " + fileLocation + "( size: " + byteArray.length + " bytes)");
outputStream.write(byteArray,0,byteArray.length); //copying byteArray to socket
outputStream.flush(); //flushing socket
System.out.println("Done sending!");
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
And here is my call to the above method:
sendFile(clientSocket, "/home/limafoxtrottango/Downloads/serverDownloads/"+sender);
The thing is that the server is successfully writing the byte into the stream, but the client doesn't seem to have any data in it's input stream.

https://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read(byte[],%20int,%20int)
inputStream.read(byteArray,0,byteArray.length); may return -1 in some cases as given in documentation above. Please cater for such situations.
In addition, I would suggest to use solution given here for both client and server: Efficient way to write InputStream to a File in Java (Version 6)
Client code:
final Path destination = Paths.get(fileLocation);
try (
final InputStream in = socket.getInputStream();
) {
Files.copy(in, destination);
}
Server code:
try (
final InputStream in = new FileInputStream(fileLocation);
) {
Files.copy(in, socket.getOutputStream());
}
Kind regards,
Bala

The server isn't sending anything, contrary to your title. It is closing the connection immediately, so bytesRead is initially -1 and never changes, and you aren't defending against that, so you get the ArrayIndexOutOfBoundsException.
However in the code you posted, the server is sending something but never closing the socket, which is another bug you need to fix. It is also ignoring the count returned by FileInputStream.read() and assuming it filled the buffer, which isn't part of the specification.
So either this is not the real server code or you are connecting to something else, or the server got an IOException that you haven't mentioned.
It's curious that you use two different pieces of code for copying. The standard way to copy a stream in Java is this:
char buffer = new char[8192]; // or whatever size you prefer > 0
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Use this at both ends. There is no need for heroically sized buffers, or buffers the size of the file, or assuming that the file size fits into an int.

OutOfMemoryException while decrypting file using facebook conceal

I'm working on android application where I need to save Videos in SD Card which must not be transferable that's why I'm encrypting and decrypting as and when needed with Facebook Conceal which works perfectly fine if video size is smaller.
Whenever I tries to do encryption and decryption to large video files not more than 10MB in GenyMotion running 2.3.7 it crashes with OutOfMemoryException which means I'm running out of heap memory allocated to my application which can't be handled but must be prevented.
Tried :
Apache Common Utils IO package
Various IO Utils
Facebook Conceal : Says while decrypting
You must read the entire stream to completion.
The verification is done at the end of the stream.
Thus not reading till the end of the stream will cause
a security bug. For safety, you should not
use any of the data until it's been fully read or throw
away the data if an exception occurs.
Code I'm invoking which encryption and decryption with Facebook Conceal :
Encryption :
public void startEncryption() {
// Creates a new Crypto object with default implementations of
// a key chain as well as native library.
// Check for whether the crypto functionality is available
// This might fail if android does not load libaries correctly.
if (!crypto.isAvailable()) {
return;
}
OutputStream fileStream;
try {
File mEncryptedFile = new File(mPlainFile.getPath().substring(0,
mPlainFile.getPath().length() - 4)
+ "_encrypted"
+ mPlainFile.getPath().substring(
mPlainFile.getPath().length() - 4,
mPlainFile.getPath().length()));
fileStream = new BufferedOutputStream(new FileOutputStream(
mEncryptedFile));
// Creates an output stream which encrypts the data as
// it is written to it and writes it out to the file.
OutputStream outputStream;
outputStream = crypto.getCipherOutputStream(fileStream, entity);
outputStream.write(FileUtils.readFileToByteArray(mPlainFile));
// fileStream.flush();
// fileStream.close();
// outputStream.flush();
// outputStream.close();
// outputStream.flush();
File mRenameTo = new File(mPlainFile.getPath());
mPlainFile.delete();
mEncryptedFile.renameTo(mRenameTo);
} catch (FileNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (CryptoInitializationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (KeyChainException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Decryption :
public String startDecryption() {
// Get the file to which ciphertext has been written.
try {
FileInputStream fileStream = new FileInputStream(mPlainFile);
// Creates an input stream which decrypts the data as
// it is read from it.
InputStream inputStream;
inputStream = crypto.getCipherInputStream(fileStream, entity);
ByteArrayOutputStream out = new ByteArrayOutputStream();
// org.apache.commons.io.output.ByteArrayOutputStream out = new
// org.apache.commons.io.output.ByteArrayOutputStream(1024);
// Read into a byte array.
// int read;
// byte[] buffer = new byte[1024];
// // You must read the entire stream to completion.
// // The verification is done at the end of the stream.
// // Thus not reading till the end of the stream will cause
// // a security bug.
// int i = 0;
// while ((read = inputStream.read(buffer)) != -1) {
// out.write(buffer, 0, read);
// Log.i(TAG, "bytearrayoutputstream "+i++ + " "+read + " " +
// buffer.length + " "+out.size());
// }
mDecryptedFile = new File(mPlainFile.getPath().substring(0,
mPlainFile.getPath().length() - 4)
+ "_decrypted"
+ (mPlainFile.getPath().substring(mPlainFile.getPath()
.length() - 4, mPlainFile.getPath().length())));
OutputStream outputStream = new FileOutputStream(mDecryptedFile);
// IOUtils.copy(inputStream, outputStream);
try {
final byte[] buffer = new byte[1024];
int read;
while ((read = inputStream.read(buffer)) != -1)
outputStream.write(buffer, 0, read);
outputStream.flush();
} catch (Exception e) {
} finally {
outputStream.close();
}
// out.writeTo(outputStream);
// out.flush();
// out.close();
// fileStream.close();
inputStream.close();
// outputStream.flush();
outputStream.close();
return mDecryptedFile.getPath();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (CryptoInitializationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (KeyChainException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
Is there any solution which can work around and encrypts and decrypts large video files too?

Looks like you are reading in the whole file into a byte array. What you should do is create an input stream and feed it to the crypto engine.
So this:
outputStream.write(FileUtils.readFileToByteArray(mPlainFile));
Should look like your decryption loop.
BufferedInputStream bis = new BufferedInputStream(mPlainFile);
InputStream inputStream = crypto.getCipherInputStream(
bis,
entity);
while ((read = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, read);
}
Not sure if this is what you ended up with. I pulled the crypto decoding from the docs. I think this confirms that their intent is to say the stream must be read until the end.

As Bruce did highlight on bottleneck on Encryption which leads to OutOfMemoryException at time of Decryption. So here's code which i'm executing while encrypting and decrypting which no more leads to OutOfMemoryException.
Encryption :
fileStream = new BufferedOutputStream(new FileOutputStream(mEncryptedFile));
OutputStream outputStream;
outputStream = crypto.getCipherOutputStream(fileStream, entity);
int read;
byte[] buffer = new byte[1024];
BufferedInputStream bis = new BufferedInputStream(newFileInputStream(mPlainFile));
while ((read = bis.read(buffer)) != -1) {
outputStream.write(buffer, 0, read);
}
outputStream.close();
bis.close();
Decryption :
InputStream inputStream;
inputStream = crypto.getCipherInputStream(fileStream, entity);
ByteArrayOutputStream out = new ByteArrayOutputStream();
OutputStream outputStream = new FileOutputStream(mDecryptedFile);
BufferedInputStream bis = new BufferedInputStream(inputStream);
int mRead;
byte[] mBuffer = new byte[1024];
while ((mRead = bis.read(mBuffer)) != -1) {
outputStream.write(mBuffer, 0, mRead);
}
bis.close();
out.writeTo(outputStream);
inputStream.close();
outputStream.close();
out.close();

OutOfMemory when creating Base64 string in java?

I used ostermillerutils library to create base64 string but I get OutOfMemory error if the image is heavy. If the image I try to convert is a simple image, the code is working fine.
public String createBase64String(InputStream in) {
//collect = new ByteArrayOutputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for(int readNum; (readNum = in.read(buf)) != -1; ) {
bos.write(buf, 0, readNum);
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
return null;
}
finally {
if (in != null) {
try {
in.close();
}
catch (IOException ex) {
;
}
}
}
byte[] ba = bos.toByteArray();
String coded = Base64.encodeToString(ba);
return coded;
}
I also tried doing this but the base64 was incorrect when I tried to decode it.
public void createBase64String(InputStream in) throws IOException {
//collect = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int readNum = 0;
try {
while((readNum = in.read(buf)) != -1)
{
smtp.addBase64(Base64.encodeBase64String(buf));
}
}
catch (IOException ex) {
Logger.getInstance().debug("XML createBase64String: IOException");
}
finally {
if (in != null) {
in.close();
}
}
}
Please suggest solutions for JDK 1.4 and also for later versions of Java.

If you like to write the encoded content straight into a file then use the following code
public void encode(File file, OutputStream base64OutputStream) {
InputStream is = new FileInputStream(file);
OutputStream out = new Base64OutputStream(base64OutputStream)
IOUtils.copy(is, out);
is.close();
out.close();
}
IOUtils class from Apache Commons IO.
EDIT
Since you want to do it using BufferedWriter, use it as follows
OutputStream out = Base64OutputStream(smtpSocket.getOutputStream());
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(out));
IOUtils.copy(is, bw);

It sounds like the problem is that you're having to manipulate too much data in memory when you read the entire image. One fix would be to increase the Java heap size until you have enough memory, but that would just be avoiding the problem rather than solving it.
A better option would be to look at a streaming implementation of a Base64 encoder. This would mean you're only working on a subset of the image at any time. I believe that Base64OutputStream from Apache Commons would do the job for you.

I've fixed my problem by using javabase64-1.3.1.jar library.
OutputStream fos2 = FileUtil.getOutputStream(base64FileName, FileUtil.HDD);
InputStream in2 = FileUtil.getInputStream(fileName, FileUtil.HDD);
Base64.encode(in2, fos2);
in2.close();
fos2.close();
I stored the base64 string to a text file first.
public void createBase64String(InputStream in) throws IOException {
baos = new ByteArrayOutputStream();
byte[] buf = new byte[BUFFER_SIZE];
int readNum = 0;
smtp.addBase64("\t\t");
try {
while ((readNum = in.read(buf)) >= 0) {
baos.write(buf, 0, readNum);
smtp.addBase64(baos.toString());
baos.reset();
}
}
catch (IOException ex) {
LogUtil.error("Sending of Base64 String to SMTP: IOException: " + ex);
}
finally {
if (in != null) {
in.close();
baos.close();
}
}
baos = null;
buf = null;
}
then send each line to smtp's socket outputstream.

From Java 8 onwards, there is a simple way to implement base64 encoding in an output stream with one line of code and no external dependencies:
import java.util.Base64;
OutputStream os = ...
OutputStream base64 = Base64.getEncoder().wrap(os);
Base64 also provides other flavors of base64 encoder; see javadocs:
Base64
Base64.Encoder.wrap

Weird stuff while viewing a file which was transferred with sockets in java

Well i am trying to transfer a file using sockets in java
Here is the code
Client Code
try{
// get streams
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
DataInputStream din = new DataInputStream (socket.getInputStream());
dos.writeUTF(fileName);
dos.flush();
boolean isOk = din.readBoolean();
if(!isOk){
throw new StocFileNotFound("Fisierul: " + fileName +" was not found on:" + address.toString());
} else {
baos = new ByteArrayOutputStream();
byte biti [] = new byte[1024];
while(din.read(biti,0,1024) != -1){
baos.write(biti,0,biti.length);
}
}
}
catch(IOException e){}
finally {
try{ socket.close(); } catch (IOException e){}
}
and then I return the baos.toByteArray() and write it to a file with the OutputStream`s write method.
Server code
try{
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
DataInputStream din = new DataInputStream (socket.getInputStream());
// check if it is really a file or if it is an existing file
File file = new File(din.readUTF());
// write false
if ( !file.exists() || !file.isFile() ){
dos.writeBoolean(false);
dos.flush();
}
// write true and write the file
else {
byte biti[] = new byte[1024];
dos.writeBoolean(true);
FileInputStream fis = new FileInputStream(file);
while(fis.read(biti,0,1024) != -1){
dos.write(biti,0,biti.length);
}
dos.flush();
try{ fis.close(); } catch (IOException e){}
}
} catch (IOException e){}
finally {
try{socket.close();}catch(IOException e){}
}
The problem
When i transfer a .txt file and view it in gedit it shows the text followed by multiple \00\00\00, though when i open it using notepad(in wine) it shows only the text. Plus viewing images and .doc works also. So is it something with gedit or is it with my program?
Edit
i was sending something like "hi, hope it works!"

This is the problem (or at least a problem):
while(fis.read(biti,0,1024) != -1)
{
dos.write(biti,0,biti.length);
}
You're always writing out the whole buffer, however many bytes were actually read. You should have:
int bytesRead;
while ((bytesRead = fis.read(biti, 0, 1024)) != -1)
{
dos.write(biti, 0, bytesRead);
}
(You've got the same problem in both bits of code.)
You might want to look at Guava which has various utility methods to relieve you of a lot of the tedium (and possible error) of writing this kind of code over and over again.

The read method will return the actual number of bytes read from the stream. You should use that as a parameter to your write method, or else you will be writing garbage to it.

Java Android Sockets

I'm currently working on an Android app which sends an string and a file to a java server app running on remote computer. This java server app should find the index on the file and send back the value of this index (The file structure is: index value. Example: 1 blue) The file is properly sent and received on the remote machine and I have a method which finds the value of the received index on the file. But when I'm trying to send the found value back to the phone I get an exception (closed socket), but I'm not closing the socket or any buffer. I'm not sure if the socket which is closed is the mobile app socket or the java server app socket. I'm using the same socket I use to send to receive (which is the way to work on Android). Sending the answer back to the phone is what my project is missing and is what I need help in. Here is my code:
Client app (Android app):
private class HeavyRemProcessing extends AsyncTask<String, Void, String>
{
protected String doInBackground(String... urls)
{
begins = System.currentTimeMillis();
remoteExecution();
ends= System.currentTimeMillis();
procTime=ends-begins;
aux= Long.toString(procTime);
return aux;
} //doInBackground() ends
protected void onPostExecute(String time)
{
textView1.setText("Result: "+result+". Processing Time: "+time+" milisecs");
}// onPostExecute ends
} //HeavyRemProcessing ends
public void executor(View view)
{
key="74FWEJ48DX4ZX8LQ";
HeavyRemProcessing task = new HeavyRemProcessing();
task.execute(new String[] { "????" });
} //executor() ends
public void remoteExecution()
{
// I have fixed IP and port I just deleted
String ip; //SERVER IP
int port; // SERVER PORT
try
{
cliSock = new Socket(ip, port);
file= new File("/mnt/sdcard/download/Test.txt");
long length = file.length();
byte[] bytes = new byte[(int) length];
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
BufferedOutputStream out = new BufferedOutputStream(cliSock.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(cliSock.getInputStream()));
int count;
key=key+"\r\n";
out.write(key.getBytes());
while ((count = bis.read(bytes)) > 0)
{
out.write(bytes, 0, count);
} //It works perfectly until here
//// PROBABLY HERE IS THE PROBLEM:
out.flush();
out.close();
fis.close();
bis.close();
result= in.readLine(); //RECEIVE A STRING FROM THE REMOTE PC
}catch(IOException ioe)
{
// Toast.makeText(getApplicationContext(),ioe.toString() + ioe.getMessage(),Toast.LENGTH_SHORT).show();
}
}catch(Exception exp)
{
//Toast.makeText(getApplicationContext(),exp.toString() + exp.getMessage(),Toast.LENGTH_SHORT).show();
}
} //remoteExecution ends
Java Server App (Remote PC)
public void receivingFile()
{
System.out.println("Executing Heavy Processing Thread (Port 8888).");
try
{
serverSocket = new ServerSocket(8888);
InputStream is = null;
OutputStream os= null;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
BufferedOutputStream boSock =null;
DataOutputStream dataOutputStream=null;
int bufferSize = 0;
try
{
socket = serverSocket.accept();
System.out.println("Heavy Processing Task Connection from ip: " + socket.getInetAddress());
} catch (Exception ex)
{
System.out.println("Can't accept client connection: "+ex);
}
try
{
is = socket.getInputStream();
dataOutputStream = new DataOutputStream(socket.getOutputStream());
bufferSize = socket.getReceiveBufferSize();
}
catch (IOException ex)
{
System.out.println("Can't get socket input stream. ");
}
try
{
fos = new FileOutputStream(path);
bos = new BufferedOutputStream(fos);
}
catch (FileNotFoundException ex)
{
System.out.println("File not found. ");
}
byte[] bytes = new byte[bufferSize];
int count;
System.out.println("Receiving Transfer File!.");
while ((count = is.read(bytes)) > 0)
{
bos.write(bytes, 0, count);
}
System.out.println("File Successfully Received!.");
fos.close();
bos.flush();
bos.close();
is.close();
result= obj.searchIndex();
System.out.println("Found: "+result); //This correctly print the found value
dataOutputStream.writeUTF(result);
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("Data sent back to the Android Client. ");
} catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
} // receivingFile() ends
Please if someone can help me I will really appreciate it. I'm thinking is something probably related with the buffers and the socket. My java server app throws an exception: "Closed Socket"... Thanks for your time,
Alberto.

I think your problem is that you closing the outputstream before closing the inputstream. This is a bug in android. Normally in java closing outputstream only flushes the data and closing inputstream causes the connection to be closed. But in android closing the outputstream closes the connection. That is why you are getting closed socket exception,
Put the statements
out.flush();
out.close();
after
result=in.readLine();
or just avoid those statements(out.flush and out.close). I had also faced a similar problem. See my question