Avro Communication over TCP Sockets - java

I am currently working on a project that will involve communication of applications written in C and Java. Therefore, I chose to work with Apache Avro. I have seen on the website that Avro can (de-)serialize objects from files using the DataFileWriter class.
But, in my case I want to use TCP sockets between my applications. Therefore, DataFileWriter class is not going to work for me. After reading the documentation, I have not found any information on how to send objects through TCP sockets.
Any ideas on how to do that? I specifically want to know what kind of Input and Output Streams I should use on the Java Clients.
I have developed the following code for the Java Server:
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.util.HashMap;
import middleman.bigpeer.BigPeer;
import org.apache.avro.generic.GenericDatumWriter;
import org.apache.avro.io.BinaryDecoder;
import org.apache.avro.io.BinaryEncoder;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DecoderFactory;
import org.apache.avro.io.EncoderFactory;
import org.apache.avro.specific.SpecificDatumReader;
import org.apache.avro.specific.SpecificDatumWriter;
public class MiddleManWorker implements Runnable {
private InputStream in;
private OutputStream out;
private Socket clientSocket;
public MiddleManWorker(Socket clientSocket, HashMap<Integer, NodeType> dbNodesDirectory,
HashMap<Integer, NodeType> workersDirectory) {
this.clientSocket = clientSocket;
try {
this.in = clientSocket.getInputStream();
this.out = clientSocket.getOutputStream();
} catch (IOException e) {
e.printStackTrace();
}
}
#Override
public void run() {
EncoderFactory encoderFactory = new EncoderFactory();
DecoderFactory decoderFactory = new DecoderFactory();
BinaryEncoder binaryEncoder = encoderFactory.binaryEncoder(out, null);
BinaryDecoder binaryDecoder = decoderFactory.binaryDecoder(in, null);
SpecificDatumReader<BigPeer> peerDatumReader = new SpecificDatumReader<BigPeer>(BigPeer.class);
BigPeer bigPeer = null;
SpecificDatumWriter<BigPeer> writer = new SpecificDatumWriter<BigPeer>();
try {
peerDatumReader.read(bigPeer, binaryDecoder);
System.out.println("Received: " + bigPeer.getType());
} catch (IOException e) {
e.printStackTrace();
}
try {
writer.write(bigPeer, binaryEncoder);
} catch (IOException e) {
e.printStackTrace();
}
}
}
A sample Java Client is the following:
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.Socket;
import middleman.bigpeer.BigPeer;
import org.apache.avro.io.BinaryDecoder;
import org.apache.avro.io.BinaryEncoder;
import org.apache.avro.io.DecoderFactory;
import org.apache.avro.io.EncoderFactory;
import org.apache.avro.specific.SpecificDatumReader;
import org.apache.avro.specific.SpecificDatumWriter;
public class SystemClient {
public static void connect(String serverIPAddress, Integer serverPort) throws IOException, ClassNotFoundException {
/**
* Create Connection with the server
*/
Socket socket = new Socket(serverIPAddress, serverPort);
InputStream in = socket.getInputStream();
OutputStream out = socket.getOutputStream();
EncoderFactory encoderFactory = new EncoderFactory();
DecoderFactory decoderFactory = new DecoderFactory();
BinaryEncoder binaryEncoder = encoderFactory.binaryEncoder(out, null);
BinaryDecoder binaryDecoder = decoderFactory.binaryDecoder(in, null);
BigPeer bigPeer = new BigPeer();
bigPeer.setType("test");
SpecificDatumReader<BigPeer> reader = new SpecificDatumReader<BigPeer>(BigPeer.class);
SpecificDatumWriter<BigPeer> writer = new SpecificDatumWriter<BigPeer>(BigPeer.class);
System.out.println("Before: " + bigPeer.getType());
writer.write(bigPeer, binaryEncoder);
System.out.println("Waiting for response...");
reader.read(bigPeer, binaryDecoder);
System.out.println("After: " + bigPeer.getType());
}
}
And the server seems to halt on the peerDatumReader.read(bigPeer, binaryDecoder); line of code. Any ideas?
Thank you,
Nick

BinaryEncoder uses an internal buffer for performance reasons. You may need to call flush on the encoder to send the data through the pipe.
See the reference for more information on this behaviour:
The BinaryEncoder implementation returned may buffer its output. Data may not appear on the underlying OutputStream until Flushable.flush() is called. The buffer size is configured with configureBufferSize(int).
If buffering is not desired, and lower performance is acceptable, use directBinaryEncoder(OutputStream, BinaryEncoder)

Related

java: Transfer a file to the server and get the file from the server in upper case

i want to send a .txt file from the client to server and get it back in upper case.
But this code do nothing.can anyone tell what is wrong here..?
SERVER : getting file from client and sending it back in upper case to the client.
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Arrays;
public class Assignment4_Server {
public static void main(String[] args) throws IOException {
byte[] bytearray = new byte[4096];
try (ServerSocket ss = new ServerSocket(4444)) {
Socket s = ss.accept();
InputStream is = s.getInputStream();
OutputStream os = s.getOutputStream();
int count;
String data = null ;
while((count = is.read(bytearray))>0){
data = Arrays.toString(bytearray).toUpperCase();
byte[] bytearrayout = data.getBytes();
os.write(bytearrayout);
}
s.close();
}
}
}
CLIENT : sending text.txt file to the server and getting file back after converted in upper case.
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
public class Assignment4_client {
public static void main(String[] args) throws IOException {
File file = new File("test.txt");
byte[] bytearray = new byte[4096];
Socket sc = new Socket("localhost",4444);
//send file
int countS , countR;
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
OutputStream os = sc.getOutputStream();
while((countS = bis.read(bytearray))>0){
os.write(bytearray);
}
//recieve file in uppercase from server
InputStream is = sc.getInputStream();
byte[] bytearray2 = new byte[4096];
FileOutputStream fos = new FileOutputStream(file);
BufferedOutputStream bos = new BufferedOutputStream(fos);
while((countR = is.read(bytearray2))>0){
bos.write(bytearray2);
}
}
}
Here is a code that should help you. But before reading it you should be aware of what is happening:
Your client is not sending a 'stop reading' information to the server (read the client code below). That's why the server is stuck in the while loop when it is trying to read the data sent by the client. That is probably why you have tried to send the data back directly to the client. Shut down the socket output from the client side to respect the Socket contract and correctly free the socket (see TCP/IP).
The solution given doesn't take in account that the server should stay up after it has done its duty. Then, the server will not be able to serve more than one client at a time. This server is offering a one time service, which is pointless. To overcome this issue you should put everything in a while loop and bind every new server process into a new thread (I let you do that, its quite a joy).
The server doesn't take in account the whole size of the data an it could possibly run into an out of memory error if the data is too heavy. You should find a way to avoid this problem in a real implementation.
Both program should catch the exception and log it somewhere so you could be aware of any errors.
Writing a server is not so simple. You should normally write some kind of protocol with headers and other stuff like that. To avoid that, use objects like ObjectOutputStream and ObjectInputStream but it has some limitation like constraining your server in the Java world.
CLIENT
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.net.Socket;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;
public class Client {
public void send(File file)
{
Socket sc = null;
try
{
byte[] bytearray = new byte[4096];
sc = new Socket("localhost", 4444);
// 1. Read the file, send its content, close it.
int count;
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
OutputStream os = sc.getOutputStream();
while((count = bis.read(bytearray))>0)
{
os.write(bytearray);
}
fis.close();
sc.shutdownOutput();
// 2. Delete old file, receive data, write it to new File.
InputStream is = sc.getInputStream();
bytearray = new byte[4096];
// Eventually do what you want with the file: new one, append, etc.
file.delete();
file.createNewFile();
FileOutputStream fos = new FileOutputStream(file);
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(file));
count = 0;
while((count = is.read(bytearray)) > 0)
{
bos.write(bytearray, 0, count);
}
fos.close();
bos.close();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
if (sc != null)
{
try
{
sc.close();
} catch (IOException e) {}
}
}
}
}
SERVER
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
public class Server
{
Server()
{
Socket s = null;
byte[] bytearray = new byte[4096];
try (ServerSocket ss = new ServerSocket(4444))
{
s = ss.accept();
InputStream is = s.getInputStream();
// 1. Recieve data and put it to UpperCase.
String data = "";
int count;
while((count = is.read(bytearray)) > 0)
{
data += new String(bytearray, 0, count);
}
data = data.toUpperCase();
System.out.println(data);
// 2. Send back data.
OutputStream os = s.getOutputStream();
os.write(data.getBytes());
os.close();
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
TEST PROGRAM
This one should help you to test your both programs in the same project in an IDE.
import java.io.File;
public class Test
{
public static void main(String[] args)
{
Client c = new Client();
(new Thread()
{
public void run()
{
Server s = new Server();
}
}).start();
c.send(new File("test.txt"));
}
}
What is wrong here is two simple things.
The server is reading until end of stream, on a socket that must be used for the reply. The client therefore cannot close it after sending the request to provide the EOS, so it must shutdown the socket for output after sending the request.
Your copy loops are wrong. The general form is:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
You are ignoring the read count when writing, so you will write junk at end of stream, or any other time that read() doesn't fill buffer, which can be any time at all.

outputstream does not write the data as expected

My server looks like this:
package marshexample;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.Reader;
import java.net.ServerSocket;
import java.net.Socket;
public class marshServer {
private ServerSocket ses;
private Reader br;
private OutputStream os;
public marshServer() {
StringBuilder sb = new StringBuilder();
try {
ses = new ServerSocket(7824);
Socket s = ses.accept();
br = new InputStreamReader(s.getInputStream());
char[] request = new char[6];
int count = br.read(request);
while (!sb.toString().contains("project")) {
sb.append(new String(request, 0, count));
count = br.read(request);
System.out.println(sb.toString());
}
System.out.println(sb);
os = s.getOutputStream();
os.write("string from server".getBytes());
os.write("empty line".getBytes());
os.flush();
} catch (Exception e) {
e.printStackTrace();
}
System.out.println(sb);
}}
The issue is I have to add the string os.write("empty line".getBytes()); in order to make it work correctly. Without this string the massage is not fully sent to the client. (The same situation is with client). So why flush method does not solve this problem? Thank you for any ideas!

Java socket communication

I have a problem with my program.
For explain it I would use an example:
I have two programs, A and B.
A controls B with sockets, A have the client and B the server.
The A client code:
package clientTimer;
import java.io.*;
import java.net.*;
import javax.swing.JOptionPane;
import java.net.*;
import java.io.*;
public class Client {
private String action;
private String ip;
public Client(String action,String ip) {
this.action=action;
this.ip=ip;
String serverName = "server";
int port = 9999;
try {
Socket client = new Socket(ip, port);
OutputStream outToServer = client.getOutputStream();
DataOutputStream out = new DataOutputStream(outToServer);
out.writeUTF(action);
InputStream inFromServer = client.getInputStream();
DataInputStream in =new DataInputStream(inFromServer);
new Countdown(in.readUTF(),true);
client.close();
}catch(IOException e) {
e.printStackTrace();
}
}
}
The B server code:
package serverTimer;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketException;
import java.net.*;
import java.io.*;
public class Server extends Thread {
private String correct;
public Server() {
run();
}
public void run() {
correct="q";
try {
ServerSocket serverSocket = new ServerSocket(9999);
while(true) {
Socket server = serverSocket.accept();
DataInputStream in = new DataInputStream(server.getInputStream());
new Viewer(in.readUTF());
DataOutputStream out = new DataOutputStream(server.getOutputStream());
if (correct.length()==5) {
out.writeUTF(correct);
} else {
out.writeUTF("a");
}
server.close();
}
}catch(SocketTimeoutException s) {
System.out.println("Socket timed out!");
}catch(IOException e) {
e.printStackTrace();
}
}
}
As you can see, the client send and receive a string, the same for the server.
My problem is: the server must send a toString() method (in the code the string is called "correct") of an object in the same package to client.
I don't know how I can do it.
The object with the toString method is initialized in another file of the package (the Viewer(String)).
Thanks.
edit:
I'm sorry, is a little bit difficult for me explain it.
In a simple scheme what I need is:
A says "do something" to -> B says "I do that" to -> A
or
A outputstream-> B inputstream - B outputstream -> A inputstream.
A have problems with the B outputstream -> A inputstream.
The outputstream must send a string product by a toString() method of another class (in the B package).
The program is a timer controlled remotely, the A package have the controller and the B package have the viewer.
I send the A commands to B via socket, but I need that B "occasionally" send to A a "report" with the current time.
I can't write all the program here, it's too "big".
If you want I can send you an email with all the file.

Java socket reading and writeing hex code

I'm working on a project and i want to comunicate with a device. I made a socket connection with the device, the connection works but the device is sending me the message: 0xd7d0 and i have to write that message back. It's a keep alive message. I'm haveing trouble reading and sending back that message.
here's the code i've writen so far:
package Server;
import java.io.IOException;
import java.net.ServerSocket;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Server {
public static ServerSocket serverSocket;
public static void main (String [] args) {
try {
serverSocket = new ServerSocket(1234);
while (true) {
ServerThread serverThread = new ServerThread(serverSocket.accept());
serverThread.start();
}
} catch (IOException ex) {
Logger.getLogger(Server.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
package Server;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.util.logging.Level;
import java.util.logging.Logger;
public class ServerThread extends Thread {
public Socket socket;
public BufferedReader in;
public PrintWriter out;
public ByteBuffer buf;
int count;
public ServerThread (Socket socket) {
try {
this.socket = socket;
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out = new PrintWriter(socket.getOutputStream(), true);
} catch (IOException ex) {
Logger.getLogger(ServerThread.class.getName()).log(Level.SEVERE, null, ex);
}
}
#Override
public void run() {
while (true) {
try {
count = in.read();
buf = ByteBuffer.allocate(100);
buf.put((byte) count);
buf.flip();
out.println(buf);
String line = in.readLine();
System.out.println(line);
} catch (IOException ex) {
Logger.getLogger(ServerThread.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
Can anyone tell me what i am doing wrong?
This
while (true) {
ServerThread serverThread = new ServerThread(serverSocket.accept());
serverThread.start();
}
doesn't look right. You're looping and creating a new thread repeatedly (which will consume resources and create an enormous number of threads). You should simply create that thread once. If your program is doing nothing else then a new thread may be superfluous.
Your mistake is that you mix different styles of network data exchange. First, what format are messages in? Are they characters in some encoding, or some binary data? In the first case, you should not use Buffer to read message in, but read and write using in and out character streams you created already. In the second case, you have 2 options: read and write with socket and byte streams, or with channels from java.nio.channels and Buffers. To write back a message in a buffer, you can use
buf.flip();
buffer.position(buffer.limit());

java.io.EOFException error on object serialization with HttpHandler

I am trying to serialize an object in a HttpHandler class.
I have 2 files, Server3.java:
package server3;
import java.io.IOException;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.io.Serializable;
import java.net.InetSocketAddress;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
public class Server3 {
public static void main(String[] args) throws Exception {
HttpServer server = HttpServer.create(new InetSocketAddress(3333), 0);
server.createContext("/", new MyHandler());
server.setExecutor(null); // creates a default executor
server.start();
}
static class MyHandler implements HttpHandler {
public void handle(HttpExchange t) throws IOException {
String response = "Kjo eshte nje pergjigje nga serveri! n";
t.sendResponseHeaders(200, response.length());
OutputStream os = t.getResponseBody();
os.write(response.getBytes());
os.close();
Personat obj = new Personat();
ObjectOutputStream objOut = new ObjectOutputStream(t.getResponseBody());
objOut.writeObject(obj);
objOut.close();
}
}
}
class Personat implements Serializable{
private static final long serialVersionUID = 1L;
int ID=3;
String Name="Andi";
}
and Client3.java:
package server3;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.ObjectInputStream;
import java.net.HttpURLConnection;
import java.net.URL;
//te gjithe personat qe jan ne database me nej objekt
public class Client3 {
public static void main(String[] args) throws Exception {
try {
URL url = new URL("http://localhost:3333");
HttpURLConnection s = (HttpURLConnection) url.openConnection();
s.setDoOutput(true);
s.setDoInput(true);
s.setRequestMethod("POST");
s.setUseCaches(false);
InputStream in = s.getInputStream();
InputStreamReader isr = new InputStreamReader(in);
BufferedReader br = new BufferedReader(isr);
int c;
while ((c = br.read()) != -1) {
System.out.print((char) c);
}
ObjectInputStream ios = new ObjectInputStream(s.getInputStream());
Personat oin = (Personat) ios.readObject();
String emri=oin.Name;
System.out.println(emri);
ios.close();
s.disconnect();
} catch (IOException ex) {
System.err.println(ex);
System.out.print(ex);
}
}
}
But when I run it eclipse shows me
java.io.EOFException Kjo eshte nje pergjigje nga serveri! njava.io.EOFException`
and I cant understand why.
The problem is that you are trying to fit both the string response and the object into response.length() bytes. What happens is that only response.length() bytes are sent and so if you try to read more you get the EOFException.
If you instead set the responseLength parameter to be 0 it will allow you to transmit an arbitrary amount of data
t.sendResponseHeaders(200, 0);
You also shouldn't close the stream if you are going to write more data into it. Don't call os.close() until all the writing is complete.

Categories

Resources