I am working on a chat implementation with Java sockets. I have focused on few functionalities, like authentication, one person chat and a group chat. I was thinking about adding file transfer functionality, and I wonder what's the good practice about this. Should I have separate socket on the server with different port listening just for file transfers? Right now input and output streams that I get from server socket are binded to Scanner and PrintWriter objects respectively, so I find it hard to use that for file transfer.
Any patterns you guys could recommend or give me good recommendations are very appreciated.
Thanks,
ZeKoU
Well, FTP creates a new socket for each file transfer. Note that the connection can be established on the same port as the one used for chat, but with a different initialization dialog.
If you can use a native lib do that, it will be faster. Failing that unless there is a really good reason use a library which will have had all the kinks worked out.
If you really need to send a file in pure Java. Here's one way based on an old project of mine. Note however that there is some serialisation overhead however much of this could be removed via use of the NIO equivalents as discussed here or you could go slightly further and use Zero Copy which lets you tell the OS to copy a file direct to a socket without any intermediary copies. This only works with files though not other data.
You should do this in a separate thread so that the chat still works while the file is transferring. Create a socket and give it some standard port (you can give it port 0 which just chooses the next available port but then you will need to send that information to the other recipient so just using a standard port is easier). Then split your file into blocks and transmit it using the socket:
//Send file (Server)
//Put this in a thread
Socket socket = new Socket(destinationIP, destinationPort);
ObjectOutputStream sender = new ObjectOutputStream(socket.getOutputStream());
sender.writeObject(dataToSend);
//Receive File (Client)
//Kick off a new thread to receive the file to preserve the liveness of the program
ServerSocket serverSocket = new ServerSocket(ListenPort);
socket = new Socket();
while (true) {
new Thread(new TCPReceiver(socket)).start();
}
//Receive file thread TCPReceiver(Socket socket)
//Get the stream where the object is to be sent
ObjectInputStream objectInputStream = new ObjectInputStream(socket.getInputStream());
/* Where type is the type of data you send... String or anything really...
read part of the file into something and send it then at this end use the same data
type to recieve it and it will magically pull the entire object across.
*/
while(fileIsIncomplete){
type recievedData = (type) objectInputStream.readObject();
//Reconstruct file
}
Hope that is enough for you to get a quick file sender up and running :)
Edit: Removed nonsense statement. Added native point and zero copy mention.
Related
I am trying to set up a program where the server can communicate with multiple clients. The program is written in Java. I got it all working on the same machine so I decided to try LAN. I converted the program to JAR files and I tried connecting my laptop to my PC (both are on the same network). The connection is successful but unfortunately only 1 message arrives to the server. As you can see in the code below, I send multiple messages (Meaning that i write multiple times) via DataOutputStream. One defines the datatype (in the following example 0 means that it's a String) and the other sends the actual message data. I also print the size of the packets in bytes and it always matches the size of the DataOutputStream instance.
DataOutputStream dOut = new DataOutputStream(clientSocket.getOutputStream());
String str = "Hello";
//Type
System.out.println("Type output size: 1");
dOut.writeByte(0);
//Message
System.out.println("Message output size: " + (str.getBytes(StandardCharsets.UTF_8).length + 2));
dOut.writeUTF(str);
System.out.println("Length of all: " + (dOut.size()));
dOut.flush();
So now when the data from the client is sent we need to handle it on the server, which the code below does. It retrieves the InputStream from the Socket called client and inserts it into the DataInputStream. This is were it gets weird on LAN as the stream only contains the first message.
InputStream stream = client.getInputStream();
DataInputStream dIn = new DataInputStream(stream);
while(dIn.available() > 0) {
byte type = dIn.readByte();
switch(type) {
case 0:
System.out.println(dIn.readUTF());
break;
case 1:
System.out.println(dIn.readInt());
break;
case 2:
System.out.println(dIn.readByte());
break;
default:
throw new IllegalStateException("Unexpected value: " + type);
}
}
If you run the Client in the IDE on lets say a laptop connected to the same network and then you run the Server on a PC connected to the same network it will work. However, not if the programs are in JARS.
The actual stacktrace is the following:
java.net.SocketException: Connection reset
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:961)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:393)
The stacktrace does not tell me anything, but it points at case 0: in the switch case. It can't read the String as the DataInputStream does not contain any data (I guess?).
I would also like to state that the Server is multithreaded! I have one thread that adds the sockets when they are accepted through ServerSocket.accept() and I use the second (main thread) to read the data sent from clients.
I have selected the code above as I believe that the issue lies within it, however I am new to Socket Programming and I know that some of you would like to see other parts of the code. I will add more relevant code when I am asked.
I do not know why it acts like this, does anyone know why?
What have i tried?
I have tried waiting for packets - but that has only resulted in the
Server looping forever. With waiting for packets I mean not going forward until the DataInputStream contains enough bytes.
I have disabled Nagels Algorithm through setTCPNoDelay(false).
Tried to send different datatypes, but that also failed
I tried changing the first packet to a String which resulted in the String showing up in the DataInputStream.
I have tried portforwarding the port used and I have tried disabling the firewall on both computers.
Update 1
I have been taking advice from the comments which has led a to a few discoveries:
Closing the DataOutputStream successfully sends all packets to the client.
It is also possible to build your own buffer and decode it in the server. However, it is still not possible to send any more messages after this.
It worked as a JAR because IntelliJ was being nice (Eclipse threw the same error when running in IDE)
Update 2:
I think this post is relevant. It states that SocketException is sent when a client closes it's socket "ungracefully". And because my Client closes (as it is not in a loop) and I don't close the socket properly - it will close "ungracefully" and the data will be lost. Hence the error.
The issue is now solved, and the solution is quite logical. My client does not operate in a loop, rather it sends the data and closes the program. That sounds fine, but I forgot to properly close the socket of the client.
The reason why the second 'packet' never arrived was due to me doing this tiny mistake. The packet was on it's way through the local network but the client socket improperly closed it's socket before the packet arrived to the server, which is why I got a SocketException error. See this.
I solved the issue by putting socket.close() ,where socket is the client's socket, after I had sent all the messages I wanted to send.
Take it easy with me, I'm new in socket programming.
I'm about making a program which is similar to Teamviewer. I could make a simple Server/client application which is multithreaded. But the problem is I couldn't figure out how to do something like :
Let's say I have Server and a client connected to server.
I could transfer a file from client to server. I could make a simple chat with a client. But my problem is : How can I chat with the client while file is transferring in the same time? I mean I couldn't make more than one function in a time.Because we have just one input and one output for both server and client. So how could I send more that one function to client and how could the client read more that one function in a time and respond to requests ?
I did something like : I sent a request to client and I got the response in a new thread which contains a new DataInputStream but I couldn't figure out how the main DataInputStream will receive a new response from the client because what is receiving the requests in this way is the new DataInputStream. I'm really lost in this situation because I feel my concept is completely wrong but I couldn't figure out the right concept to do something like that.
Is it possible with ServerSocket or should I look at NIO Socket ?
NOTE :
I don't want a piece of code, I would like to understand the concept of the whole operation for something like that. Thank you
Use a packet based massaging system over a single connection
Outgoing
{"type":"chat", "message":"hi"}
{"type":"xfer", "fileName":"fileX", "data":"some_binary_data"}
Incoming
switch(getString("type")){
case "chat":
System.out.println("User said" + getString("message"));
break;
case "xfer":
File f = new File(getString("fileName"));
f.write(getString("data"));
break;
...
}
Obviously if the binary file is fairly large, you should break it into many different messages and reassemble it on the other side, this would allow chat messages to make their way across the wire while the transfer is also still taking place.
My app can transfer files and messages between server and client. Server is multithreaded and clients simply connects to it. While file is being transferred, if sender sends a message, it will be consumed as bytes of file.
I don't want to open more ports,
Can I establish a new connection to the server for file transfer? Or I
should open a separate port for files.
I don't want to block communication while a file is being transferred.
The question was marked as a duplicate but its not, i am trying to send messages and files simultaneously not one by one. I can already receive files one by one. Read again.
Also, as server is multithreaded, I cannot call server socket.accept() again to receive files in new connection because main thread listening for incoming will try to handle it instead. Is there a way around?
Seems to me like trying to multiplex files and messages onto the same socket stream is an XYProblem.
I am not an expert on this, but it sounds like you should do some reading on "ports vs sockets". My understanding is that ip:port is the address of the listening service. Once a client connects, the server will open a socket to actually do the communication.
The trick is that every time a client connects, spawn a new thread (on a new socket) to handle the request. This instantly frees up the main thread to go back to listening for new connections. Your file transfer and your messages can come into the same port, but each new request will get its own socket and its own server thread --> no collision!
See this question for a java implementation:
Multithreading Socket communication Client/Server
you could use some system of all the lines of a file start with a string like this (file:linenum) and then on the other side it puts that in a file then to send text you could do the same thing but with a tag like (text)
Server:
Scanner in = new Scanner(s.getInputStream());
while(true) {
String message = in.nextLine();
if(message.length > 14 && message.substring(0,6).equalsIgnoreCase("(file:") {
int line = Integer.valueOf(message.substring(6).replaceall(")", ""));
saveToFile(message.substring(6).replaceAll(")","").replaceAll("<1-9>",""));
} else {
System.out.println(message);
}
}
I think that code works but I haven't checked it so it might need some slight modifications
You could introduce a handshake protocol where clients can state who they are (probably happening already) and what they want from the given connection. The first connection they make could be about control, and perhaps the messages, and remain in use all the time. File transfer could happen via secondary connections, which may come and go during a session. Having several parallel connections between a client and a server is completely normal, that is what #MikeOunsworth was explaining too.
A shortcut you can take is issuing short-living, one-time tokens which clients can present when opening the secondary connection and then the server will immediately know which file it should start sending. Note that this approach easily can raise various security (if token encodes actual request data) and/or scalability issues (if token is something completely random and has to be looked up in some table).
I have Java Client-Server application I built using standard java.io (Socket and ServerSocket). In both server and client, I use DataInputStream to read the message using readUTF and DataOutputStream to write the message using writeUTF. I use thread-per-connection architecture for this application.
I have an additional feature that I need to add now, which is allowing a client to have a persistent connection open to the server and always listening to what server will write. I found out, with my current thread-per-connection architecture, my application won't scale because it holds the thread when the client opens the persistent connection.
I did some research and think to refactor my server to use java.nio (SocketChannel and ServerSocketChannel). I try to make it compatible with the client (so I don't need to change the client). This is when the problem occurs because I need to change from readUTF method to using ByteBuffer class to read the message, now I got a weird character in my message.
This is my message that works before using readUTF (Server) that come from writeUTF (Client)
{"command":"PUBLISH","resource":{"name":"","description":"","tags":[]}
When I sent the same message to my new server, I got this in my read method
�{"command":"PUBLISH","resource":{"name":"","description":"","tags":[]}
My read method is:
else if (clientKey.isReadable()) {
SocketChannel clientSocket = (SocketChannel) clientKey.channel();
ByteBuffer buffer = ByteBuffer.allocate(4096);
clientSocket.read(buffer);
String message = new String(buffer.array());
Logger.debug(message);
clientSocket.register(selector, SelectionKey.OP_WRITE);
}
This is how I write the message from client:
try (Socket echoSocket = new Socket(hostName, portNumber);
DataOutputStream streamOut = new DataOutputStream(echoSocket.getOutputStream());) {
streamOut.writeUTF(message.toJson());
}
toJson() is just a method to convert java object to JSON string and I use Jackson library to do that.
I have tried to remove it using regex and Normalizer library but it won't work.
Is there anyone experience the same things and solve it?
I am currently working on a one server many clients system. I am trying to get it so the server can send out one command, through a PrintWriter, that will go through to all of the clients connected on that socket. However in practice the command only goes through to one client. All of the clients are created on one socket, and all use the same Scanner. Is what I am trying to do possible?
Some code(incase it helps)
Creation of the socket:
serverSocketRefresh = new ServerSocket(PORTREFRESH);
refresh = serverSocketRefresh.accept();
Creation of the Print Writer and the Scanner:
networkOutputRefresh = new PrintWriter(refresh.getOutputStream(), true);
networkInput = new Scanner(refresh.getInputStream());
Ceation of the clients:
do
{
// Wait for client...
client = serverSocket.accept();
System.out.println("\nNew client accepted.\n");
handler = new ClientHandler(client,networkOutputRefresh, itemArray, bidderArray);
handler.start();
} while (true);
The command im trying to transmit to all of the clients:
public static void updatePrice()
{
networkOutputRefresh.println("1");
}
I am not sure if I correctly understand your code but it seems you are using a single reference of client. And your client reference will be holding the last client reference and hence the printwrite is writing only for that client. Ideally if you want to publish something to all the clients then you should have a collection of client references.Whenever you get an accept on the server socket, add the new client reference to your collection. And whenever you have to publish to all the clients just iterate over your client collection and publish using their associated printwriters.
Why not just use a BufferedReader and BufferedWriter, and make a new one each time you accept a client?
Edit: Or, if that '1' is the only thing you will ever send over that socket, just send it over the socket directly, as a byte. I believe the method is something like socket.write(new byte[] { 1 }, 0, 1), and to read on the other end, socket.read(buffer, 0, 1), where buffer is a byte array of length 1.