im trying to write a simple web server in java.
right now ive only got a simple program but id like to extend it so that it can serve multiple browsers by establishing multiple tcp connections.
ive been reading about threading. my understanding is that you can make a new thread and that will continue as if its another program entirely. so with a new thread, it can be like there are 2 web servers which can serve 2 browsers, or x web servers which can serve x web browsers.
im a bit lost on how to create new threads in java though and to give each new thread a connection.
my thoughts are that i would have a loop like this which gets new connections and passes each new connection to a new thread
// make new ServerSocket
while (true) {
Socket newConn = serverSocket.accept();
// make new thread, and pass in newConn
}
can someone give me some guidance on how to move forward?
(also if ive made an error somewhere, please do point it out. im new to threaded programming so its entirely possible ive not properly understood it)
rob
edit:
k thanks all.
i went and wrote something, that java tutorial helped a lot.
ive got a new issue now
i added a loop in my run() method in the new thread which contains a 10 second countdown (using Thread.sleep(1000)) whenever the server receives a request for an image, so i can see which threads are running when. (index.html has 4 images in it)
so i requested the index.html page and my server works fine. then i opened up about a dozen new tabs. my expectation was that the request for the index.html page would be instant but it would take 10 seconds for the images to be sent to the browser (because of that delay i put in there), at which point the server would receive the request for the next index.html page, and so on. overall, i thought that the dozen index.html pages would be served instantly while it would take 10 seconds for the 4 * 12 = 36 images to be served on all tabs.
what actually happened was it took 10 seconds to get the first 4 images, then 10 seconds for the next 4 images, etc. so rather than serving multiple web pages, my server just queues up requests and deals with one page at a time.
i think my program is at fault. but i feel like i might not properly understand how a browser interacts with a server. i thought the browser requests new objects as the html page is parsed. so my server should be receiving dozens of requests if i open a dozen pages. i tried opening up several tabs in FF and then several windows in FF but this did not help.
HOWEVER, when i opened up IE, FF and Chrome, and i asked for index.html at different times (about 2 seconds apart), it looked like each browser was receiving the page simultaneously, in other words, at one point, there were 12 different images being served, 4 to each browser
so i guess im looking for a bit of confirmation that this is the expected behavior? and if so, why is it that i could only see this behavior when i opened up 3 different browsers and not when i opened up multiple tabs?
(for those that asked, i plan ok taking a networks course next year, but im trying to do some of the basic stuff now. so half self learning, half h/w)
If you are looking for something robust look online for a working solution.
If it is for learning purposes, then create your own.
There are several ways to do this. The easiest is to do this As taken from the Java Tutorial :
import java.net.*;
import java.io.*;
public class MultiServer {
public static void main(String[] args) throws IOException {
ServerSocket serverSocket = null;
boolean listening = true;
try {
serverSocket = new ServerSocket(4444);
} catch (IOException e) {
System.err.println("Could not listen on port: 4444.");
System.exit(-1);
}
while (listening)
new MultiServerThread(serverSocket.accept()).start();
serverSocket.close();
}
}
import java.net.*;
import java.io.*;
public class MultiServerThread extends Thread {
private Socket socket = null;
public MultiServerThread(Socket socket) {
super("MultiServerThread");
this.socket = socket;
}
public void run() {
try {
PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(
socket.getInputStream()));
String inputLine, outputLine;
KnockKnockProtocol kkp = new KnockKnockProtocol();
outputLine = kkp.processInput(null);
out.println(outputLine);
while ((inputLine = in.readLine()) != null) {
outputLine = kkp.processInput(inputLine);
out.println(outputLine);
if (outputLine.equals("Bye"))
break;
}
out.close();
in.close();
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
You would implement your logic for processing your requests in:
KnockKnockProtocol kkp = new KnockKnockProtocol();
outputLine = kkp.processInput(null);
You could optimize your code by putting your Threads in a Thread Pool so you did not need to create a new Thread each time.
The part below is subjective and depends on the types of requests and what you do with each one.
If you have a lot of concurrent client requests then NIO is the way to go.
If your requests are short and you have over 10 concurrent ones create a pool.
If your requests are more than 100, then I would start looking at NIO.
You might also consider Netty and Java NIO. There's more than one way to do it.
Sun's classic Java tutorial includes a section on programming with sockets that walks you through an example very similar to the program you're trying to write.
a web server is nothing but a glorified socket server with messaging. tech has been around since the very first network connection was made. i had a project about a year and a half ago that was similar to what your trying to do. Java NIO is the best bet to start with, has connection and thread pooling and all the advanced stuff that a web server needs, but its a bit complicated. if you want a VERY good baseline to start with, check out http://www.quickserver.org/ the system i wrote was based on this and it now handles about 45,000 devices on one server last i heard.
Related
I'm currently working / almost finished building the beta version of my communication application. I'm running my server on EC2 instance but somehow the server application keeps get stuck at "Port already in use" exception" when I check the server status few days later. But it is working fine for first few days. This could be due to some mistake I have made when I was coding but I'm also doubting I haven't set up something on my linux OS yet. Is this something that happens occasionally to everyone?
=================================================
I'm mainly using follow lines of code to accept users constantly inside a loop
while (blinker != null) {
try {
ServerSocket ss = new ServerSocket(PORT, 10);
Socket connection = ss.accept();
//other stuffs going on
ss.close(); //forgot to mention that i actually close it all the time after accepting a new user
}
}
===================================
In addition to that, I'm mainly using port 443 and 80
Move the creation of the ServerSocket outside the loop. At present you are trying to create a new one every time you accept a connection, so you are conflicting with yourself. You only need one for the lifetime of the program.
When creating a standalone server in Java (not using a container like tomcat/jetty), what are the various techniques to keep the service running and not ending?
I have seen where people use a ServerSocket (since you will be communicating with the service presumably), and they use ServerSocket.accept() which blocks until it receives a message. And this is usually done in a while loop:
while(...) {
serverSocket.accept();
}
(http://docs.oracle.com/javase/6/docs/api/java/net/ServerSocket.html#accept())
Is this the only way? If not, what other ways are there and any pros/cons with it?
Are there any libraries that help with building your own service, or its pretty much roll your own.
There are various libraries that help you roll your own Windows/Unix service in Java:
Apache Commons Daemon
Akuma
Java Service Wrapper
How you keep the application running depends on the actual needs of your application. If your application is a server, you would normally want to listen for incoming connections which usually involves some sort of blocking/polling. How you do that again depends on the type of server you want to build. Of the generic solutions there's the ServerSocket class and its method accept() that you already mentioned. Another possibility is to use java.nio and implement a reactor which provides a single-threaded server that can handle multiple connections at once (see http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf for details), but may be a bit hard to grasp and debug.
What you probably want is a multi-threaded server.
Each time the server accepts a connection, the server creates a thread to handle sending/reciving to that client. If you do not use threads in your server, it will only be able to handle one connection at a time.
So, as you meantioned, the server loops infinitly and listens for incomming connections:
while(true){
serverSocket.accept();
ClientHandler c = new ClientHandler(serverSocket);
A instance of the class ClientHandler will be created each time a connection is accepted. This class implements Runnable, and loops for incomming messages using getInputStream and getOutputStream on that socket:
public class ClientHandler implements Runnable{
DataInputStream in;
DataOutputStream out;
//ClientHandler constructor
public ClientHandler(Socket s) throws IOException{
in= new DataInputStream(socket.getInputStream());
out=new DataOutputStream(socket.getOutputStream());
thread.start();
}
The run method:
public void run() {
while(true){
String temp="";
while ((temp = (String) in.readUTF()) != null){ // Read from the input stream each iteration. When temp is not null a message is recived
System.out.println(temp);
Please that the above code does not take into account different exceptions that might occur and is very basic. But it should give you a basic idea on how a server using Sockets can be implemented.
For a quick solution (in a testing environment only!) you can go for something often dubbed as "Enterprise Loop" (because it is too often found in production systems):
while (true)
try {
// do something
} catch (Throwable t) {
// maybe log
}
However, this is not good style in the production code.
(see [1] for a parody of that idiom)
To create a service, you want one of the libraries from this answer.
If you "just need multithreading", have a look into the Java concurrency framework. I stronly suggest reading Java Concurrency in Practice, as multi-threading is much more that just starting another thread and errors are hard to debug.
[1] http://blog.antiblau.de/2016/01/26/java-enterprise-loop/
I have the starts of a very basic multi-hreaded web server, it can recieve all GET requests as long as they come one at a time.
However, when multiple GET requests come in at the same time, sometimes they all are recieved, and other times, some are missing.
I tested this by creating a html page with multiple image tags pointing to my webserver and opening the page in firefox. I always use shift+refresh.
Here is my code, I must be doing something fundamentally wrong.
public final class WebServer
{
public static void main(String argv[]) throws Exception
{
int port = 6789;
ServerSocket serverSocket = null;
try
{
serverSocket = new ServerSocket(port);
}
catch(IOException e)
{
System.err.println("Could not listen on port: " + port);
System.exit(1);
}
while(true)
{
try
{
Socket clientSocket = serverSocket.accept();
new Thread(new ServerThread(clientSocket)).start();
}
catch(IOException e)
{
}
}
}
}
public class ServerThread implements Runnable
{
static Socket clientSocket = null;
public ServerThread(Socket clientSocket)
{
this.clientSocket = clientSocket;
}
public void run()
{
String headerline = null;
DataOutputStream out = null;
BufferedReader in = null;
int i;
try
{
out = new DataOutputStream(clientSocket.getOutputStream());
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
while((headerline = in.readLine()).length() != 0)
{
System.out.println(headerline);
}
}
catch(Exception e)
{
}
}
First, #skaffman's comment is spot on. You should not catch-and-ignore exceptions like your code is currently doing. In general, it is a terrible practice. In this case, you could well be throwing away the evidence that would tell you what the real problem is.
Second, I think you might be suffering from a misapprehension of what a server is capable of. No matter how you implement it, a server can only handle a certain number of requests per second. If you throw more requests at it than that, some have to be dropped.
What I suspect is happening is that you are sending too many requests in a short period of time, and overwhelming the operating system's request buffer.
When your code binds to a server socket, the operating system sets up a request queue to hold incoming requests on the bound IP address/port. This queue has a finite size, and if the queue is full when a new request comes, the operating system will drop requests. This means that if your application is not able to accept requests fast enough, some will be dropped.
What can you do about it?
There is an overload of ServerSocket.bind(...) that allows you to specify the backlog of requests to be held in the OS-level queue. You could use this ... or use a larger backlog.
You could change your main loop to pull requests from the queue faster. One issue with your current code is that you are creating a new Thread for each request. Thread creation is expensive, and you can reduce the cost by using a thread pool to recycle threads used for previous requests.
CAVEATS
You need to be a bit careful. It is highly likely that you can modify your application to accept (not drop) more requests in the short term. But in the long term, you should only accept requests as fast as you can actually process them. If it accepts them faster than you can process them, a number of bad things can happen:
You will use a lot of memory with all of the threads trying to process requests. This will increase CPU overheads in various ways.
You may increase contention for internal Java data structures, databases and so on, tending to reduce throughput.
You will increase the time taken to process and reply to individual GET requests. If the delay is too long, the client may timeout the request ... and send it again. If this happens, the work done by the server will be wasted.
To defend yourself against this, it is actually best to NOT eagerly accept as many requests as you can. Instead, use a bounded thread pool, and tune the pool size (etc) to optimize the throughput rate while keeping the time to process individual requests within reasonable limits.
I actually discovered the problem was this:
static Socket clientSocket = null;
Once I removed the static, it works perfectly now.
I'm trying some multiplayer game ideas out at the moment and am trying to create a Java application to serve a web browser based multiplayer game.
My development environment is Eclipse on the main machine, and notepad + Google Chrome on this laptop.
I'm creating the websocket using javascript at the client end, and using the java.net.Socket at the server end.
I've managed to get a connection acknowledged at both ends, but can't seem to send or recieve any data between them without the client closing the connection (doesn't even error; just seems to freak out at something and call socket.close).
Does anyone have any ideas?
Here's some code:
Client:
<script type="text/javascript">
var socket;
function init() {
socket = new WebSocket("ws://192.168.0.3:10000");
socket.onopen = function() { alert('OPEN: ' + socket.readyState); }
socket.onmessage = function (msg) { alert('DATA: ' + msg.data); }
socket.onerror = function (msg) { alert('DATA: ' + msg.data); }
socket.onclose = function () { alert('CLOSED: ' + socket.readyState); }
}
function onClick() {
socket.send("YAY!");
}
</script>
Server:
public static void main(String args[])
{
System.out.printLn("Websocket server test");
ServerSocket connectSocket = null;
try
{
Socket clientSocket;
connectSocket = new ServerSocket(10000);
System.out.printLn("Waiting for connection...");
clientSocket = connectSocket.accept();
System.out.printLn("Got one!");
PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(clientSocket.getInputStream()));
for(int i=0;i<100;i++) //Shit but easy
{
String data = in.readLine();
System.out.printLn("Got data: " + data);
out.printLn("YAY!");
}
}
catch (IOException e)
{
System.out.printLn("You fail: " + e.getMessage());
}
System.out.printLn("Finished!");
}
Rather than going the painful way of implementing the spec in Java, I'd suggest that you use an existing solution like jWebSocket.
Also if you don't mind leaving Java land, I'd also suggest that you take a look at Node.js for your Server.
Doing both Server and Client in JavaScript will save you lots of time and lots of Code, especially since JSON just doesn't fit that well into static land. Also creating multiplayer servers in Node.js is trivial, since the event based, single threaded model fits the whole thing pretty well.
More information on WebSocket can be found in the FAQ. In case you want to get started with Node.js take a look at the TagWiki.
shameless plug follows
For two multiplayer games that were written using Node.js take a look at my GitHub page.
Try this lib - https://github.com/mrniko/netty-socketio
Based on high performance socket lib Netty. It supports latest protocol of Socket.IO server. Several transports including websocket.
On web side use Socket.IO client javascript lib:
<script type="text/javascript">
var socket = io.connect('http://localhost:81', {
'transports' : [ 'websocket' ],
'reconnection delay' : 2000,
'force new connection' : true
});
socket.on('message', function(data) {
// here is your handler on messages from server
});
// send object to server
var obj = ...
socket.json.send(obj);
</script>
I would suggest our high level solution: Bristleback Server. It contains both server and client, you can choose from several existing low level WebSocket engines (like Jetty, Netty or Tomcat), developing with Bristleback is extremally fast and easy. However, it is still Beta and we are working hard to release a final 1.0.0 version. If you use Maven, we have provided an archetype with ready to use web application.
I am one of the co-creators of Bristleback Server.
As no one yet really answered your question: the reason it does not work, is because you are not implementing the websocket specification. It takes of lot more work to setup a proper websocket connection than just opening a socket, as the websocket connection setup starts with a HTTP upgrade request. Your client is closing the connection, because it does not receive a positive answer on the upgrade request to start with.
I can't help you with sockets, but can i suggest you to use RMI technology? I'm trying to make a multiplayer rpg in java, and i'm using remote method invocation between server and client (it is possible also call-back the client from the server). It's really easy use it, but it uses TCP instead of UDP. In LAN experience there is no lag, on internet I have not tried yet. However, if your game tolerates just a bit retard between request and response, there is no problem.
This is the link of my project, Client and Server classes may be useful for you.
A while ago I developed a little LAN chat app. in Java which allows chatting with other hosts, send images, etc. Although it was created just for fun, now it's being used where I work.
Currently, there is no "chat server" on the app. where each client registers, updates it's status, etc. (I liked the idea of symmetric design and not depending on a server running on some other machine).
Instead, each host is a client/server which has a hosts.properties file with the hostname of the other hosts, and - for instance - broadcasts to each one of them when sending a massive message/image/whatever.
In the beginning there were just a couple of hosts, so this hosts.properties file wasn't an issue. But as the amount of users increased, the need of updating that file was a bit daunting. So now I've decided to get rid of it, and each time the app. starts, dynammically find the other active hosts.
However, I cannot find the correct way of implement this. I've tried starting different threads, each one of them searching for other hosts in a known range of IP addresses. Something like this (simplified for the sake of readability):
/** HostsLocator */
public static void searchForHosts(boolean waitToEnd) {
for (int i=0; i < MAX_IP; i+= MAX_IP / threads) {
HostsLocator detector = new HostsLocator(i, i+(MAX_IP / threads - 1)); // range: from - to
new Thread(detector).start();
}
}
public void run() {
for (int i=from; i<=to; i++)
findHosts( maskAddress + Integer.toString(i) );
}
public static boolean findHosts(String IP) {
InetAddress address = InetAddress.getByName(IP);
if ( address.isReachable(CONNECTION_TIME_OUT) )
// host found!
}
However:
With a single thread and a low value in CONNECTION_TIME_OUT (500ms) I get wrong Host Not Found status for for hosts actually active.
With a high value in CONNECTION_TIME_OUT (5000ms) and only one single thread takes forever to end
With several threads I've also found problems similar like the first one, due to collisions.
So... I guess there's a better way of solving this problem but I couldn't find it. Any advice? Thanks!
You could try UDP Broadcast to a specific port. All running instances of your app on the network could listen to that port and then respond with a message identifying them as a host.
You could do this a lot easier using UDP. Check this tutorial for examples.
Use Bonjour/Zeroconf.
The jmdns project has all you need.
For finding all hosts in lan in java execute commands from java and add the result to JList
Here is the small code that will help to you to read all hosts in lan in windows there will be other commands for other os take look at the following code
try {
Runtime rt = Runtime.getRuntime();
FileWriter write=new FileWriter("mylist.txt");
BufferedWriter writer=new BufferedWriter(write);
Process pr = rt.exec("net view");
BufferedReader input = new BufferedReader(new InputStreamReader(pr.getInputStream()));
String line=null;
String hosts="";
while((line=input.readLine()) != null) {
Thread.sleep(100);
if((!(line.equals("")))&&(!(line.equalsIgnoreCase("Server Name Remark")))&&(!(line.equalsIgnoreCase("-------------------------------------------------------------------------------")))&&(!(line.equalsIgnoreCase("The command completed successfully."))))
{
line=line.replace('\\',' ');
line=line.trim();
listModel.addElement(line);
hosts=hosts+line.trim()+",";
hosts=hosts.trim();
}
}
writer.write(hosts);
writer.close();
} catch(Exception e) {
System.out.println(e.toString());
e.printStackTrace();
}
Each host keeps track of all the hosts they have met. When you shut down, save the known hosts to file and reuse the next time you start up.
Every so many minutes, send each of the known hosts a list of all known hosts.
That way
a) No network scanning
b) A new host will spread around the network
Then when a new host joins, he just needs to know 1 other host to learn about everyone.
A host that isn't seen for a week, or is seen from a new IP is dropped from the list of updated.
You could attempt to use DNS service discovery
There seems to be a project on sourceforge (that I have not looked at, beyond doing a cursory search...)