I need a java program to download a series of file from a web server and I need to be able to close and then reopen the tcp/ip connection between each file download.
Not sure how to do this.
Here is a low-level way of retrieving any information from not only socket 80 (http) but generically to retrieve mail, do a 'telnet' etc:
socket = new Socket();
socket.connect(new InetSocketAddress(host, port));
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
now to do a call to retrieve a page, send the get command, set the content-type, and send two '\n'.
There is a higher-level solution:
Send HTTP GET request with header
You can create a Socket for the IP and port you want to send it to, then read the file into a byte array using a FileInputStream and send that byte array through a DataOutPutStream, then when done just flush the dataOutPutStream and close the Socket and call the function again sending the next file.
use the logic:
Please post something that you have tried.
do{
connection.open;
while(!eof){
fetchfiles();
}
connection.close;
}while(no of files yet to download);
Related
I have a problem, where a message I write to the OutputStream of a SSLSocket is send in multiple parts instead of one packet.
So, I have a SSLSocket:
final Socket newSocket = SSLSocketFactory.getDefault ().createSocket (server, port);
Since I connect to a special service, I manually send a non HTTP-standard connect header, which works just fine:
final OutputStream outputStream = newSocket.getOutputStream ();
outputStream.write (connect.getBytes ());
outputStream.flush ();
newSocket.setKeepAlive (true);
I then wait for a response, until I get a newline in it:
final InputStream inputStream = newSocket.getInputStream ();
final InputStreamReader inputStreamReader = new InputStreamReader (inputStream, "ISO-8859-1");
final BufferedReader bufferedReader = new BufferedReader (inputStreamReader);
I finish reading when I received a HTTP/1.1 200 Ok in a loop reading from the BufferedReader. Again, until this point everything is fine.
Then I begin using this socket to send messages from my own protocol like this:
outputStream.write (message.getBytes ());
outputStream.flush ();
When I now run a tcpdump on the target device, I can see that the socket connection is established and in fact two packets are send. Assuming the message would be Hello World, there is one packet which is marked as malformed, only containing the H. Then another one follows with the content ello World. No Exception is thrown during the run of this program.
Would could I possibly have done or missed to cause this kind of behaviour?
There are reasons for that I can not give you the complete code, but this question already contains all the significant passages.
Update:
As proposed in the comments, I tried to disable the Nagle algorithm using newSocket.setTcpNoDelay(true); just after creating the socket. But still, the packets sent are fragmented.
I got a question for something I can't seem to understand.
If I use the snippet:
DataOutputStream dataoutput = new DataOutputStream(Socket.getOutputStream());
dataoutput.writeBytes(diagnostics);
dataoutput.flush();
The output:diagnostics that is sent from server to client hangs, where even using .flush() will not help. Only closing the socket will release the data packet.
However, if I add a newline character:
dataoutput.writeBytes(diagnostics+"\n");
it works perfectly. And if you duplicate the code,
dataoutput.writeBytes(diagnostics);
dataoutput.writeBytes(diagnostics);
You get the same output as the one with the /newline without duplicate data. Why?
EDIT: Client-side.
incoming = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
response = incoming.readLine();
Like the good lads puts in the comment you should use a flush to force send the payload to destination of the stream. But if that isn't working i suggest to try the following :
DataOutputStream dataoutput = new DataOutputStream(System.out);
dataoutput.writeBytes(diagnostics);
dataoutput.flush();
So now the destination will be the "local" console. and no remote connection is between the source and the destination of the stream, perhaps you can try for yourself if anything is working properly "locally" (and focus in the DataOutputStream behaviour) and then start to tackle the remote streaming process.
Is it not to much, but hope it helps a little!
Greetings!!
Supposing I have a servlet that reads and writes a file to a socket and I want to establish a connection between the servlet and the socket, how could this be done?
Well, for a socket, i read somewhere that one just has to do this:
URL url = new URL("http://example.com/getFile");
URLConnection con = url.openConnection();
con.setDoOutput(true);
How could I do the same for a servlet?
I wanted to do this since I will be sending and receiving files between the servlet and the scoket. Also, how would both of them know when the other has sent a file and should read it?
I have searched for this but can't find any site that explains it.
I am unsure what do you mean by 'Socket'. If the socket is a standard protocol (http/https/ftp) then you can use the above piece of code in the Servlet just as you would use in a stand alone program.
If your socket does NOT prescribe to any standard then you might want to open a direct connection using the below piece of code:
Socket socket = new Socket(server,port);
//Get an input stream from the socket
BufferedReader inputStream =
new BufferedReader(new InputStreamReader(
socket.getInputStream()));
//Get an output stream to the socket. Note
// that this stream will autoflush.
PrintWriter outputStream =
new PrintWriter(new OutputStreamWriter(
socket.getOutputStream()),true);
outputStream.println("send_ data");
System.out.println(inputStream.readLine());
socket.close();
Regarding the file sending piece of code, if the primary task involved is file transfer you can use FTP protocol to send files between the servlet and socket.
you can have a file watcher on the socket end to know when a file is transferred. Refer filewatcher and ftp for more information.
You confuse servlets and sockets it looks like. Here is the difference:
Sockets are used for a low-level communication between two points in network. By low-level I mean that you basically send whatever binary data you want over socket and receive some data back. In order to establish a connection there should be a ServerSocket waiting for connections and client socket which establishes connection with server socket.
Servlet is J2EE abstraction. Servets are used in web application containers and those work on top of some protocol (in most cases HTTP). Servlets don't work with sockets, although web server uses sockets internally. Servlets work with Request and Response objects. Your servlet is responsible for processing user request and forming a response.
I want to send an InputStream containing XML from the client to a server. Said server manipulates the XML stream using Streaming API for XML (StAX) and sends it back to the client.
I mange to send the XML to the server, the client receives the answer, but doesn't exit because the server never finishes reading the InputStream form the client.
But when the server is just dumping the input to System.out instead of sending something, reading finishes...
edit I forgot to mention that IOUtils.copy() is running in two separate threads on the client.
Here some code to reproduce this, I've replaced the StAX part with IOUtils.copy() on the server.
Server
ServerSocket serverSocket = new ServerSocket(port);
Socket clientSocket = serverSocket.accept();
InputStream in = clientSocket.getInputStream();
OutputStream out = clientSocket.getOutputStream();
IOUtils.copy(in, out);
Client
Socket socket = new Socket(host, port);
FileInputStream fin = new FileInputStream(file);
OutputStream out = socket.getOutputStream();
IOUtils.copy(fin, out)
InputStream in = socket.getInputStream();
IOUtils.copy(in, System.out)
You must flush and close InputStream and OutputStream.
like this:
ServerSocket serverSocket = new ServerSocket(port);
Socket clientSocket = serverSocket.accept();
InputStream in = clientSocket.getInputStream();
OutputStream out = clientSocket.getOutputStream();
IOUtils.copy(in, out);
//add
out.flush();
in.close();
out.close;
You are probably creating a deadlock.
Client - sends data to the server.
Server - loops data back to the client.
Client - reads data (that the server looped back)
So, while the client is sending data to the server, the server is sending it back to the client.
The client isn't reading that data, it's busy sending.
At some point the TCP buffers fill up, and the server blocks on sending more data to the client. This means the server is blocked on sending data, so eventually the TCP buffers on the receiving side of the server fills up as well, and the client blocks on sending data. Since the client is now blocked on sending, it'll never get to the part where it reads the data that is looped back, so the server will never un-block.
If you're sending a small document, you might not experience this, as all the data fits in the buffers, but once you fill up the buffers, the system will deadlock.
You have to multiplex the sending/receiving. e.g. you could start up a thread on the client that does
InputStream in = socket.getInputStream();
IOUtils.copy(in, System.out)
That way receiving can be done in parallel with the client sending data
Or you could use 2 threads on the server side. One for receiving the data, posting data to be sent back to another thread doing the sending (you'll have to build up an internal queue in this case between the threads).
As far as I remember IOUtils.copy() does not flush and does not close the stream. It seems that you do not call neither flush nor close too. Try it. I believe that this is your problem.
I'd like to write some messages to the server.
Each time, for the tramsmitting only, I'm closing the outputStream and reopen it when I have to send the next message.
os.write(msgBytes);
os.write("\r\n".getBytes());
os.flush();
os.close();
How Can I keep this Socket's OutputStream, os, open and still be able to send the message?
Thanks.
I am missing something here. If you don't call close, it will not close. For example,
os.write(msgBytes);
os.write("\r\n".getBytes());
os.flush();
// Do something
os.write("more message");
os.flush();
// When you are finally done
os.close();
In most protocols, the server accepts som kind of EOF symbol. Send such a symbol instead of closing the stream.
For example, IRC servers interpret "\r\n" as the end of a message. This would be 2 messages on one open OutputStream:
PrintStream printStream = new PrintStream(socket.getOutputStream());
printStream.print("JOIN #channel1\r\n");
printStream.flush( );
printStream.print("JOIN #channel2\r\n");
printStream.flush( );
Also, you should wrap your outputStream with DataOutputStream. This wrapper creates more portable output. Plain OutputStream can cause trouble with certain primitive datatypes if server and client have different computer architectures.
Wrap the Socket's OutputStream in a PrintWriter and call the PrintWriter's println method.
PrintWriter pw = new PrintWriter(socket.getOutputStream());
....
pw.println(message); // call repeatedly to send messages.
....
pw.close(); // when finished writing to server and want to close conn.
I have found the problem and it lays on the client's side.
On the client, I have used -
count = inputStream.read(buffer)) > -1
That means, the client waits till server's outputStream closed, and then processing the incoming data.