Develop Reference

Clarification about REST request in Java and output/input streams

I have to implement a post request in plain Java.
I have read the following question:
How to make a post request to a json RESTful Service from Java?
this is a part of the example
String url = "https://myStore.com/REST-API/";
String requestBody = "{\"searchProduct\": \"" + searchProduct + "\"}";
URL obj = new URL(url);
HttpsURLConnection connection = (HttpsURLConnection) obj
.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/json");
OutputStream outputStream = connection.getOutputStream();
outputStream.write(requestBody.getBytes());
My question is: why the parameters are written on the output stream? As far as I know, output stream is for collecting the output of a request, not to make it.
So just a curiosity, consider that I am obviously not skilled on this.

First let explain how HttpConnectionURL works.
When you want to request data from a server,
you first create a connection to that server.
Then you write data to the connection (request)
and finally read data from the connection (response).
So to write data to the connection you get a reference to the Connection's OutputStream and write data to it.
OutputStreamWriter writer = new OutputStreamWriter(
connection.getOutputStream());
writer.write("message=" + message);
To read data from the connection you get a reference to the Connection's InputStream and read data from it.
InputStreamReader reader = new InputStreamReader(connection.getInputStream());
reader.read();
Generally you use OutputStream when data is flowing out of your program (to file,network etc.,) and InputStream when data is flowing into your program (from file,network etc.,).
I think this will give you the clarity you are looking for.
This answer explains in detail how HttpConnectionURL works

The goal of InputStream and OutputStream is to abstract streams. By stream, I mean the way of the processed data (Input of the program or Output)
If the application receives information from the stream, use the InputStream. If it sends data then OutputStream
InputStreamused to read data from a source.
var input = new FileInputStream("input.txt");// Read the data
OutputStreamused for writing data to a destination.
var output = new FileOutputStream("output.txt");// Write the data
You should read answers in the related question : There are more explanations.

How to wait till HttpURLConnection completes?

If I upload 100K file to certain url of my service, wget takes ~20 seconds to complete:
wget --quiet --post-file data.txt --output-document - --header "Content-Type: text/csv" http://localhost:8080/ingest
But if I do it like this in java, strangely this happens immediately:
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "text/csv;charset=UTF-8");
con.setDoOutput(true);
OutputStream outputStream = con.getOutputStream();
outputStream.write(str.getBytes("UTF-8"));
outputStream.flush();
outputStream.close();
System.out.println("code=" + con.getResponseCode());
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
so my guess is that actually this code is not waiting for data to be submitted, but is doing this in background. How can I force it to block until actual data transfer is finished?

That code should be waiting for the response to complete. The con.getResponseCode() call will not (cannot!) return until the server has at least responded with the HTTP reply header containing the response code.
It may be that the server is sending the HTTP reply header before it has finished reading the data that the client has posted. That would be a mistake. (If the server sends the response too soon, it can't set the response code correctly!)
It is also possible that the server response is not a 2xx response, and there are server error messages / diagnostics on the error stream rather than the input stream. (Read the javadocs on getInputStream versus getErrorStream.)
So the most likely reason that is not blocking for ~20 seconds is because the request has failed ... and this is not being reported properly, due to server or client-side implementation issues.
UPDATE - It turns out that the real issues was that "curl" was behaving strangely on some platforms, probably due to network config issues.

Server receive unencoded characters using Java socket

I'm trying to send a message from a computer and read it from another using java socket. The problem is that the message received by the server is something unencoded represented by two question marks inside a diamond.
To read and write I use the following objects:
PrintWriter out = new PrintWriter(new OutputStreamWriter( socket.getOutputStream(), "UTF-8"), true);
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream(), "UTF-8")));
And the methods:
out.println("OK");
in.readLine();

Use DataInputStream and DataOutputStream to read/write the socket. These have readUTF() and writeUTF() methods which will send Strings properly.

reading bytes from web site

I am trying to create a proxy server.
I want to read the websites byte by byte so that I can display images and all other stuff. I tried readLine but I can't display images. Do you have any suggestions how I can change my code and send all data with DataOutputStream object to browser ?
try{
Socket s = new Socket(InetAddress.getByName(req.hostname), 80);
String file = parcala(req.url);
DataOutputStream out = new DataOutputStream(clientSocket.getOutputStream());
BufferedReader dis = new BufferedReader(new InputStreamReader(s.getInputStream()));
PrintWriter socketOut = new PrintWriter(s.getOutputStream());
socketOut.print("GET "+ req.url + "\n\n");
//socketOut.print("Host: "+req.hostname);
socketOut.flush();
String line;
while ((line = dis.readLine()) != null){
System.out.println(line);
}
}
catch (Exception e){}
}
Edited Part
This is what I should have to do. I can block banned web sites but can't allow other web sites in my program.
In the filter program, you will open a TCP socket at the specified port and wait for connections. If a
request comes (i.e. the client types a URL to access a web site), the application will process it to
decide whether access is allowed or not and then, using the same socket, it will send the reply back
to the client. After the client opened her connection to WebPolice (and her request has been checked
and is allowed), the real web page needs to be shown to the client. Therefore, since the user already gave her request, now it is WebPolice’s turn to forward the request so that the user can get the web page. Thus, WebPolice acts as a client and requests the web page. This means you need to open a connection to the web server (without closing the connection to the user), forward the request over this connection, get the reply and forward it back to the client. You will use threads to handle multiple connections (at the same time and/or at different times).

I don't know what exactly you're trying to do, but crafting an HTTP request and reading its response incorporates somewhat more than you have done here. Readline won't work on binary data anyway.
You can take a look at the URLConnection class (stolen here):
URL oracle = new URL("http://www.oracle.com/");
URLConnection yc = oracle.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
Then you can read textual or binary data from the in object.

Read line will treat the line read as a String, so unless you want to mess around with conversions over to bytes, I wouldn't recommend that.
I would just read bytes until you can't read anymore, then write them out to a file, this should allow you to grab the images, keeping file headers intact which can be important when dealing with files other than text.
Hope this helps.

Instead of using BufferedReader you can try to use InputStream.
It has several methods for reading bytes.
http://docs.oracle.com/javase/6/docs/api/java/io/InputStream.html

Url connection Inputstream and outputstream objects

This is my code
URL url = new URL("http://172.16.32.160:8080/epramaan/loginotp");
URLConnection connection1 = url.openConnection();
connection1.setDoOutput(true);
ObjectOutputStream out=new ObjectOutputStream(connection1.getOutputStream());
out.writeObject(send);
out.flush();
out.close();
ObjectInputStream in = new ObjectInputStream(connection1.getInputStream());
String output=(String)in.readObject();
in.close();
//Rest of the code
Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?

I assume that by stop execution you mean block.
Just noticed that you are using readObject and not read. Please elaborate what kind of data you are reading/writing and why are u using object streams ?
As you mentioned you are using String, I would suggest to use method readFully(byte[] buf). This method blocks till all the bytes are read. Once you have the byte array, a String can be created from this byte array.

You can use InputStream.read(byte[]) for reading the entire byte array to memory (you can get the array length from the HTTP Content-Length header) and use URLConnection.setReadTimeout() for timing out if you are blocking for too long.
From the byte array you can construct your object, constructing your ObjectInputStream over a ByteArrayInputStream

Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?
Not precisely. Opening the InputStream doesn't block anything, and doesn't even cause the request headers to be sent. However, reading from the InputStream will do both of those things.
I suspect that the real cause of your problems is that you are getting an error response from the server that is something other than a serialized object; e.g. it could be a generic HTML error page from the server. Naturally, attempting to deserialize this fails.
The correct procedure is:
Create the URLConnection object.
Set any request headers you need to.
Connect it (or skip this ... it will happen implicitly).
Open and write to the OutputStream.
Close the OutputStream.
Use getResponseCode() to see if the request succeeded or failed.
If it succeeded, call getInputStream() and read and process the response.
If it failed, call getErrorStream() and process the error output.