Develop Reference

Clarification about REST request in Java and output/input streams

I have to implement a post request in plain Java.
I have read the following question:
How to make a post request to a json RESTful Service from Java?
this is a part of the example
String url = "https://myStore.com/REST-API/";
String requestBody = "{\"searchProduct\": \"" + searchProduct + "\"}";
URL obj = new URL(url);
HttpsURLConnection connection = (HttpsURLConnection) obj
.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/json");
OutputStream outputStream = connection.getOutputStream();
outputStream.write(requestBody.getBytes());
My question is: why the parameters are written on the output stream? As far as I know, output stream is for collecting the output of a request, not to make it.
So just a curiosity, consider that I am obviously not skilled on this.

First let explain how HttpConnectionURL works.
When you want to request data from a server,
you first create a connection to that server.
Then you write data to the connection (request)
and finally read data from the connection (response).
So to write data to the connection you get a reference to the Connection's OutputStream and write data to it.
OutputStreamWriter writer = new OutputStreamWriter(
connection.getOutputStream());
writer.write("message=" + message);
To read data from the connection you get a reference to the Connection's InputStream and read data from it.
InputStreamReader reader = new InputStreamReader(connection.getInputStream());
reader.read();
Generally you use OutputStream when data is flowing out of your program (to file,network etc.,) and InputStream when data is flowing into your program (from file,network etc.,).
I think this will give you the clarity you are looking for.
This answer explains in detail how HttpConnectionURL works

The goal of InputStream and OutputStream is to abstract streams. By stream, I mean the way of the processed data (Input of the program or Output)
If the application receives information from the stream, use the InputStream. If it sends data then OutputStream
InputStreamused to read data from a source.
var input = new FileInputStream("input.txt");// Read the data
OutputStreamused for writing data to a destination.
var output = new FileOutputStream("output.txt");// Write the data
You should read answers in the related question : There are more explanations.

HttpsUrlConnection response returned from servlet contains extra 'b''0''\r\n' characters when read through python library

I am using HttpsURLConnection to call a server and return the response returned from the HttpsURLConnection from my servlet. I am copying the response from HttpssURLConnection to HttpServletresponse using streams, copying bytes from the httpconnection response input stream to the response's output stream, checking the end by seeing if read returns < 0.
Following is the code for copying the response. The variable response is of type HttpServletResponse and the variable httpCon is of type HttpsURLConnection.
InputStream responseStream = httpCon.getInputStream();
if (responseStream != null)
{
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
int len;
while ((len = responseStream.read(buffer)) >= 0)
{
os.write(buffer, 0, len);
}
os.flush();
os.close();
}
On the client side, I am using python requests library to read the response.
What I am seeing that if I use the curl to test my servlet, I am getting the proper response json, response = u'{"key":"value"}'.
If i read it from the requests python, it is putting some extra characters in the response , the response looks like the following
response = u'b0\r\n{"key":"value"}\r\n0\r\n\r\n'
Both the strings are unicode. But the second one has extra characters.
Same resonse if I try from curl/Postman restclient, I am able to get it properly. But from python requests, it is not working. I tried another livetest library in python, with that also, it is not working and the response has same characters. I also tried to change the accept-encoding header but it did not have any effect.
Because of this, I am not able to parse the json.
I don't want to change the client to parse this kind of string.
Can I change something on the server so that it will work correctly?

Did the response contain the below header "Transfer-Encoding: chunked"?
The response should be in Chunked transfer encoding
https://en.wikipedia.org/wiki/Chunked_transfer_encoding.
In this case, you get \r\n0\r\n\r\n at the end of the response is as expected since it is terminating symbol of this encoding. I guest curl/Postman just help us to handle Chunked transfer encoding, so you can't find these chunked symbols.

new to android - understanding the use of OutputStreamWriter in web service calls

I have been going over the following tutorial and came across this code which I do not understand the purpose of:
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write( data );
wr.flush();
I don't know what is the purpose of the above code and where is it writing this data to...
From what I could gather, the documentation states that it converts character to bytes... but then it writes it to some where... not sure why this is.

It is basically used for turning a character stream into a byte stream.
The byte streams and character streams are incompatible for linking as the first one operates on 8-bit ASCII characters and the other on 16-bit Unicode characters. To link them explicitly, two classes exist in java.io package, InputStreamReader and OutputStreamWriter.
InputStreamReader links a byte stream, with the character stream BufferedReader (on reading-side) .
Whereas with the OutputStreamWriter, the characters of 2-bytes are encoded (converted) into bytes of 1-byte (InputStreamReader does it other way – bytes to characters).
For a Java program to interact with a server-side process it simply must be able to write to a URL, thus providing data to the server. It can do this by following these steps:
1.Create a URL.
2.Retrieve the URLConnection object.
3.Set output capability on the URLConnection.
4.Open a connection to the resource.
5.Get an output stream from the connection.
6.Write to the output stream.
7.Close the output stream.
Now in the snippet you provided ,
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
creates an output stream on the connection and opens an
OutputStreamWriter on it--Step 5 If the URL does support output, then
this method returns an output stream that is connected to the input
stream of the URL on the server side — the client's output is the
server's input.If the URL does not support output, getOutputStream
method throws an UnknownServiceException.
And
wr.write( data );
wr.close();
wr.flush();
It writes the required information to the output stream and closes the
stream. The data written to the output stream on the client side is
the input on the server side.

It's writing it to the output stream of the URLConnection - which is basically used for the body of an HTTP request (assuming it's an HTTP URL, of course).

Url connection Inputstream and outputstream objects

This is my code
URL url = new URL("http://172.16.32.160:8080/epramaan/loginotp");
URLConnection connection1 = url.openConnection();
connection1.setDoOutput(true);
ObjectOutputStream out=new ObjectOutputStream(connection1.getOutputStream());
out.writeObject(send);
out.flush();
out.close();
ObjectInputStream in = new ObjectInputStream(connection1.getInputStream());
String output=(String)in.readObject();
in.close();
//Rest of the code
Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?

I assume that by stop execution you mean block.
Just noticed that you are using readObject and not read. Please elaborate what kind of data you are reading/writing and why are u using object streams ?
As you mentioned you are using String, I would suggest to use method readFully(byte[] buf). This method blocks till all the bytes are read. Once you have the byte array, a String can be created from this byte array.

You can use InputStream.read(byte[]) for reading the entire byte array to memory (you can get the array length from the HTTP Content-Length header) and use URLConnection.setReadTimeout() for timing out if you are blocking for too long.
From the byte array you can construct your object, constructing your ObjectInputStream over a ByteArrayInputStream

Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?
Not precisely. Opening the InputStream doesn't block anything, and doesn't even cause the request headers to be sent. However, reading from the InputStream will do both of those things.
I suspect that the real cause of your problems is that you are getting an error response from the server that is something other than a serialized object; e.g. it could be a generic HTML error page from the server. Naturally, attempting to deserialize this fails.
The correct procedure is:
Create the URLConnection object.
Set any request headers you need to.
Connect it (or skip this ... it will happen implicitly).
Open and write to the OutputStream.
Close the OutputStream.
Use getResponseCode() to see if the request succeeded or failed.
If it succeeded, call getInputStream() and read and process the response.
If it failed, call getErrorStream() and process the error output.

Compressing and decompressing streams

I found this article about simple proxy server implemented in JAVA:
http://www.java2s.com/Code/Java/Network-Protocol/Asimpleproxyserver.htm
The code simply gets some stream from the client, after sends it to the server and after it gets stream from the server and sends the response to the client. What I would like to do is to compress this streams before it is sent and decompress after it is received.
I found the class GZIPInputStream but I'm not sure how to use it and what I found on internet didn't help me. I either didn't understand that so much or it was not a good solution for me.
My idea is too that but I'm not sure if its ok:
final InputStream streamFromClient = client.getInputStream();
final OutputStream streamToClient = client.getOutputStream();
final InputStream streamFromServer = server.getInputStream();
final OutputStream streamToServer = server.getOutputStream();
InputStream gzipStream = new GZIPInputStream(streamFromClient );
try
{
while ((bytesRead = gzipStream.read(request)) != -1)
{
streamToServer.write(request, 0, bytesRead);
streamToServer.flush();
}
}
catch (Exception e) {
System.out.println(e);
}
Now the data sent to the server should be compressed before sending (but I'm not sure if it's a correct solution). IS IT?
Now imagine the server sends me the compressed data.
So this stream:
final InputStream streamFromServer = server.getInputStream();
is compressed.
How can I decompress it and write to the
final OutputStream streamToClient = client.getOutputStream();
Thanks for the help, guys!

Read the javadoc of these streams : http://download.oracle.com/javase/6/docs/api/java/util/zip/GZIPInputStream.html and http://download.oracle.com/javase/6/docs/api/java/util/zip/GZIPOutputStream.html.
GZIPOutputStream compresses the bytes you write into it before sending them to the wrapped output stream. GZIPInputStream reads compressed bytes from the wrapped stream and returns uncompressed bytes.
So, if you want to send compressed bytes to anyone, you must write to a GZIPOutputStream. But of course, this will only work if the receiving end knows it and decompresses the bytes it receives.
Similarly, if you want to read compressed bytes, you need to read them from a GZIPInputSTream. But of course, it'll only work if the bytes are indeed compressed using the same algorithm by the sending end.