I'm writing a simple client-server system and the question is: how to structure my client code in order to get POST request-response working in a loop?
At the moment it looks something like this (and it's is NOT a loop right now):
open HttpURLConnection
set properties
setDoOutput(true)
writing to output stream
closing output stream
new DataInputStream
reading response
exiting method
I'm not sure which objects do I have to save for the next iterations and which ones I should close.
you need to save the connection object and you should make use of setDoInput(true) for reading data but if you just want to read responseCode and responseMessage you dont need InputStream. check the code below.
HttpURLConnection connection =(HttpURLConnection)new URL("url").openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-type", "text/xml"); // depend on you
connection.setRequestProperty("Accept", "text/xml, application/xml"); // depend on you
connection.setRequestMethod("POST");
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write(yaml);
writer.close();
int statusCode = connection.getResponseCode();
String message = connection.getResponseMessage();
for InputStreamReader
connection.setDoInput(true);
InputStreamReader reader = new InputStreamReader(connection.getInputStream());
char[] cbuf = new char[100];
reader.read(cbuf);
// there are 3 read method you can choose as per your convenience
//and put a check for end of line in while loop for reading whole content.
reader.close();
After managing my own 'research' on this subject (thanks to Google and Nokia Developer forums) I've come to the final view of my code. It's a file upload loop:
path = Paths.get(requestString);
in = Files.newInputStream(path);
int i = 0;
while ((bytesRead = in.read(buf)) != -1) {
URL u = new URL(defaultURL);
huc =
(HttpURLConnection) u.openConnection();
huc.setRequestMethod("POST");
huc.setDoOutput(true);
huc.setDoInput(true);
os = huc.getOutputStream();
os.write(buf, 0, bytesRead);
os.flush();
os = null;
// thanks to dku.rajkumar for the following block of code !
InputStreamReader reader =
new InputStreamReader(huc.getInputStream());
char[] cbuf = new char[400];
reader.read(cbuf);
reader.close();
String s = new String(cbuf);
messagebuffer.append(s + "\n\n");
huc.disconnect();
Thread.sleep(16);
}
Related
For my project i need to download a pdf file from google drive using java
I get my httpresponse code 200 and by using following method i store it in abc.pdf file
String url = "https://docs.google.com/uc?id="+fileid+"&export=download";
URL obj = new URL(url);
HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
// optional default is GET
conn.setRequestMethod("GET");
//add request header
conn.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
OutputStream f0 = new FileOutputStream("C:\\Users\\Darshil\\Desktop\\abc.pdf",true);
while ((inputLine = in.readLine()) != null) {
//System.out.println(inputLine);
byte b[]=inputLine.getBytes();
//System.out.println(b);
f0.write(b);
}
in.close();
f0.close();
But when i try to open abc.pdf in my adobe reader x i get following error:
There was an error opening this document.The file is damaged and could not be repaired
You seem to be directly accessing the Google drive using Raw HTTP requests.
You may be better of using the Google Drive SDK. This link contains good examples to address the use cases you state in your question.
However if you do want to stick to your technique then you should not be using a BufferedReader.readLine(). This is because the PDF file is a binary finally that would depend upon the correct byte sequences to be preserved in order to be read correctly by the PDF reader software. Hopefully the below technique should help you:
//read in chunks of 2KB
byte[] buffer = new byte[2048];
int bytesRead = 0;
try(InputStream is = conn.getInputStream())
{
try(DataOutputStream os = new DataOutputStream(new FileOutputStream("file.pdf"))
{
while((bytesRead = is.read(buffer)) != -1)
{
os.write(buffer, 0, bytesRead);
}
}
}
catch(Exception ex)
{
//handle exception
}
Note that I am using the try-with-resources statement in Java 7
Hope this helps.
NOT a duplicate of my other question.
I am sending a POST request like this:
String urlParameters = "a=b&c=d";
String request = "http://www.example.com/";
URL url = new URL(request);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setInstanceFollowRedirects(false);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("charset", "utf-8");
connection.setRequestProperty("Content-Length", "" + Integer.toString(urlParameters.getBytes().length));
connection.setUseCaches(false);
DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
connection.disconnect();
How can I read the xml response returned from a HTTP POST request? Particularly, I want to save the response file as a .xml file, and then read it. For my usual GET requests, I use this:
SAXBuilder builder = new SAXBuilder();
URL website = new URL(urlToParse);
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream("request.xml");
fos.getChannel().transferFrom(rbc, 0, 1 << 24);
fos.close();
// Do the work
Addendum: I'm using the following code and it works just fine. However, it neglects any spacing and new lines and treats the complete XML contents as a single line. How do I fix it?
InputStream is = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
StringBuilder sb1 = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
sb1.append(line);
}
FileOutputStream f = new FileOutputStream("request.xml");
f.write(sb1.toString().getBytes());
f.close();
br.close();
don't use Readers and readLine() with xml data. use InputStreams and byte[]s.
Thanks to Pangea, I modified his code and this now works:
TransformerFactory transFactory = TransformerFactory.newInstance();
Transformer t= transFactory.newTransformer();
t.setOutputProperty(OutputKeys.METHOD, "xml");
t.setOutputProperty(OutputKeys.INDENT,"yes");
Source input = new StreamSource(is);
Result output = new StreamResult(new FileOutputStream("request.xml"));
transFactory.newTransformer().transform(input, output);
The code below
OutputStream outputWeb = conn.getOutputStream();
File file = new File("myfile");
InputStream inputFile = new FileInputStream(file);
InputStream inputWeb = conn.getInputStream();
while((ch=inputFile.read()) >= 0) {
outputWeb.write(ch);
}
causes IOException with message "Server returned HTTP response code: 500 for URL: myurl" just at the line where InputStream is taken from the connection.
While code below does not cause any exceptions and works fine
OutputStream outputWeb = conn.getOutputStream();
File file = new File("myfile");
InputStream inputFile = new FileInputStream(file);
while((ch=inputFile.read()) >= 0) {
outputWeb.write(ch);
}
InputStream inputWeb = conn.getInputStream();
The only difference is that taking input stream located after some writing to connection.
How one can deduce this from API contract? What is happening and why?
conn configuration
URL url = new URL("https://myurl");
HttpURLConnection conn;
// creating http connection
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestProperty("Content-type", "audio....");
UPDATE
It is clear in principle now, but I still wish to see this in API contract. I.e. it should be written somewhere, that I can't gen input stream until write something. Implementations can allow this in any case, just obliging readers of that stream to wait.
i'm stuck on this process from two days, before posting i've searched a lot of topic and looks like it's a so simple issue. But i didn't get the problem.
Scenario is basic: i want to parse an XML from a remote computer through http connection:
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
try {
URL url = new URL("http://host:port/file.xml");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept","application/xml");
InputStream is = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
PrintWriter pw = new PrintWriter("localfile_pw.xml");
FileOutputStream fos = new FileOutputStream("localfile_os.xml");
Then i tried three different ways to read the XML
Reading byte stream
byte[] buffer = new byte[4 * 1024];
int byteRead;
while((byteRead= is.read(buffer)) != -1){
fos.write(buffer, 0, byteRead);
}
Reading charachter per character
char c;
while((c = (char)br.read()) != -1){
pw.print(c);
System.out.print(c);
}
Reading line per line
String line = null;
while((line = br.readLine()) != null){
pw.println(line);
System.out.println(line);
}
In all cases my xml reading stops at the same point, after the same exact nuumber of bytes. And gets stuck without reading and without giving any exception.
Thanks in advance.
How about this (see IOUtils from Apache):
URL url = new URL("http://host:port/file.xml");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept","application/xml");
InputStream is = connection.getInputStream();
FileOutputStream fos = new FileOutputStream("localfile_os.xml");
IOUtils.copy(is, fos);
is.close();
fos.close();
The class supports persistent HTTP connections by default. If the size of the response is know at the time of the response, after it sends your data, the server will wait for another request. There are 2 ways of handling this:
Read the content-length:
InputStream is = connection.getInputStream();
String contLen = connection.getHeaderField("Content-length");
int numBytes = Integer.parse(contLen);
Read numBytes bytes from the input stream. Note: contLen may be null, in this case you should read until EOF.
Disable connection keep alive:
connection.setRequestProperty("Connection","close");
InputStream is = connection.getInputStream();
After sending the last byte of data the server will close the connection.
[Java 1.5; Eclipse Galileo]
HttpsURLConnection seems to stall when the getInputStream() method is called. I've tried using different websites to no avail (currently https://www.google.com). I should point out I'm using httpS.
The code below has been modified based on what I've learned from other StackOverflow answers. However, no solutions I've tried thus far have worked.
I'd be very grateful for a nudge in the right direction :)
public static void request( URL url, String query )
{
try{
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
//connection.setReadTimeout( 5000 ); //<-- uncommenting this line at least allows a timeout error to be thrown
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
System.setProperty("http.keepAlive", "false");
connection.setRequestMethod( "POST" );
// setting headers
connection.setRequestProperty("Content-length",String.valueOf (query.length()));
connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded"); //WAS application/x-www- form-urlencoded
connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)");
////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////
System.out.println( "THIS line stalls" + connection.getInputStream() );
////////////////////////////////////////////////////////////////////////////////////
}catch( Exception e ) {
System.out.println( e );
e.printStackTrace();
}
Typical errors look like:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:782)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:739)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:681)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:626)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:983)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:234)
at https_understanding.HTTPSRequest.request(HTTPSRequest.java:60)
at https_understanding.Main.main(Main.java:17)
connection.setDoOutput(true);
This means that you have to open, write to, and close the connection's output stream before you attempt to read from its input stream. See the docs.
I reproduced the problem in Android 2.2: when downloading from a web-server over wireless and a HTTPS URL, the error is a socket "read time out" at URLConnection.getInputStream()
To fix it, use url.openStream() for the InputStream instead of connection.getInputStream()
Bonus: you can get the length of the file you're downloading so you can show a % complete indicator
code sample:
private final int TIMEOUT_CONNECTION = 5000;//5sec
private final int TIMEOUT_SOCKET = 30000;//30sec
file = new File(strFullPath);
URL url = new URL(strURL);
URLConnection ucon = url.openConnection();
//this timeout affects how long it takes for the app to realize there's a connection problem
ucon.setReadTimeout(TIMEOUT_CONNECTION);
ucon.setConnectTimeout(TIMEOUT_SOCKET);
//IMPORTANT UPDATE:
// ucon.getInputStream() often times-out over wireless
// so, replace it with ucon.connect() and url.openStream()
ucon.connect();
iFileLength = ucon.getContentLength();//returns -1 if not set in response header
if (iFileLength != -1)
{
Log.i(TAG, "Expected Filelength = "+String.valueOf(iFileLength)+" bytes");
}
//Define InputStreams to read from the URLConnection.
// uses 5KB download buffer
InputStream is = url.openStream();//ucon.getInputStream();
BufferedInputStream inStream = new BufferedInputStream(is, 1024 * 5);
outStream = new FileOutputStream(file);
bFileOpen = true;
byte[] buff = new byte[5 * 1024];
//Read bytes (and store them) until there is nothing more to read(-1)
int total=0;
int len;
int percentdone;
int percentdonelast=0;
while ((len = inStream.read(buff)) != -1)
{
//write to file
outStream.write(buff,0,len);
//calculate percent done
if (iFileLength != -1)
{
total+=len;
percentdone=(int)(total*100/iFileLength);
//limit the number of messages to no more than one message every 10%
if ( (percentdone - percentdonelast) > 10)
{
percentdonelast = percentdone;
Log.i(TAG,String.valueOf(percentdone)+"%");
}
}
}
//clean up
outStream.flush();//THIS IS VERY IMPORTANT !
outStream.close();
bFileOpen = false;
inStream.close();
Also don't set the content-length header. Java will do that for you.