Line InputStream is = new GZIPInputStream (con.getInputStream ()); returns this "java.util.zip.ZipException: Not in GZIP format" exception. Does anyone know how to solve this?
my code:
private String getJsonFromRapidAPI(final String url) throws Exception {
final String token = generateSessionToken();
final HttpClient httpclient = new DefaultHttpClient();
final HttpGet httpget = new HttpGet(url);
String fileContents = null;
StringBuilder sb = new StringBuilder();
BufferedReader in = null;
if (inetAddress == null) {
inetAddress = InetAddress.getLocalHost();
}
final String serverIP = inetAddress.getHostAddress();
URL urlToCall = new URL(url);
HttpURLConnection con = (HttpURLConnection) urlToCall.openConnection();
con.setRequestProperty("Accept", "application/json");
con.setRequestProperty("Accept-Encoding", "gzip");
con.setRequestProperty("Authorization", token);
con.setRequestProperty("Customer-Ip", serverIP);
con.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36");
InputStream is = new GZIPInputStream(con.getInputStream());
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
return writer.toString();
}
Two issues here:
You can tell a server you want GZIP encoding. That doesn't mean it'll neccessarily comply.
As a consequence, your current code goes: Well, I asked for gzip, so I will assume i Must gunzip the stream. That's incorrect - you need to ask for gzip as you do, but you need to let 'should I GZipInputStream the response' depend on whether the server actually did that. To figure this out, use con.connect to actually connect (.getInputStream implies it, but now we need to fetch some response info before opening the inputstream so you need to explicitly connect now), and you can call ."gzip".equals(con.getContentEncoding()) to figure it out. If that is true, then wrap con.getInputStream() in GZIPInputStream. If it isn't, don't do that.
More generally you're abusing HttpURLConnection.
HttpURLConnection's job is to deal with the stream/processing of the data itself. That means headers like Transfer-Encoding, Host, and Accept-Encoding aren't 'your turf', you shouldn't be messing with this stuff. It needs to send 'hey, I can support gzip!' and it needs to wrap the response in a GZIPInputStream if the server says: "Great, okay, therefore I gzipped it for you!".
Unfortunately HttpURLConnection is extremely basic and messes up this distinction itself, for example it looks at the Transfer-Encoding header you set and will chunk sends (I think - oof, I haven't used this class in forever because its so bad).
If you have needs to make things even slightly complicated (and I'd say this qualifies), then stop using it. There's OkHttp from square. Java itself (Available in JDK11 and up, I think) calls HttpURLConnection obsolete and has replaced it - here is a tutorial, and even apache has HttpComponents but note that this is outdated and as with most apache java libraries, oof, it's bad API design. I wouldn't use that one, even if it is rather popular. It's still much better than HttpURLConnection though!
Related
I have a short Android-Java client program which sends a basic information to bottle-python server with POST method. In the first version of code, server does not show anything. However, In the second version it works but I cannot understand what this additional line do because it has anything to do with posting content. I really appreciate if someone helps me figure this out.(There is nothing wrong with the server code since I can properly send request with python requests and my browsers).
This is the first version of client code:
String url = "http://192.168.1.23:8080/";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setDoOutput(true);
OutputStream os = con.getOutputStream();
PrintStream myPusher = new PrintStream(os );
myPusher.print("param1=hey");
Second version:
String url = "http://192.168.1.23:8080/";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setDoOutput(true);
OutputStream os = con.getOutputStream();
PrintStream myPusher = new PrintStream(os );
myPusher.print("param1=hey");
InputStream in= con.getInputStream(); //Nothing changed but only this additional line
Bottle(python) server:
#app.route('/', method="POST")
def hello():
print("it works")
name = request.forms.get("param1")
print(name)
return name
#app.route('/')
def hello():
i=0
print("it works")
run(app, host="192.168.1.23", port=8080)
With first client code server shows nothing.
With second code server shows:
it works
hey
192.168.1.24 - - [31/Dec/2018 17:10:28] "POST / HTTP/1.1" 200 3
Which is as I expected.
With your first code snippet the output stream is still open. So the server does not know if it got the complete request. Probably just closing the stream would work as well.
However, I would make at least a call to getResponseCode to see the outcome of the request.
Your java code seems incomplete for sending a post request.
I think by using this code, you can make it work for yourself.
The PrintStream is a buffered type, this means you should add a flush operation after each print(), or use println() instead.
I'm going to keep this question short and sweet. I have a function that takes a URL to read as a string and returns a string of the HTML source of a webpage. Here it is:
public static String getHTML(String urlToRead) throws Exception // Returns the source code of a given URL.
{
StringBuilder result = new StringBuilder();
URL url = new URL(urlToRead);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36");
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream(), Charset.forName("UTF-8")));
String line;
while ((line = rd.readLine()) != null)
{
result.append(line + System.getProperty("line.separator"));
}
rd.close();
result.toString();
}
It works like a charm, with the exception of one tiny quirk. Certain characters are not being read correctly by the InputStreamReader. The "ł" character isn't correctly read, and is instead replaced by a "?". That's the only character I've found thus far that follows this behaviour but there's no telling what other characters aren't being read correctly.
It seems like an issue with the character set. I'm using UTF-8 as you can see from the code. All other character sets I've tried using in its place have either outright not worked or have had trouble with far more than just one character.
What kind of thing could be causing this issue? Any help would be greatly appreciated!
Have you tried :
conn.setRequestProperty("content-type", "text/plain; charset=utf-8");
You should use the same charset as the resource you read. First make sure what is the encoding used by that HTML. Usually its content type is sent in response header. You can easily get this information using any web browser with network tracking (since you have GET request).
For example using Chrome - open empty tab, open dev tools (F12), and load desired web page. Than you can look at network tab in dev tools and examine response headers.
I've read lots and tried lots relating to HTTP POSTS using HttpURLConnection and almost everything I come across has a similar structure which starts with these 3 lines:
url = new URL(targetURL);
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
When I try this I always get a "Connection Already Established" exception when calling setRequestMethod, which makes perfect sense as I'm clearly calling openConnection before setting the request type. Although reading the docs openConnection doesn't actually open the connection in theory.
There are several posts about this problem on SO such as this and this. I don't understand however why every piece of advice about how to write this code has these 3 lines in this order.
I'm guessing this code must work in most instances as someone must have tested it, so why doesn't this code work for me? How should I be writing this code?
I am aware these are other libraries I can use out there, I'm just wondering why this doesn't work.
Why the suspect code in the question has been duplicated all over the internet is something I can't answer. Nor can I answer why it seems to work for some people and not others. I can however answer the other question now, mainly thanks to this link that Luiggi pointed me to.
The key here is understanding the intricacies of the HttpURLConnection class. When first created the class defaults to a "GET" request method, so nothing needs to be changed in this instance. The following is rather unintuitive, but to set the request method to "POST" you should not call setRequestMethod("POST"), but rather setDoOutput(true) which implicitly sets the request method to post. Once you've done that you're good to go.
Below, I believe, is what a post method should look like. This is for posting json, but can obviously be altered for any other content type.
public static String doPostSync(final String urlToRead, final String content) throws IOException {
final String charset = "UTF-8";
// Create the connection
HttpURLConnection connection = (HttpURLConnection) new URL(urlToRead).openConnection();
// setDoOutput(true) implicitly set's the request type to POST
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestProperty("Content-type", "application/json");
// Write to the connection
OutputStream output = connection.getOutputStream();
output.write(content.getBytes(charset));
output.close();
// Check the error stream first, if this is null then there have been no issues with the request
InputStream inputStream = connection.getErrorStream();
if (inputStream == null)
inputStream = connection.getInputStream();
// Read everything from our stream
BufferedReader responseReader = new BufferedReader(new InputStreamReader(inputStream, charset));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = responseReader.readLine()) != null) {
response.append(inputLine);
}
responseReader.close();
return response.toString();
}
As per https://stackoverflow.com/a/3324964/436524, you need to call connection.setDoOutput(true) for it to expect a POST request.
This makes your code like this:
url = new URL(targetURL);
connection = (HttpURLConnection)url.openConnection();
connection.setDoOutput(true);
I am going to send sms via java. The problem is the sms gateway ask me to send in this format
http://push1.maccesssmspush.com/servlet/com.aclwireless.pushconnectivity.listen
ers.TextListener?userId=xxxxx&pass=xxxx&appid=xxxx&subappid=xxxx&msgtyp
e=1&contenttype=1&selfid=true&to=9810790590,9810549717&from=ACL&dlrre
q=true&text=This+is+a+test+msg+from+ACL&alert=
The problem how to call this from a java application is it possible or does it need special libraries? IS it using HttpURLConnection will do the job? Thank you.
A Sample code I have done below is this correct.
URL sendSms1 = new URL("http://push1.maccesssmspush.com/servlet/com.aclwireless.pushconnectivity.listen
ers.TextListener?userId=xxxxx&pass=xxxx&appid=xxxx&subappid=xxxx&msgtyp
e=1&contenttype=1&selfid=true&to=9810790590,9810549717&from=ACL&dlrre
q=true&text=This+is+a+test+msg+from+ACL&alert=");
URLConnection smsConn1 =
sendSms1.openConnection();
It's just an HTTP call, you don't need anything special in Java (or any modern language, I expect). Just build up the string as appropriate*, then make an HTTP request to that URL.
Take a peek at the Sun tutorial Reading from and Writing to a URLConnection if you need to pick up the basics of how to do the request part in Java. This uses the built-in classes, I'm sure there are dozens of libraries that handles connections in funky and/or convenient ways too, so by all means use one of those if you're familiar with it.
*One potential gotcha which might not have occurred to you - your query string arguments will have to be URL-encoded. So the + characters for example in the text parameter, are encoded spaces (which would have a different meaning in the URL). Likewise, if you wanted to send a ? character in one of your parameters, it would have to appear as %3F. Have a look at the accepted answer to HTTP URL Address Encoding in Java for an example of how you might build the URL string safely.
It looks like a simple GET request, you can use Apache HttpClient libarary for executing such a request. Have a look into a tutorial by Vogella here: http://www.vogella.de/articles/ApacheHttpClient/article.html for sample source code and explanations.
You can try to use java.net.URL library。
like this
// at this before you need to generate the urlString as "http://push1.maccesssmspush.com/servlet/com.aclwireless.pushconnectivity.listen
ers.TextListener?userId=xxxxx&pass=xxxx&appid=xxxx&subappid=xxxx&msgtyp
e=1&contenttype=1&selfid=true&to=9810790590,9810549717&from=ACL&dlrre
q=true&text=This+is+a+test+msg+from+ACL&alert="
URL url = new URL(urlString);
// send sms
URLConnection urlConnection = url.openConnection();// open the url
// and you, also can get the feedback if you want
BufferedReader br = new BufferedReader(new InputStreamReader(
urlConnection.getInputStream()));
URL url = new URL("http://smscountry.com/SMSCwebservice.asp");
HttpURLConnection urlconnection = (HttpURLConnection) url.openConnection();
[Edit]
urlconnection.setRequestMethod("POST");
urlconnection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
urlconnection.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(urlconnection.getOutputStream());
out.write(postData);
out.close();
BufferedReader in = new BufferedReader(new InputStreamReader(urlconnection.getInputStream()));
String decodedString;
while ((decodedString = in.readLine()) != null) {
retval += decodedString;
}
I am currently trying to get HttpComponents to send HttpRequests and retrieve the Response.
On most URLs this works without a problem, but when I try to get the URL of a phpBB Forum namely http://www.forum.animenokami.com the client takes more time and the responseEntity contains passages more than once resulting in a broken html file.
For example the meta tags are contained six times. Since many other URLs work I can't figure out what I am doing wrong.
The Page is working correctly in known Browsers, so it is not a Problem on their side.
Here is the code I use to send and receive.
URI uri1 = new URI("http://www.forum.animenokami.com");
HttpGet get = new HttpGet(uri1);
get.setHeader(new BasicHeader("User-Agent", "Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0"));
HttpClient httpClient = new DefaultHttpClient();
HttpResponse response = httpClient.execute(get);
HttpEntity ent = response.getEntity();
InputStream is = ent.getContent();
BufferedInputStream bis = new BufferedInputStream(is);
byte[] tmp = new byte[2048];
int l;
String ret = "";
while ((l = bis.read(tmp)) != -1){
ret += new String(tmp);
}
I hope you can help me.
If you need anymore Information I will try to provide it as soon as possible.
This code is completely broken:
String ret = "";
while ((l = bis.read(tmp)) != -1){
ret += new String(tmp);
}
Three things:
This is converting the whole buffer into a string on each iteration, regardless of how much data has been read. (I suspect this is what's actually going wrong in your case.)
It's using the default platform encoding, which is almost never a good idea.
It's using string concatenation in a loop, which leads to poor performance.
Fortunately you can avoid all of this very easily using EntityUtils:
String text = EntityUtils.toString(ent);
That will use the appropriate character encoding specified in the response, if any, or ISO-8859-1 otherwise. (There's another overload which allows you to specify which character encoding to use if it's not specified.)
It's worth understanding what's wrong with your original code though rather than just replacing it with the better code, so that you don't make the same mistakes in other situations.
It works fine but what I don't understand is why I see the same text multiple times only on this URL.
It will be because your client is seeing more incomplete buffers when it reads the socket. Than could be:
because there is a network bandwidth bottleneck on the route from the remote site to your client,
because the remote site is doing some unnecessary flushes, or
some other reason.
The point is that your client must pay close attention to the number of bytes read into the buffer by the read call, otherwise it will end up inserting junk. Network streams in particular are prone not filling the buffer.