I am trying to do REST request on the StackExchange API on java and I have some problems to read the output. I made a small code example to see if I can read properly:
package Utils;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class RestClient
{
public static void main(String[] args)
{
try
{
URL url = new URL("https://api.stackexchange.com/2.2/questions?order=desc&sort=activity&site=stackoverflow");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
System.out.println(conn.getResponseCode());
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String output;
while ((output = reader.readLine()) != null)
{
System.out.println(output);
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
Below is a sample output from the website:
{"items":[{"tags":["c++","templates","covariant"],"owner":{"reputation":3,"user_id":7447292,"user_type":"registered","profile_image":"https://www.gravatar.com/avatar/8979f1b328b9b0f786dec8b4edd514bc?s=128&d=identicon&r=PG&f=1","display_name":"B.Oudot","link":"http://stackoverflow.com/users/7447292/b-oudot"}
However when I print the output of my java program, I get the status code 200 and then the rest is just bunch of non-ascii characters. I am new with REST and JSON. I would like to not use any 3rd party library if possible.
EDIT
I have put the output of my program as an image.
you need to use GZIPInputStream api, when the content encoding used gZip. it is also an inputstream.
i used gipinput stream to your response and i see output in json.
public static void main(String[] args){
try
{
URL url = new URL("https://api.stackexchange.com/2.2/questions?order=desc&sort=activity&site=stackoverflow");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
System.out.println(conn.getResponseCode());
GZIPInputStream gzis = new GZIPInputStream(conn.getInputStream());
InputStreamReader reader = new InputStreamReader(gzis);
BufferedReader in = new BufferedReader(reader);
String readed;
while ((readed = in.readLine()) != null) {
System.out.println(readed);
}
//BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String output;
while ((output = in.readLine()) != null)
{
System.out.println(output);
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
it worked for me.
Related
I want to send JSON Data to an application using HTTPS connection from java code. please guide me.
`import java.net.URL;
import java.io.*;
import javax.net.ssl.HttpsURLConnection;
public class JavaHttpsExample
{
public static void main(String[] args) throws Exception {
String httpsURL = "https://your.https.url.here/";
URL myUrl = new URL(httpsURL);
HttpsURLConnection conn = (HttpsURLConnection)myUrl.openConnection();
InputStream is = conn.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String inputLine;
while ((inputLine = br.readLine()) != null) {
System.out.println(inputLine);
}
br.close();
}
}`
I am getting response from Wikipedia page and paste the response in html file. If I open the html file in browser I am not able to get the languages other than English as it is (I used UTF-8). I am attaching the picture of languages as in html.
I tried in couple of ways to get the response using java, and they are as follows,
Way 1,
URL url = new URL ("https://en.wikipedia.org/wiki/Sachin_Tendulkar");
byte[] encodedBytes = Base64.encodeBase64("root:pass".getBytes());
//System.out.println("Host --------"+url.getHost());
String encoding = new String (encodedBytes);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "text/xml; charset=UTF-8");
connection.setDoInput (true);
connection.setRequestProperty ("Authorization", "Basic " + encoding);
connection.connect();
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
String s = line.toString();
System.out.println(s);
}
I also tried the following code, but this also not showing the fonts as it is in
wiki,
URL url;
HttpURLConnection conn;
BufferedReader rd;
String line;
StringBuilder result = new StringBuilder();
try {
url = new URL("https://en.wikipedia.org/wiki/Sachin_Tendulkar");
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept-Charset", "UTF-8");
conn.setRequestProperty("Content-Type", "text/xml; charset=UTF-8");
rd = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
while ((line = rd.readLine()) != null) {
byte [] b = line.getBytes("UTF-8");
result.append(line);
System.out.println(result.append(line));
}
rd.close();
} catch (Exception e) {
e.printStackTrace();
}
Couple of points:
Your code does not show how exactly you persist the response to the HTML file. Do you just redirect the standard output of the process to a file? Make sure you use UTF-8 even while writing to the output file.
Why do you System.out.println the whole StringBuffer instance in each iteration of the read loop?
Why do you call line.getBytes() and never use the output?
EDIT - Based on your comments, I really think the problem is with the clipboard manipulation. Try the code below, which stores the response directly to an output file.
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
public class HtmlDownloader {
private static final String USER_AGENT = "Mozilla/5.0";
private static final String ENCODING = "UTF-8";
public boolean download(String urlAddress, String outputFileName) {
HttpURLConnection con = null;
BufferedInputStream is = null;
BufferedOutputStream os = null;
try {
URL url = new URL(urlAddress);
con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Accept-Charset", ENCODING);
is = new BufferedInputStream(
con.getInputStream()
);
os = new BufferedOutputStream(
new FileOutputStream(outputFileName)
);
byte[] buffer = new byte[1024];
int len;
while ((len = is.read(buffer)) >= 0) {
os.write(buffer, 0, len);
}
} catch (Exception e) {
e.printStackTrace();
return false;
} finally {
if (is != null) {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (os != null) {
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return true;
}
public static void main(String[] args) {
HtmlDownloader d = new HtmlDownloader();
if (d.download("https://en.wikipedia.org/wiki/Sachin_Tendulkar", "c:\\wiki.html"))
System.out.println("SUCCESS");
else
System.out.println("FAIL");
}
}
I am new to this type of problems. Here I want to make a request to a website API and get response in JSON format.
I then want to send this response directly to a a different remote URL.
See the below sample code
PostMethod post = new PostMethod("Your URL");
post.setRequestBody("your json data");
post.setRequestHeader("Content-type", "application/json;charset=utf-8");
// Get HTTP client
HttpClient httpclient = new HttpClient();
// Execute request
try
{
int result = httpclient.executeMethod(post);
s_log.info("Response status code: " + result);
s_log.info(post.getResponseBodyAsString());
}
catch (HttpException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
post.releaseConnection();
}
Reading from and writing to a remote URL can be done using the standard Java API and is detailed in the official Java tutorials:
https://docs.oracle.com/javase/tutorial/networking/urls/readingWriting.html
Reading:
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static void main(String[] args) throws Exception {
URL oracle = new URL("http://www.oracle.com/");
URLConnection yc = oracle.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(
yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
}
}
Writing
import java.io.*;
import java.net.*;
public class Reverse {
public static void main(String[] args) throws Exception {
if (args.length != 2) {
System.err.println("Usage: java Reverse "
+ "http://<location of your servlet/script>"
+ " string_to_reverse");
System.exit(1);
}
String stringToReverse = URLEncoder.encode(args[1], "UTF-8");
URL url = new URL(args[0]);
URLConnection connection = url.openConnection();
connection.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(
connection.getOutputStream());
out.write("string=" + stringToReverse);
out.close();
BufferedReader in = new BufferedReader(
new InputStreamReader(
connection.getInputStream()));
String decodedString;
while ((decodedString = in.readLine()) != null) {
System.out.println(decodedString);
}
in.close();
}
}
I have a http end point which serves xml responses to my queries. I tried to fire the http requests like this -
import java.io.*;
import java.net.*;
public class c {
public String getHTML(String urlToRead) {
URL url;
HttpURLConnection conn;
BufferedReader rd;
String line;
String result = "";
try {
url = new URL(urlToRead);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
// how to store the response as an xml file on disc
rd.close();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
public static void main(String args[])
{
c c = new c();
System.out.println(c.getHTML(args[0]));
}
}
Now, the endpoint sends xml response which i want to store as a xml file on disc. how can i do that. can someone help?
I have not tried it, but maybe you could start from something like this:
String dest = "c:\\filename.xml";
// this line substitutes your rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
InputStream is = conn.getInputStream();
// these others goes in your "how to store..." comment
byte[] bytes = IOUtils.toByteArray(is);
FileOutputStream fos = new FileOutputStream(dest);
fos.write(bytes);
fos.flush();
fos.close();
I am trying to read source code from a webpage. My java code is
import java.net.*;
import java.io.*;
import java.util.*;
import javax.swing.JOptionPane;
class Testing{
public static void Connect() throws Exception{
URL url = new URL("http://excite.com/education");
URLConnection spoof = url.openConnection();
spoof.setRequestProperty( "User-Agent", "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; H010818)" );
BufferedReader in = new BufferedReader(new InputStreamReader(spoof.getInputStream()));
String strLine = "";
while ((strLine = in.readLine()) != null){
System.out.println(strLine);
}
System.out.println("End of page.");
}
public static void main(String[] args){
try{
Connect();
}catch(Exception e){
}
}
When i compile and run this code, it gives the following output:
�I�%&/m�{J�J��t�$ؐ#������iG#)�*��eVe]f#�흼��{���{���;�N'���?\fdl��J�ɞ!���?~|?"~�$}�>�������4�����7N�����+�ӲM�N��?J�tZfM��G�j����R��!�9�?>JgE��Ge[����ⳏ���W�?�����8������
�|8�
���������ho����0׳���|փ:--�|�L�Uο���m�zt�n3��l\�w��O^f�G[�CG<�y6K��gM�rg��ǟy�Eִy����h˜��ؗ˲X���l=�ڢZ�/����(կ^O�UU6�����&�6_�#yC}�p�y���lAH�ͯ��zF#�V�6_��}��)�v=J+�$��̤�G�Y�L�b���wS"�7�y^����Z�m���Y:ɛ���J<N_�Y=���U�f���,���y�Q2(J٩P!ͨ�i����1&F0&ૼn�?�x�T��h�Qzw�+����n�)�h��K��2����8g����⮥��A0
���1I�%����Q�Z����{��������w���?x����N�?�<d�S��۫�%a|4�j��z���k�Bak��k-�c�z�g��z���l>���֎s^,��5��/B�{����]]����Ý�ֳ���y{�_l�8g�k�ӫ�b���"+|��(��M��^[���J�P��_�..?������x�Z�$������E>��느�u���E~����{媘���f�e1ͷ�QZ,�����f��e�3Jٻb�^��4��۴���>��y��;��<렛{�l��ZfW
S# {�]��1��Q�����n[�,t�?����~�n�S�u#SL��n�^��������EC��q�/�y���FE�tpm������e&��oB���z9eY��������P��IK?����̦����w�N��;�;J?����;�/��5���M���rZ��q��]��C�dᖣ��F�nd���}���A5���M�5�.�:��/�_D�?�3����'�c�Z7��}��(OI),ۏi����{�<�w�������DZ?e����'q���eY]=���kj���߬������\qhrRn���l�o-��.���k��_���oD8��GA�P�r��|$��ȈPv~Y�:�[q?�sH�� <��C��ˬ�^N�[ v(��S��l�c�C����3���E5&5�VӪL�T��۔���oQrĈ��/���#[f�5�5"����[���t�vm�\��.0�nh����aڌWYM
^T�|\,��퓜�L�u����B�̌�C�r������ �������'�%�{��)�);�fV�]��g,�>�C �c2���p�4��}H���P��(�%j"�}�&�:�Oh\5I�l�氪��{�/�]�LB�l��2��I"��=��Y�|�>�֏n�������}�����~�[��'��O��
��:/�)�Wz�3��lo�.5�k�&����H[ji�����b������WWy}�5�֝Q�|f�����]�KjH5��}yNm�����g�ӷ�ǣ��>��'o��泏��<���G�g���>->�xQM�����%<�|����u�.��3���[�[r���ٝ;���]4E��6[����]����1���*�8}��n�w�������ݽ����|����}|qo|�~u����w|�i�i���Z�`z�ŧ����Q}�u��!���w �O���R9�)�~��g~w6��{���wd�o��/Z�uUS��݄l��I^�����>��[�U1�o�_��J��}��#�#�U�/��/?���i�7|CZT?(�2b~����c�W�c5'����EeFĿꇙ�0��T��{��W�2����/���O���YJj����K/���>��:'_l�
Other than URLs from this directory i.e. "excite.com/education" all URLs are giving correct source codes but these URLs are creating problems.
Anyone Please Help.
Thanks in advance.
It works for me.
private static String getWebPabeSource(String sURL) throws IOException {
URL url = new URL(sURL);
URLConnection urlCon = url.openConnection();
BufferedReader in = null;
if (urlCon.getHeaderField("Content-Encoding") != null
&& urlCon.getHeaderField("Content-Encoding").equals("gzip")) {
in = new BufferedReader(new InputStreamReader(new GZIPInputStream(
urlCon.getInputStream())));
} else {
in = new BufferedReader(new InputStreamReader(
urlCon.getInputStream()));
}
String inputLine;
StringBuilder sb = new StringBuilder();
while ((inputLine = in.readLine()) != null)
sb.append(inputLine);
in.close();
return sb.toString();
}
Try reading it this way:
private static String getUrlSource(String url) throws IOException {
URL url = new URL(url);
URLConnection urlConn = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(
urlConn.getInputStream(), "UTF-8"));
String inputLine;
StringBuilder a = new StringBuilder();
while ((inputLine = in.readLine()) != null)
a.append(inputLine);
in.close();
return a.toString();
}
and set your encoding according to the web page - notice this line:
BufferedReader in = new BufferedReader(new InputStreamReader(
urlConn.getInputStream(), "UTF-8"));
First you have to uncompress the content using GZIPInputStream. Then put the uncompressed stream to Input Stream and read it using BufferedReader
Use Apache HTTP Client 4.1.1
Maven dependency
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.1.1</version>
</dependency>
Sample Code to parse gzip content.
package com.gzip.simple;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.zip.GZIPInputStream;
import org.apache.http.Header;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class GZIPFetcher {
public static void main(String[] args) {
try {
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet getRequest = new HttpGet("http://excite.com/education");
getRequest.addHeader("accept", "application/json");
HttpResponse response = httpClient.execute(getRequest);
if (response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ response.getStatusLine().getStatusCode());
}
InputStream instream = response.getEntity().getContent();
// Check whether the content-encoding is gzip or not.
Header contentEncoding = response
.getFirstHeader("Content-Encoding");
if (contentEncoding != null
&& contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
BufferedReader in = new BufferedReader(new InputStreamReader(
instream));
String content;
System.out.println("Output from Server .... \n");
while ((content = in.readLine()) != null)
System.out.println(content);
httpClient.getConnectionManager().shutdown();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}