I am getting response from Wikipedia page and paste the response in html file. If I open the html file in browser I am not able to get the languages other than English as it is (I used UTF-8). I am attaching the picture of languages as in html.
I tried in couple of ways to get the response using java, and they are as follows,
Way 1,
URL url = new URL ("https://en.wikipedia.org/wiki/Sachin_Tendulkar");
byte[] encodedBytes = Base64.encodeBase64("root:pass".getBytes());
//System.out.println("Host --------"+url.getHost());
String encoding = new String (encodedBytes);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "text/xml; charset=UTF-8");
connection.setDoInput (true);
connection.setRequestProperty ("Authorization", "Basic " + encoding);
connection.connect();
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
String s = line.toString();
System.out.println(s);
}
I also tried the following code, but this also not showing the fonts as it is in
wiki,
URL url;
HttpURLConnection conn;
BufferedReader rd;
String line;
StringBuilder result = new StringBuilder();
try {
url = new URL("https://en.wikipedia.org/wiki/Sachin_Tendulkar");
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept-Charset", "UTF-8");
conn.setRequestProperty("Content-Type", "text/xml; charset=UTF-8");
rd = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
while ((line = rd.readLine()) != null) {
byte [] b = line.getBytes("UTF-8");
result.append(line);
System.out.println(result.append(line));
}
rd.close();
} catch (Exception e) {
e.printStackTrace();
}
Couple of points:
Your code does not show how exactly you persist the response to the HTML file. Do you just redirect the standard output of the process to a file? Make sure you use UTF-8 even while writing to the output file.
Why do you System.out.println the whole StringBuffer instance in each iteration of the read loop?
Why do you call line.getBytes() and never use the output?
EDIT - Based on your comments, I really think the problem is with the clipboard manipulation. Try the code below, which stores the response directly to an output file.
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
public class HtmlDownloader {
private static final String USER_AGENT = "Mozilla/5.0";
private static final String ENCODING = "UTF-8";
public boolean download(String urlAddress, String outputFileName) {
HttpURLConnection con = null;
BufferedInputStream is = null;
BufferedOutputStream os = null;
try {
URL url = new URL(urlAddress);
con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Accept-Charset", ENCODING);
is = new BufferedInputStream(
con.getInputStream()
);
os = new BufferedOutputStream(
new FileOutputStream(outputFileName)
);
byte[] buffer = new byte[1024];
int len;
while ((len = is.read(buffer)) >= 0) {
os.write(buffer, 0, len);
}
} catch (Exception e) {
e.printStackTrace();
return false;
} finally {
if (is != null) {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (os != null) {
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return true;
}
public static void main(String[] args) {
HtmlDownloader d = new HtmlDownloader();
if (d.download("https://en.wikipedia.org/wiki/Sachin_Tendulkar", "c:\\wiki.html"))
System.out.println("SUCCESS");
else
System.out.println("FAIL");
}
}
Related
Here is my code:
public String readTheUrl(String place) throws IOException {
String data = "";
InputStream inputStream = null;
HttpURLConnection httpURLConnection = null;
try {
URL url = new URL(place);
httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setConnectTimeout(10000);
httpURLConnection.setReadTimeout(10000);
httpURLConnection.setRequestMethod("GET");
httpURLConnection.setUseCaches(false);
httpURLConnection.setAllowUserInteraction(false);
httpURLConnection.connect();
int response=httpURLConnection.getResponseCode();
inputStream = httpURLConnection.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
StringBuffer stringBuffer = new StringBuffer();
String line = "";
while ((line = (bufferedReader.readLine())) != null) {
stringBuffer.append(line);
}
data = stringBuffer.toString();
bufferedReader.close();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
inputStream.close();
httpURLConnection.disconnect();
return data;
}
When I am using this code for loading other url, it is working perfectly, but in case of
"https://api-crt.cert.havail.sabre.com/v1/shop/flights?origin=FRA&destination=DFW&departuredate=2019-10-28&returndate=2019-11-10&pointofsalecountry=DE"
it always return null. I tested this api with postman, and I loaded JSON file. Is there any problem with url or it needs some specific loading? Thank you!
I am trying to make a post from Java to make a market order using my Bitstamp account but the following code is returning a file not found for the URL.
It may be because of CSRF but I am unsure, if anyone has had any experience with the bitstamp API that would be great.
public static void postToken() throws IOException, JSONException {
URL url = null;
String sig = encode();
try {
url = new URL("https://www.bitstamp.net/api/v2/buy/market/" + feedbackType.toLowerCase() +"usd/");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setConnectTimeout(5000);//5 secs
connection.setReadTimeout(5000);//5 secs
connection.setRequestMethod("POST");
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
JSONObject cred = new JSONObject();
cred.put("key",api_key);
cred.put("signature", sig);
cred.put("nonce", nonce);
cred.put("amount", feedback);
OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());
out.write(cred.toString());
out.flush();
out.close();
int res = connection.getResponseCode();
System.out.println(res);
InputStream is = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
String line = null;
while((line = br.readLine() ) != null) {
Log.d(TAG, line);
}
connection.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
Error: W/System.err: java.io.FileNotFoundException: https://www.bitstamp.net/api/v2/buy/market/btcusd/
I am trying to do REST request on the StackExchange API on java and I have some problems to read the output. I made a small code example to see if I can read properly:
package Utils;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class RestClient
{
public static void main(String[] args)
{
try
{
URL url = new URL("https://api.stackexchange.com/2.2/questions?order=desc&sort=activity&site=stackoverflow");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
System.out.println(conn.getResponseCode());
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String output;
while ((output = reader.readLine()) != null)
{
System.out.println(output);
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
Below is a sample output from the website:
{"items":[{"tags":["c++","templates","covariant"],"owner":{"reputation":3,"user_id":7447292,"user_type":"registered","profile_image":"https://www.gravatar.com/avatar/8979f1b328b9b0f786dec8b4edd514bc?s=128&d=identicon&r=PG&f=1","display_name":"B.Oudot","link":"http://stackoverflow.com/users/7447292/b-oudot"}
However when I print the output of my java program, I get the status code 200 and then the rest is just bunch of non-ascii characters. I am new with REST and JSON. I would like to not use any 3rd party library if possible.
EDIT
I have put the output of my program as an image.
you need to use GZIPInputStream api, when the content encoding used gZip. it is also an inputstream.
i used gipinput stream to your response and i see output in json.
public static void main(String[] args){
try
{
URL url = new URL("https://api.stackexchange.com/2.2/questions?order=desc&sort=activity&site=stackoverflow");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
System.out.println(conn.getResponseCode());
GZIPInputStream gzis = new GZIPInputStream(conn.getInputStream());
InputStreamReader reader = new InputStreamReader(gzis);
BufferedReader in = new BufferedReader(reader);
String readed;
while ((readed = in.readLine()) != null) {
System.out.println(readed);
}
//BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String output;
while ((output = in.readLine()) != null)
{
System.out.println(output);
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
it worked for me.
I have a http end point which serves xml responses to my queries. I tried to fire the http requests like this -
import java.io.*;
import java.net.*;
public class c {
public String getHTML(String urlToRead) {
URL url;
HttpURLConnection conn;
BufferedReader rd;
String line;
String result = "";
try {
url = new URL(urlToRead);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
// how to store the response as an xml file on disc
rd.close();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
public static void main(String args[])
{
c c = new c();
System.out.println(c.getHTML(args[0]));
}
}
Now, the endpoint sends xml response which i want to store as a xml file on disc. how can i do that. can someone help?
I have not tried it, but maybe you could start from something like this:
String dest = "c:\\filename.xml";
// this line substitutes your rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
InputStream is = conn.getInputStream();
// these others goes in your "how to store..." comment
byte[] bytes = IOUtils.toByteArray(is);
FileOutputStream fos = new FileOutputStream(dest);
fos.write(bytes);
fos.flush();
fos.close();
I'm trying to make a GET AJAX request on some site using java.
My code is the following:
String cookie = getRandomString(16); //Getting a random 32-symbol string
String url = "https://e-kassa.org/core/ajax/stations_search.php?"
+ "q=%D0%BE&limit=10×tamp=1352028872503";
HttpURLConnection conn = (HttpURLConnection) new URL(url).openConnection();
conn.setRequestProperty("Cookie", "PHPSESSID=" + cookie);
InputStream is = conn.getInputStream();
int buffer;
while((buffer = is.read()) != -1)
System.out.print(buffer);
is.close();
conn.disconnect();
But the problem is that there's nothing to download from the InputStream is. But if I use my browser to do the same thing, I'll get a response, composed of text lines of the following format:
CITY_NAME|SOME_DIGITS
So, can anybody tell me, how can I make such a request in an appropriate manner?
UPD: without cookies I have the same behaviour (in the browser everything's fine, but not in Java).
Can you please try with:
BufferedReader rd = null;
try {
URL url = new URL("https://e-kassa.org/core/ajax/stations_search.php?"
+ "q=%D0%BE&limit=10×tamp=1352028872503");
URLConnection conn = url.openConnection();
String cookie = (new RandomString(32)).nextString();
conn.setRequestProperty("Cookie", "PHPSESSID=" + cookie);
// Get the response
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuffer sb = new StringBuffer();
String line;
while ((line = rd.readLine()) != null) {
sb.append(line);
}
System.out.println(sb.toString());
} catch (Exception e) {
e.printStackTrace();
} finally {
if (rd != null) {
try {
rd.close();
} catch (IOException e) {
}
}
}
This is peace of code that works properly in my projects. :)
Try the following thing.
HttpURLConnection connection = null;
try {
String url = "https://e-kassa.org/core/ajax/stations_search.php?"
+ "q=%D0%BE&limit=10×tamp=1352028872503";
URL url = new URL(url);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("Cookie", "PHPSESSID=" + cookie);
connection.connect();
connection.getInputStream();
int buffer;
while((buffer = is.read()) != -1)
System.out.print(buffer);
} catch (MalformedURLException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
} finally {
if(null != connection) { connection.disconnect(); }
}