I'm trying to make a program to check for announcements via a web API - This connects to a remote server and reads the JSON on the page - I cannot test my code as the server is not live yet. Would this work & be the correct way to go about this?
public class AnnouncementChecker implements Runnable{
private final String announcementsURL = "REDACTED";
private String lastAnnouncement = "";
#Override
public void run(){
try {
URL url = new URL(announcementsURL);
HttpURLConnection http = (HttpURLConnection) url.openConnection();
http.setRequestMethod("conditional GET");
http.setRequestProperty("Connection", "keep-alive");
http.setUseCaches(true);
http.setAllowUserInteraction(false);
if (lastAnnouncement != ""){
http.setRequestProperty("If-Modified-Since", lastAnnouncement);
}
http.setConnectTimeout(10);
http.connect();
int status = http.getResponseCode();
if (status == 304 || (status == 200 && lastAnnouncement == "")){
lastAnnouncement = http.getHeaderField("Last-Modified");
BufferedReader br = new BufferedReader(new InputStreamReader(http.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
sb.append(line+"\n");
}
br.close();
String json = sb.toString();
JSONParser parser = new JSONParser();
JSONObject jsonResponse = (JSONObject) parser.parse(json);
//String announcement = (String) jsonResponse.get("message");
//TODO What to do with announcement...
}
http.getInputStream().close();
http.disconnect();
} catch (IOException | ParseException e) {
e.printStackTrace();
}
}
}
I would recommend setting up a test of some kind, despite the server not being available. This would give you the answer to your question and the test would be there forever going forward to protect you when you make changes to the code and the business requirements change.
To help you with that I would recommend splitting up the code that returns the response and the code that does the parsing. That way you can test the parsing independent of the part that makes the HTTP connection.
If you have no idea how to do that then I'd be happy to post an example for you.
Related
I am trying to use this api to get report with java, and here is the link
https://developers.google.com/admin-sdk/reports/v1/appendix/activity/meet
and here is what i am using now
public static String getGraph() {
String PROTECTED_RESOURCE_URL = "https://www.googleapis.com/admin/reports/v1/activity/users/all/applications/meet?eventName=call_ended&maxResults=10&access_token=";
String graph = "";
try {
URL urUserInfo = new URL(PROTECTED_RESOURCE_URL + "access_token");
HttpURLConnection connObtainUserInfo = (HttpURLConnection) urUserInfo.openConnection();
if (connObtainUserInfo.getResponseCode() == HttpURLConnection.HTTP_OK) {
StringBuilder sbLines = new StringBuilder("");
BufferedReader reader = new BufferedReader(
new InputStreamReader(connObtainUserInfo.getInputStream(), "utf-8"));
String strLine = "";
while ((strLine = reader.readLine()) != null) {
sbLines.append(strLine);
}
graph = sbLines.toString();
}
} catch (IOException ex) {
x.printStackTrace();
}
return graph;
}
I am pretty sure it's not a smart way to do that and the string I get is quite complex, are there any jave sample that i can get the data directly instead of using java origin httpRequest
Or, are there and class I can import to switch the json string to the object!?
Anyone can help?!
I have trying this for many days already!
Thanks!!
I am going to get the JSON from the localhost db. Then, I want to put the json into ArrayList. I want to call the parseJSON function. But the JSON is null. Where I should call the function, thanks so much:))
#Override
protected String doInBackground(Void... voids) {
try {
URL url = new URL(urlWebService);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
StringBuilder sb = new StringBuilder();
InputStream input = con.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(input));
String json;
while ((json = bufferedReader.readLine()) != null) {
sb.append(json + "\n");
}
parseJSON(json);
return sb.toString().trim();
} catch (Exception e) {
return null;
}
}
}
GetJSON getJSON = new GetJSON(
);
getJSON.execute();
}
private void parseJSON(String json) {
Gson gson = new Gson();
Type type = new TypeToken<List<EssayElement>>(){}.getType();
List<EssayElement> mList = gson.fromJson(json, type);
for (EssayElement essayElement : mList){
Log.i("Message: " +
"", essayElement.id + "-" + essayElement.title + "-" + essayElement.essay + "-" + essayElement.date + "-" + essayElement.time);
}
}
null object reference with String"json"
I would suggest using a proper http library that handles making requests for you like Volley or Retrofit... JSON and Error handling are also builtin, so AsyncTask can completely be removed for that purpose
But what you have is fine, only json shouldn't be used after the while loop, it's only the last line of the http response, not the full json (assuming there's multiple lines)
You should really consider parsing the results in the onPostExecute, and possibly even having the parse method return an actual object, or display to the UI
You are appending the string to StringBuilder sb = new StringBuilder();. You have to call like this parseJSON(sb.toString()); cause String json is just a pointer doesn't hold the actual string, you want.
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
String json = sb.toString();
You can instead use my code snippet it's working fine for me. now you can use son variable for your private void parseJSON(String json) function.
I want to use Googles Custom Search Api for searching for song lyrics in the web via Java.
For getting the name and artist of current song playing I use Tesseract OCR. Even if the OCR works perfectly, I often don't get any results.
But when I try it manually: open Google in the web browser and search for the same string, then it works fine.
So now I don't really know what is the difference between the manual search engine and the api call.
Do I have to add some parameters to the Api request?
//The String searchString is what I am searching for, so the song name and artist
String searchUrl = "https://www.googleapis.com/customsearch/v1?key=(myKEY)=de&cx=(myID)&q=" + searchString + "lyrics";
String data = getData(searchUrl);
JSONObject json = new JSONObject(data);
String link = "";
try
{
link = json.getJSONArray("items").getJSONObject(0).getString("link");
URI url = new URI(link);
System.out.println(link);
Desktop.getDesktop().browse(url);
}
catch(Exception e)
{
System.out.println("No Results");
}
private static String getData(String _urlLink) throws IOException
{
StringBuilder result = new StringBuilder();
URL url = new URL(_urlLink);
URLConnection conn = url.openConnection();
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while((line = rd.readLine()) != null)
{
result.append(line);
}
rd.close();
return result.toString();
}
Try to remove =de before &cx and use + to represent the space between words. Like this - https://www.googleapis.com/customsearch/v1?key=(yourKEY)&cx=(yourID)&q=paradise+coldplay+lyrics
I'm trying to create an application, written in java, that uses my university class search function. I am using a simple http get request with the following code:
public static String GET_Request(String urlToRead) {
java.net.CookieManager cm = new java.net.CookieManager();
java.net.CookieHandler.setDefault(cm);
URL url;
HttpURLConnection conn;
BufferedReader rd;
String line;
String result = "";
try {
url = new URL(urlToRead);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = rd.readLine()) != null) {
result += line;
}
rd.close();
}
catch (Exception e) {
e.printStackTrace();
}
return result;
}
But it is not working.
Here is the url I am trying to scrape:
https://webapp4.asu.edu/catalog/classlist?c=TEMPE&s=CSE&n=100&t=2141&e=open&hon=F
I tried looking into jsoup but when I go to their try jsoup tab and fetch the url it is coming up with the same results as the get request is coming up with.
The, repeated, failed results that I'm getting with the http get request and jsoup is that it is bring up the search page of the university but not the actual classes and information about if they are open or not.
What I am ultimately looking for is a way to scrape the website that shows if the classes have open seats or not. Once I get the contents of the web page I could parse through it I'm just not getting any good results.
Thanks!
You need to add a cookie to answer the initial course offerings question:
class search course catalog
Indicate which course offerings you wish to see
* ASU Campus
* ASU Online
You do this by simply adding
conn.setRequestProperty("Cookie", "onlineCampusSelection=C");
to the HttpURLConnection.
I found the cookie by using Google Chrome's Developer Tools (Ctrl-Shift-I) and looked at Resources tab then expanded Cookies to see the webapp4.asu.edu cookies.
The following code (mostly yours) gets the HTML of the page you are looking for:
public static void main(String[] args) {
System.out.println(download("https://webapp4.asu.edu/catalog/classlist?c=TEMPE&s=CSE&n=100&t=2141&e=open&hon=F"));
}
static String download(String urlToRead) {
java.net.CookieManager cm = new java.net.CookieManager();
java.net.CookieHandler.setDefault(cm);
String result = "";
try {
URL url = new URL(urlToRead);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Cookie", "onlineCampusSelection=C");
BufferedReader rd = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
result += line + "\n";
}
rd.close();
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
Although, I'd use a real parser like jsoup or HTML Parser to do the actual parsing job.
I use the code below which in my http get request,but what I get from return is a null.I don't know why.
public static String getResponseFromGetUrl(String url) throws Exception {
StringBuffer sb = new StringBuffer();
try {
HttpResponse httpResponse = httpclient.execute(httpRequest);
String inputLine = "";
if (httpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
InputStreamReader is = new InputStreamReader(httpResponse
.getEntity().getContent());
BufferedReader in = new BufferedReader(is);
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);
}
in.close();
}
} catch (Exception e) {
e.printStackTrace();
return "net_error";
} finally {
httpclient.getConnectionManager().shutdown();
}
return sb.toString();
}
And what I have use the function is
String json_str = HttpUtils.getResponseFromGetUrl("www.xxx.com/start");
if ((json_str == null)) Log.d("Chen", "lastestTimestap----" + "json_str == null");
And sometimes the Log will be printed.Not always,in fact like 1%.But I don't know why it caused.
This code will not produce a "null". There must be more code you are not showing.
If this is all the code you have I suggest you remove the StringBuffer and replace it with
return "";
More likely you have forgetten to mention some code which is doing something like
Object o = null;
sb.append(o); // appears as "null"
EDIT: Based on your update, I would have to assume you are reading a line like "null"
It is highly unlikely you want to discard the newline between each line. I suggest either you append("\n") as well or just record all the text you get without regard for new lines.
BTW Please don't use StringBuffer as its replacement StringBuilder has been around for almost ten years. There is a common misconception that using StringBuffer helps with multi-threading but more often it results in incorrect code because it is very harder, if not impossible to use StringBuffer correctly in a multi-threaded context