Develop Reference

Getting membername from JsonObject

I'm using a threaded server to get data out of an SQLite database, I have created in this case Drivers to get the various strings and ints from the database. On the server I'm creating a HashMap.
The HashMap is sent to a client handler where I create a Json Object and then override the run method to send it to the client.
On the client I'm using swing and creating buttons and tables for a GUI. When a button is pressed it calls this method which parses the reply into a Json object:
private void showDrivers() {
if (printWriter != null && bufferedReader != null) {
String toSend = "get";
printWriter.println(toSend);
String reply = null;
statusLabel.setText("Status: wait for server to reply");
try {
reply = bufferedReader.readLine();
statusLabel.setText("Status: received reply from server");
JsonParser.parseString(reply).getAsJsonObject();
} catch (IOException ex) {
statusLabel.setText("IOException " + ex);
}
System.out.println(jsonDriver);
JsonObject jsonReply = JsonParser.parseString(reply).getAsJsonObject();
JsonArray jsonDriverArray = jsonReply.getAsJsonArray("driverId");
System.out.println(jsonDriverArray);
}
}
I'm trying to get all the members from the array and then place them in a table, using the console to check that I'm actually getting the data from the array.
I have run everything in the debugger, making sure that I'm actually sending the data to the client, which I am. I expected that by using this:
JsonArray jsonDriverArray = jsonReply.getAsJsonArray("driverId");
It should get driverId 1, 2, 3, etc printing out to the console
System.out.println(jsonDriverArray);
Returns null, I thought that by using driverId as the member name it would get all the driverIds, likwise if I changed it to forenames it should get all the forenames.
In the debugger I get this which is stored in jsonReply:
1 -> {JsonObject#3072} "{"driverId":1,"driverRef":"hamilton","number":"44","code":"HAM","forename":"Lewis","surname":"Hamilton","dob":"07/01/1985","nationality":"British","url":"http://en.wikipedia.org/wiki/Lewis_Hamilton"}"
2 -> {JsonObject#3074} "{"driverId":2,"driverRef":"heidfeld","number":"","code":"HEI","forename":"Nick","surname":"Heidfeld","dob":"10/05/1977","nationality":"German","url":"http://en.wikipedia.org/wiki/Nick_Heidfeld"}"
3 -> {JsonObject#3076} "{"driverId":3,"driverRef":"rosberg","number":"6","code":"ROS","forename":"Nico","surname":"Rosberg","dob":"27/06/1985","nationality":"German","url":"http://en.wikipedia.org/wiki/Nico_Rosberg"}"
4 -> {JsonObject#3078} "{"driverId":4,"driverRef":"alonso","number":"14","code":"ALO","forename":"Fernando","surname":"Alonso","dob":"29/07/1981","nationality":"Spanish","url":"http://en.wikipedia.org/wiki/Fernando_Alonso"}"

Why am I getting 403 status code in Java after a while?

When I try to check status codes within sites I face off 403 response code after a while. First when I run the code every sites send back datas but after my code repeat itself with Timer I see one webpage returns 403 response code. Here is my code.
public class Main {
public static void checkSites() {
Timer ifSee403 = new Timer();
try {
File links = new File("./linkler.txt");
Scanner scan = new Scanner(links);
ArrayList<String> list = new ArrayList<>();
while(scan.hasNext()) {
list.add(scan.nextLine());
}
File linkStatus = new File("LinkStatus.txt");
if(!linkStatus.exists()){
linkStatus.createNewFile();
}else{
System.out.println("File already exists");
}
BufferedWriter writer = new BufferedWriter(new FileWriter(linkStatus));
for(String link : list) {
try {
if(!link.startsWith("http")) {
link = "http://"+link;
}
URL url = new URL(link);
HttpURLConnection.setFollowRedirects(true);
HttpURLConnection http = (HttpURLConnection)url.openConnection();
http.setRequestMethod("HEAD");
http.setConnectTimeout(5000);
http.setReadTimeout(8000);
int statusCode = http.getResponseCode();
if (statusCode == 200) {
ifSee403.wait(5000);
System.out.println("Hello, here we go again");
}
http.disconnect();
System.out.println(link + " " + statusCode);
writer.write(link + " " + statusCode);
writer.newLine();
} catch (Exception e) {
writer.write(link + " " + e.getMessage());
writer.newLine();
System.out.println(link + " " +e.getMessage());
}
}
try {
writer.close();
} catch (Exception e) {
System.out.println(e.getMessage());
}
System.out.println("Finished.");
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
public static void main(String[] args) throws Exception {
Timer myTimer = new Timer();
TimerTask sendingRequest = new TimerTask() {
public void run() {
checkSites();
}
};
myTimer.schedule(sendingRequest,0,150000);
}
}
How can I solve this? Thanks
Edited comment:
I've added http.disconnect(); for closing connection after checked status codes.
Also I've added
if(statusCode == 200) {
ifSee403.wait(5000);
System.out.println("Test message);
}
But it didn't work. Compiler returned current thread is not owner error. I need to fix this and change 200 with 403 and say ifSee403.wait(5000) and try it again the status code.

One "alternative" - by the way - to IP / Spoofing / Anonymizing would be to (instead) try "obeying" what the security-code is expecting you to do. If you are going to write a "scraper", and are aware there is a "bot detection" that doesn't like you debugging your code while you visit the site over and over and over - you should try using the HTML Download which I posted as an answer to the last question you asked.
If you download the HTML and save it (save it to a file - once an hour), and then write you HTML Parsing / Monitoring Code using the HTML contents of the file you have saved, you will (likely) be abiding by the security-requirements of the web-site and still be able to check availability.
If you wish to continue to use JSoup, that A.P.I. has an option for receiving HTML as a String. So if you use the HTML Scrape Code I posted, and then write that HTML String to disk, you can feed that to JSoup as often as you like without causing the Bot Detection Security Checks to go off.
If you play by their rules once in a while, you can write your tester without much hassle.
import java.io.*;
import java.net.*;
...
// This line asks the "url" that you are trying to connect with for
// an instance of HttpURLConnection. These two classes (URL and HttpURLConnection)
// are in the standard JDK Package java.net.*
HttpURLConnection con = (HttpURLConnection) url.openConnection();
// Tells the connection to use "GET" ... and to "pretend" that you are
// using a "Chrome" web-browser. Note, the User-Agent sometimes means
// something to the web-server, and sometimes is fully ignored.
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", "Chrome/61.0.3163.100");
// The classes InputStream, InputStreamReader, and BufferedReader
// are all JDK 1.0 package java.io.* classes.
InputStream is = con.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
StringBuffer sb = new StringBuffer();
String s;
// This reads each line from the web-server.
while ((s = br.readLine()) != null) sb.append(s + "\n");
// This writes the results from the web-server to a file
// It is using classes java.io.File and java.io.FileWriter
File outF = new File("SavedSite.html");
outF.createNewFile();
FileWriter fw = new FileWriter(outF);
fw.write(sb.toString());
fw.close();
Again, this code is very basic stuff that doesn't use any special JAR Library Code at all. The next method uses the JSoup library (which you have explicitly requested - even though I don't use it... It is just fine!) ... This is the method "parse" which will parse the String you have just saved. You may load this HTML String from disk, and send it to JSoup using:
Method Documentation: org.jsoup.Jsoup.parse​(File in, String charsetName, String baseUri)
If you wish to invoke JSoup just pass it a java.io.File instance using the following:
File f = new File("SavedSite.html");
Document d = Jsoup.parse(f, "UTF-8", url.toString());
I do not think you need timers at all...
AGAIN: If you are making lots of calls to the server. The purpose of this answer is to show you how to save the response of the server to a file on disk, so you don't have to make lots of calls - JUST ONE! If you restrict your calls to the server to once per hour, then you will (likely, but not a guarantee) avoid getting a 403 Forbidden Bot Detection Problem.

Get Geolocation by IP (Spigot 1.8 & 1.13.2)

i Want to find out the geolocation by only providing the ip adress.
My Aim is to save city, country, postal code and other informations.
CraftPlayer cp = (CraftPlayer)p;
String adress = cp.getAddress();
Any short possibilities, to find out by only using ip?

I recommend using http://ip-api.com/docs/api:newline_separated
You can then chose what information you need and create your HTTP-link like:
http://ip-api.com/line/8.8.8.8?fields=49471
The result in this example would be:
success
United States
US
VA
Virginia
Ashburn
20149
America/New_York
So you can create a method in Java to read HTTP and split it at \n to get the lines:
private void whatever(String ip) {
String ipinfo = getHttp("http://ip-api.com/line/" + ip + "?fields=49471");
if (ipinfo == null || !ipinfo.startsWith("success")) {
// TODO: failed
return;
}
String[] lines = ipinfo.split("\n");
// TODO: now you can get the info
String country = lines[1];
/*
...
*/
}
private static String getHttp(String url) {
try {
BufferedReader br = new BufferedReader(new InputStreamReader(new URL(url).openStream()));
String line;
StringBuilder sb = new StringBuilder();
while ((line = br.readLine()) != null) {
sb.append(line).append(System.lineSeparator());
}
br.close();
return sb.toString();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
just make sure not to create to many querys in a short amount of time since ip-api.com will ban you for it.

There are a lot of websites that provide free databases for IP geolocation.
Examples include:
MaxMind
IP2Location
At the plugin startup you could download one of these databases and then query it locally during runtime.
If you choose do download the .bin format you will have to initialize a local database and then import the data. Otherwise you could just use the csv file with a Java library like opencsv.
From the documentation of opencsv:
For reading, create a bean to harbor the information you want to read,
annotate the bean fields with the opencsv annotations, then do this:
List<MyBean> beans = new CsvToBeanBuilder(FileReader("yourfile.csv"))
.withType(Visitors.class).build().parse();
Link to documentation: http://opencsv.sourceforge.net

Multithreading (Stateless Classes)

Apologies for the long code post but am wondering if someone can help with a multithreading question (I am quite new to multi-threading). I am trying to design a facade class to a RESTFUL web services API that can be shared with multiple threads. I am using HttpURLConnection to do the connection and Google GSON to convert to and from JSON data.
The below class is what I have so far. In this example it has one public method to make an API call (authenticateCustomer()) and the private methods are used to facilitate the API call (i.e to build the POST data string, make a POST request etc).
I make one instance of this class and share it with 1000 threads. The threads call the authenticateCustomer() method. Most of the threads work but there is some threads that get a null pointer exception which is because I haven't implemented any synchronization. If I make the authenticateCustomer() method 'synchronized' it works. The problem is this results in poor concurrency (say, for example, the POST request suddenly takes a long time to complete, this will then hold up all the other threads).
Now to my question. Is the below class not stateless and therefore thread-safe? The very few fields that are in the class are declared final and assigned in the constructor. All of the methods use local variables. The Gson object is stateless (according to their web site) and created as a local variable in the API method anyway.
public final class QuizSyncAPIFacade
{
// API Connection Details
private final String m_apiDomain;
private final String m_apiContentType;
private final int m_bufferSize;
// Constructors
public QuizSyncAPIFacade()
{
m_apiDomain = "http://*****************************";
m_apiContentType = ".json";
m_bufferSize = 8192; // 8k
}
private String readInputStream(InputStream stream) throws IOException
{
// Create a buffer for the input stream
byte[] buffer = new byte[m_bufferSize];
int readCount;
StringBuilder builder = new StringBuilder();
while ((readCount = stream.read(buffer)) > -1) {
builder.append(new String(buffer, 0, readCount));
}
return builder.toString();
}
private String buildPostData(HashMap<String,String> postData) throws UnsupportedEncodingException
{
String data = "";
for (Map.Entry<String,String> entry : postData.entrySet())
{
data += (URLEncoder.encode(entry.getKey(), "UTF-8") + "=" + URLEncoder.encode(entry.getValue(), "UTF-8") + "&");
}
// Trim the last character (a trailing ampersand)
int length = data.length();
if (length > 0) {
data = data.substring(0, (length - 1));
}
return data;
}
private String buildJSONError(String message, String name, String at)
{
String error = "{\"errors\":[{\"message\":\"" + message + "\",\"name\":\"" + name + "\",\"at\":\"" + at + "\"}]}";
return error;
}
private String callPost(String url, HashMap<String,String> postData) throws IOException
{
// Set up the URL for the API call
URL apiUrl = new URL(url);
// Build the post data
String data = buildPostData(postData);
// Call the API action
HttpURLConnection conn;
try {
conn = (HttpURLConnection)apiUrl.openConnection();
} catch (IOException e) {
throw new IOException(buildJSONError("Failed to open a connection.", "CONNECTION_FAILURE", ""));
}
// Set connection parameters for posting data
conn.setRequestMethod("POST");
conn.setUseCaches(false);
conn.setDoInput(true);
conn.setDoOutput(true);
// Write post data
try {
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(data);
wr.flush();
wr.close();
} catch (IOException e) {
throw new IOException(buildJSONError("Failed to post data in output stream (Connection OK?).", "POST_DATA_FAILURE", ""));
}
// Read the response from the server
InputStream is;
try {
is = conn.getInputStream();
} catch (IOException e) {
InputStream errStr = conn.getErrorStream();
if (errStr != null)
{
String errResponse = readInputStream(errStr);
throw new IOException(errResponse);
}
else
{
throw new IOException(buildJSONError("Failed to read error stream (Connection OK?).", "ERROR_STREAM_FAILURE", ""));
}
}
// Read and return response from the server
return readInputStream(is);
}
/* -------------------------------------
*
* Synchronous API calls
*
------------------------------------- */
public APIResponse<CustomerAuthentication> authenticateCustomer(HashMap<String,String> postData)
{
// Set the URL for this API call
String apiURL = m_apiDomain + "/customer/authenticate" + m_apiContentType;
Gson jsonConv = new Gson();
String apiResponse = "";
try
{
// Call the API action
apiResponse = callPost(apiURL, postData);
// Convert JSON response to the required object type
CustomerAuthentication customerAuth = jsonConv.fromJson(apiResponse, CustomerAuthentication.class);
// Build and return the API response object
APIResponse<CustomerAuthentication> result = new APIResponse<CustomerAuthentication>(true, customerAuth, null);
return result;
}
catch (IOException e)
{
// Build and return the API response object for a failure with error list
APIErrorList errorList = jsonConv.fromJson(e.getMessage(), APIErrorList.class);
APIResponse<CustomerAuthentication> result = new APIResponse<CustomerAuthentication>(false, null, errorList);
return result;
}
}
}

If you are getting an error it could be because you are overloading the authentication service (something which doesn't happen if you do this one at a time) Perhaps it returning a error like 500, 503 or 504 which you could be ignoring and getting nothing you expect back, you return null http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
I would use less threads assuming you don't have 1000 cpus, its possible having this many threads will be slower rather than more efficeint.
I would also check that your service is returning correctly every time and investigate why you get a null value.
If your service can only handle say 20 requests at once, you can try using a Semaphore as a last resort. This can be using to limit the numebr of concurrent requests.

Any stateless class is inherently threadsafe, provided that the objects it accesses are either private to the thread, or threadsafe themselves.

Simple Java form that would send data to webpage?

I know virtually no java, but I need to make a simple java application for mobiles that would display a form.On being submitted The data would then be sent to a webpage. As I am a php programmer, I would prefer to have it sent to a php file, which would then use the form's data. I only need a couple of input text areas, would anybody be able to help me with the java part?
Thanks in advance.
Niall

You don't mention what platform but no matter what you choose you will need to look into HTTP Client and here is a good example of mimicking a form based submission.
As far as presenting a form, that's very platform dependent.

If you can call your PHP File by calling an URL via GET, than you should be able to solve your problem with the following piece of code:
int variable1 = 4;
String variable2 = "My Phone Service";
URL url = null;
try {
url = new URL("http://myserver.com/service.php?var1=" + variable1 + "&variable2=" + variable2);
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openConnection().getInputStream()));
String s = null;
while((s= reader.readLine())!=null){
System.out.println(s);
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}