Develop Reference

com.google.code.gson cannot parse tamil results

So, I'm trying to fetch JSON results from https://api-thirukkural.vercel.app/api?num=1139 using Java-Telegram-Bot-Api and send it to telegram. I use com.google.code.gson dependency for parsing JSON.
The expected results from API:
{"number":1139,"sect_tam":"காமத்துப்பால்","chapgrp_tam":"களவியல்","chap_tam":"நாணுத் துறவுரைத்தல்","line1":"அறிகிலார் எல்லாரும் என்றேஎன் காமம்","line2":"மறுகின் மறுகும் மருண்டு.","tam_exp":"என்னைத் தவிர யாரும் அறியவில்லை என்பதற்காக என் காதல் தெருவில் பரவி மயங்கித் திரிகின்றது போலும்!","sect_eng":"Love","chapgrp_eng":"The Pre-marital love","chap_eng":"Declaration of Love's special Excellence","eng":"My perplexed love roves public street Believing that none knows its secret","eng_exp":"And thus, in public ways, perturbed will rove"}
Here is a piece of my java code:
String results = "";
Random random = new Random();
SendMessage message = new SendMessage();
String apiUrl = "https://api-thirukkural.vercel.app/api?num=" + random.nextInt(1329 + 1);
try {
URL url = new URL(apiUrl);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
Scanner sc = new Scanner(url.openStream());
while (sc.hasNext()) {
results += sc.nextLine();
}
sc.close();
JSONArray jsonArray = new JSONArray("[" + results + "]");
JSONObject object = jsonArray.getJSONObject(0);
message.setChatId(update.getMessage().getChatId().toString());
message.setText("Number: " + object.getInt("number") + "\n\n" + object.getString("line1") + "\n"
+ object.getString("line2") + "\n\n" + object.getString("tam_exp") + "\n\n" + object.getString("eng_exp"));
conn.disconnect();
execute(message);
} catch (Exception e) {
e.printStackTrace();
}
The result in telegram:
Number: 1139
அறிகிலார� எல�லார�ம� என�றேஎன� காமம�
மற�கின� மற�க�ம� மர�ண�ட�.
என�னைத� தவிர யார�ம� அறியவில�லை என�பதற�காக என� காதல� தெர�வில� பரவி மயங�கித� திரிகின�றத� போல�ம�!
And thus, in public ways, perturbed will rove
Is this a problem in gson dependency? Can someone help me fix this? Thanks.

You need to specify the Charset on Scanner. That is probably the problem.
Example:
new Scanner(url.openStream(), StandardCharsets.UTF_8.name());
You should use the Charset that fits.

Issue load webpage after basic auth android

I need to load webpage after basic auth, but the sessionid in my code is not concatenated to url (I get this 10-08 10:17:27.424 20143-20143/com.example.marco.bella D/WebView: loadUrl=https://unimol.esse3.cineca.it/auth/Logon.do;jsessionid=
)...i don't know why...i show you my code!
variable cookie get the correct sessionID.
Thanks in advance!
URL url = null;
try {
url = new URL("https://unimol.esse3.cineca.it/auth/Logon.do");
} catch (MalformedURLException e) {
}
HttpURLConnection httpRequest = null;
try {
httpRequest = (HttpURLConnection) url.openConnection();
} catch (IOException e) {
}
try {
httpRequest.setRequestMethod("GET");
} catch (ProtocolException e) {
}
String cookie = "";
httpRequest.setDoInput(true);
String authString = "user" + ":" + "pass";
byte[] authEncBytes = android.util.Base64.encode(authString.getBytes(), android.util.Base64.DEFAULT);
String authStringEnc = new String(authEncBytes);
httpRequest.addRequestProperty("Authorization", "Basic " + authStringEnc);
System.out.println("auth:" + "" + authStringEnc);
DefaultHttpClient httpclient = new DefaultHttpClient();
List<Cookie> cookies = httpclient.getCookieStore().getCookies();
if (cookies.isEmpty()) {
System.out.println("None");
} else {
for (int i = 0; i < cookies.size(); i++) {
System.out.println("- " + cookies.get(i).toString());
cookie=cookies.get(i).toString();
}
}
webView.loadUrl("https://unimol.esse3.cineca.it/auth/Logon.do;jsessionid="+cookie);

(I get this 10-08 10:17:27.424 20143-20143/com.example.marco.bella
D/WebView:
loadUrl=https://unimol.esse3.cineca.it/auth/Logon.do;jsessionid= )...i
don't know why
The problem I can think of by looking only the portion you have posted here is that you are Overriding cookie variable inside your for loop and there is a high possibility on any iteration of cookies, one or more than 1 cookie exist whose value is blank.
for (int i = 0; i < cookies.size(); i++) {
System.out.println("- " + cookies.get(i).toString());
cookie=cookies.get(i).toString(); // This line is the culprit
}
You should check if the value of the cookie is null or empty.
Replace the culprit line with below line
I am using a utility of Strings class (isNullOrEmpty) of Guava library to check if value is null or empty, you can use your own implementation for this,
if(Strings.isNullOrEmpty(cookies.get(i).toString())) {
continue;
} else {
cookie=cookies.get(i).toString();
}
Now if during iteration you get id which you confirmed that you are getting in console statement then it will definitely get loaded up.

getting a null pointer exception when trying to parse JSON data from a url in a Async Class android

Could someone please explain or correct as to why I am getting a null pointer exception in my Async Class? I am trying to get data from a URL but get a null pointer exception for the 162, which contains the following code
int lengthJsonArr = jsonMainNode.length();
I am not sure as to why that is but if someone could help that would be great. or if someone can show me a better alternative to fetch json data from url that would also be a great help.
public class userTask extends AsyncTask<String, Void, Void>{
HttpURLConnection connection = null;
private String Content;
#Override
protected Void doInBackground(String... urls) {
BufferedReader reader = null;
try {
URL url = new URL(urls[0]);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
InputStream stream = connection.getInputStream();
reader = new BufferedReader(new InputStreamReader(stream));
StringBuffer buffer = new StringBuffer();
String line = "";
while ((line = reader.readLine()) != null) {
buffer.append(line);
} Content = buffer.toString();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (connection != null) {
connection.disconnect();
}
try {
if (reader != null) {
reader.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
return null;
}
#Override
protected void onPostExecute(Void s) {
super.onPostExecute(s);
String OutputData = "";
JSONObject jsonResponse;
try {
jsonResponse = new JSONObject(Content);
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
int lengthJsonArr = jsonMainNode.length(); //This is causing the exception
for (int i =0; i < lengthJsonArr; i++) {
JSONObject jsonChildNode = jsonMainNode.getJSONObject(i);
String name = jsonChildNode.optString("name").toString();
Double longitude = jsonChildNode.optDouble("lon");
Double latitude = jsonChildNode.optDouble("lat");
OutputData += " Name : "+ name +" "
+ "Longitude : "+ longitude +" "
+ "Latitude : "+ latitude +" "
+"-------------------------------------------------- ";
//Show Parsed Output on screen (activity)
Toast toast = Toast.makeText(getApplicationContext(), OutputData, Toast.LENGTH_LONG);
toast.show();
}
} catch (JSONException e) {
e.printStackTrace();
}
}
}

This is not a good way to fetch JSON data in android. You should use Volley or Retrofit library. These libraries will work accuratly and efficiently than normal code.
There are alot of things to take care of while fetching data. All will be done by library. And you just need to write few lines of code.
You can follow many good tutorials on google.

As this works...
jsonResponse = new JSONObject(Content);
...you at least succesfully receive a HTTP response which contains a valid JSON object.
The next line...
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
...tries to extract a JSON array, but apparently fails and as a result your jsonMainNode variable is null. That is how optJSONArray() works. It just returns null if it does not find what was asked for. (Instead of throwing a JSONException for example.)
Then the next line...
int lengthJsonArr = jsonMainNode.length();
...of course fails because you can't get the length of a null JSON array.
So it looks like the JSON you receive does not include an array called "Android". You could/should place a breakpoint on...
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
...and check what's in the JSON object. Or just print out the response. (And properly name it "content" with lowercase so people won't nag about the Java coding convention...)
As for avoiding the NullPointerException you could use code like:
if (jsonResponse.has("Android")) {
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
int lengthJsonArr = jsonMainNode.length();
// Etc.
// ...
}
else {
// TODO: Recover from the situation.
// ...
}

how to get a value by key from an json (or xml) string?

In the android app I get an xml or json string returned, However, I cant seem to figure out any way on how to get an value from the string in any way by entering an key.
In PHP you just use something like $myArray['parent']['child'] but I have no clue on how this works in java.
Any idea's would be greatly appreciated! (an example for both XML and JSON even more ;) )

Here's what I would do:
locate an XML/JSON library (there's tons) (google-gson for json)
read the documentation to find a parse method ((new JsonParser()).parse(text))
read the documentation to find out what the return value is (JsonElement)
decide what you want to do with the parsed data (myJsonObj.get(...))
write the code

public class parsingjsontest2 extends Activity {
/** Called when the activity is first created. */
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(main);
String str = connect("http://rentopoly.com/ajax.php?query=Bo"));
System.out.println("String::"+str);
}
}
private String connect(String url)
{
// Create the httpclient
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet(url);
// Execute the request
HttpResponse response;
// return string
String returnString = null;
try {
// Open the webpage.
response = httpclient.execute(httpget);
if(response.getStatusLine().getStatusCode() == 200){
// Connection was established. Get the content.
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
// Load the requested page converted to a string into a JSONObject.
JSONObject myAwway = new JSONObject(convertStreamToString(instream));
// Get the query value'
String query = myAwway.getString("query");
**// Make array of the suggestions
JSONArray suggestions = myAwway.getJSONArray("suggestions");
// Build the return string.
returnString = "Found: " + suggestions.length() + " locations for " + query;
for (int i = 0; i < suggestions.length(); i++) {
returnString += "\n\t" + suggestions.getString(i);
}
// Cose the stream.
instream.close();
}
}
else {
// code here for a response othet than 200. A response 200 means the webpage was ok
// Other codes include 404 - not found, 301 - redirect etc...
// Display the response line.
returnString = "Unable to load page - " + response.getStatusLine();
}
}
catch (IOException ex) {
// thrown by line 80 - getContent();
// Connection was not established
returnString = "Connection failed; " + ex.getMessage();
}
catch (JSONException ex){
// JSON errors
returnString = "JSON failed; " + ex.getMessage();
}
return returnString;
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
}

As you didn't specify what kind of xml you are trying to read, I'm answering based on what I know.
In Android, if you were talking about the layout and strings.xml files, you use a dot (.) operator, like R.string.appname.
Please post more details about your specific problem, if this is not what you were looking for.

Java - Login to a website that is using form-based authentication

I'm trying to login to a site that is using form-based authentication so that my application can go in, download the protected pages, and then exit (yes, I have a valid username/password combination).
I know:
1. the url to the login page
2. the url to the login authenticator
3. the method (post)
4. my information (obviously)
5. the username and password fields (which change based on...something. I already wrote a method to get the names).
Currently I'm using the code at this dream.in.code page as a base for my efforts.
Every time I run the application, it gets the login page sent back with a "bad username/password" message.
Code:
import java.net.*;
import java.util.LinkedList;
import java.io.*;
import javax.swing.JOptionPane;
public class ConnectToURL
{
// Variables to hold the URL object and its connection to that URL.
private static URL URLObj;
private static URLConnection connect;
private static String loginField;
private static String passwordField;
private static void getFields()
{
try
{
URLObj = new URL("http://url.goes.here/login.jsp");
connect = URLObj.openConnection();
// Now establish a buffered reader to read the URLConnection's input
// stream.
BufferedReader reader = new BufferedReader(new InputStreamReader(
connect.getInputStream()));
String lineRead = "";
LinkedList<String> lines = new LinkedList<String>();
// Read all available lines of data from the URL and print them to
// screen.
while ((lineRead = reader.readLine()) != null)
{
lines.add(lineRead);
}
reader.close();
while(lines.peekFirst().indexOf("<th>Username or E-mail:</th>") == -1)
{
lines.removeFirst();
}
String usernameCell = "";
while (usernameCell.indexOf("</td>") == -1)
{
usernameCell = usernameCell + lines.removeFirst().trim();
}
usernameCell = usernameCell.substring(usernameCell.indexOf("name=\"") + 6);
usernameCell = usernameCell.substring(0, usernameCell.indexOf("\""));
loginField = usernameCell;
while(lines.peekFirst().indexOf("<th>Password:</th>") == -1)
{
lines.removeFirst();
}
String passwordCell = "";
while (passwordCell.indexOf("</td>") == -1)
{
passwordCell = passwordCell + lines.removeFirst().trim();
}
passwordCell = passwordCell.substring(passwordCell.indexOf("name=\"") + 6);
passwordCell = passwordCell.substring(0, passwordCell.indexOf("\""));
passwordField = passwordCell;
}
catch (MalformedURLException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static void main(String[] args)
{
try
{
// getFields() grabs the names of the username and password fields and stores them into variables above
getFields();
// Establish a URL and open a connection to it. Set it to output
// mode.
URLObj = new URL("http://url.goes.here/login_submit.jsp");
connect = URLObj.openConnection();
HttpURLConnection.setFollowRedirects(true);
connect.setDoOutput(true);
}
catch (MalformedURLException ex)
{
System.out
.println("The URL specified was unable to be parsed or uses an invalid protocol. Please try again.");
System.exit(1);
}
catch (Exception ex)
{
System.out.println("An exception occurred. " + ex.getMessage());
System.exit(1);
}
try
{
// Create a buffered writer to the URLConnection's output stream and
// write our forms parameters.
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
connect.getOutputStream()));
// For obvious reasons, login info is editted.
// The line begins with username=& because there's a username field that send no data and is set to display:none.
// When I observed the request in Chrome, username was sent, but left blank. Without it, my request doesn't go through.
writer.write("username=&" + loginField + "=" + URLEncoder.encode("Username", "UTF-8") + "&" + passwordField + "=" + URLEncoder.encode("myPassword", "UTF-8"));
writer.close();
// Now establish a buffered reader to read the URLConnection's input
// stream.
BufferedReader reader = new BufferedReader(new InputStreamReader(
connect.getInputStream()));
String lineRead = "";
// Read all available lines of data from the URL and print them to
// screen.
while ((lineRead = reader.readLine()) != null)
{
System.out.println(lineRead);
}
reader.close();
}
catch (Exception ex)
{
System.out.println("There was an error reading or writing to the URL: "
+ ex.getMessage());
}
}
}

I would try to use something like HttpFox or Fiddler to see what exactly is being sent during the login and try to emulate that. Sometimes login pages massage what is being sent with Javascript.

Use LiveHTTPHeaders to check out EVERYTHING that gets posted. There is probably cookie/session data that you aren't passing though to the POST command.
Also, referrer is sometimes monitored and should be faked as well by passing the header "Referrer: http://homepage.com.../login.html"

have you tried using Appache httpClient (http://hc.apache.org/httpcomponents-client-ga/) rather writing your own code where you are parsing html and inserting values?
I believe, you don't have to parse html. Your steps should be
Look at the html and see to which "url" your authentication request is going to
open your connection to the url you found rather sending to the "login page" and parsing it to find.
httpclient class can help you manage your session to keep your session alive. you can do it urself but it would be a lot of work