Develop Reference

Is there any way to get the SEDE query result from my Java program using Stack Exchange API?

I want to get a query result from Stack Exchange API using my Java program. For example, I want to pass this URL and get the data of the question with id 805107. I have tried but only got the resulted web page content. I did not get the query result, i.e. the question data, although the resulted page shows the question data.
url = new URL ("https://api.stackexchange.com/docs/questions-by-ids#order=desc&sort=activity&ids=805107&filter=default&site=stackoverflow&run=true");
byte[] encodedBytes = Base64.encodeBase64("root:pass".getBytes());
String encoding = new String (encodedBytes);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoInput (true);
connection.setRequestProperty ("Authorization", "Basic " + encoding);
connection.connect();
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}

As Stephen C said, you need to use the query URL, not the URL of the documentation. You can find the query URL in the "Try it" part of the documentation page. Try using
url = new URL ("https://api.stackexchange.com/2.2/questions/805107?order=desc&sort=activity&site=stackoverflow")
It will return the result you are looking for as JSON like it is displayed on the documentation page.

Java client POST to php script - empty $_POST

I have researched extensively and cannot find a solution. I have been using the solutions provided to other users and it does not seem to work for me.
My java code:
public class Post {
public static void main(String[] args) {
String name = "Bobby";
String address = "123 Main St., Queens, NY";
String phone = "4445556666";
String data = "";
try {
// POST as urlencoded is basically key-value pairs
// create key=value&key=value.... pairs
data += "name=" + URLEncoder.encode(name, "UTF-8");
data += "&address=" +
URLEncoder.encode(address, "UTF-8");
data += "&phone=" +
URLEncoder.encode(phone, "UTF-8");
// convert string to byte array, as it should be sent
byte[] dataBytes = data.toString().getBytes("UTF-8");
// open a connection to the site
URL url = new URL("http://xx.xx.xx.xxx/yyy.php");
HttpURLConnection conn =
(HttpURLConnection) url.openConnection();
// tell the server this is POST & the format of the data.
conn.setDoOutput(true);
conn.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
conn.setRequestMethod("POST");
conn.setFixedLengthStreamingMode(dataBytes.length);
conn.getOutputStream().write(dataBytes);
conn.getInputStream();
// Print out the echo statements from the php script
BufferedReader in = new BufferedReader(
new InputStreamReader(url.openStream()));
String line;
while((line = in.readLine()) != null)
System.out.println(line);
in.close();
} catch(Exception e) {
e.printStackTrace();
}
}
}
and the php
<?php
echo $_POST["name"];
?>
The output I receive is an empty line. I tested to see if it was a php/server side issue by making an html form that sends data over to a similar script and prints the data on the screen and that worked. But, for the life of me, I cannot get this to work with a remote client.
I am using Ubuntu server and Apache.
Thank you in advance.

The problem is actually in what you read as output. You are doing two requests:
1)conn.getInputStream(); - sends POST request with desired body
2)BufferedReader in = new BufferedReader(
new InputStreamReader(url.openStream())); - sends empty GET request (!!)
Change it to:
// ...
conn.getOutputStream().write(dataBytes);
BufferedReader in = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
and see result.

SSL peer shut down incorrectly

This is my first post here. I am a hobbyist so please bear with me.
I am attempting to to grab a webpage from https://eztv.it/shows/1/24/ with the following code.
public static void WriteHTMLToFile(String URL){
try {
URI myURI=new URI(URL);
URL url = myURI.toURL();
HttpsURLConnection con= (HttpsURLConnection)url.openConnection();
File myFile=new File("c:\\project\\Test.txt");
myFile.createNewFile();
FileWriter wr=new FileWriter(myFile);
InputStream ins=con.getInputStream();
InputStreamReader isr= new InputStreamReader(ins);
BufferedReader reader = new BufferedReader(isr);
String line;
while ((line = reader.readLine()) != null) {
wr.write(line+"\n");
}
reader.close();
wr.close();
}
catch(Exception e){
log(e.toString());
}
}
When I run this I get the following:
javax.net.ssl.SSLException: SSL peer shut down incorrectly
If I run the above code on this URL: https://eztv.it/shows/887/the-blacklist/ it works as intended. The difference between the two URL file sizes seems to be a contributing factor. In testing different URLs to the same server the above code only seemed to work for files less that ~30Kb. Anything over would generate the above exception.

I figured it out. The server is responding with gzip encoding once file sizes are over a certain size.
con.setRequestProperty("Accept-Encoding", "gzip, deflate, sdch");
was added to the request header as well as some code to handle the gzip stream.

POST non latin data from Java to PHP

I post some data from Java to PHP:
try {
URL obj = new URL("http://myphpurl/insert.php");
HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
conn.setReadTimeout(10000);
conn.setConnectTimeout(15000);
conn.setRequestMethod(POST_METHOD);
conn.setDoInput(true);
conn.setDoOutput(true);
Map<String, String> params = new HashMap<String, String>();
params.put("title", "العربية");
OutputStream os = conn.getOutputStream();
BufferedWriter writer =
new BufferedWriter(new OutputStreamWriter(os, "UTF-8"));
writer.write(getQuery(params));
writer.flush();
writer.close();
os.close();
BufferedReader in =
new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
String inputLine;
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
LOG.debug("response {}", response);
in.close();
response = null;
inputLine = null;
conn.disconnect();
conn = null;
obj = null;
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
private String getQuery(Map<String, String> params) throws UnsupportedEncodingException {
StringBuilder result = new StringBuilder();
boolean first = true;
Iterator<Map.Entry<String, String>> it = params.entrySet().iterator();
while (it.hasNext()) {
if (first)
first = false;
else
result.append("&");
Map.Entry<String, String> pairs = it.next();
result.append(URLEncoder.encode(pairs.getKey(), "UTF-8"));
result.append("=");
result.append(URLEncoder.encode(pairs.getValue(), "UTF-8"));
it.remove(); // avoids a ConcurrentModificationException
}
return result.toString();
}
The insert.php file looks like this:
<?php
$posttitle = $_POST["title"];
echo "$posttitle";
echo urldecode($posttitle);
?>
The echo show some gibbrish مليون instead of the actual title العربية .
This gibbrish is then inserted in a mysql database.
Additionnal info:
The DATABASE is utf8_general_ci and does support arabic (when I manually update the post using phpMyAdmin it works).
I added UTF-8 in the InputStreamReader and InputStreamWriter, and I had the following behaviour:
Tomcat6 on windows, (PHP + mysql) on CentOS --> OK
Tomcat6 on CentOS , (PHP + mysql) on CentOS --> Not OK
Additionnal infos 2
Posting using javascript works fine: The page responds with the right encoding.

There are a number of things that can go wrong with your code, and we can't test it. Also, I suggest using a full featured HTTP client instead of URLConnection. The list of what you should check:
Pass the right source files encoding to javac (your test is hardcoded. Do you run the same binary or do you run the program from your IDE or anyway recompile on the deployment machine?)
Use UTF-8 to encode the query string
If your API uses the HTTP request body, check that both ends agree on the encoding, and/or use the Content-Type MIME header
PHP has binary strings (the encoding must be given) so make sure you use the appropriate parameters when connecting to the database, and/or transcode accordingly
When sending text from the PHP server, mind the encoding of the template and of the dynamic bits!
The number of moving parts is quite big. You should not debug via print/echo because that adds another level of transcoding. If possible, dump the raw text bytes and use a hex editor.
It's funny that Windows → Linux is ok, while Linux → Linux is not. You may want to check the locale on both CentOS machines (possibly running the operating system command from inside the target process - JVM and Apache)

Try using CharsetEncoder to reveal possible encoding exceptions.
CharsetEncoder encoder = Charset.forName("UTF-8").newEncoder();
encoder.onMalformedInput(CodingErrorAction.REPORT);
encoder.onUnmappableCharacter(CodingErrorAction.REPORT);

Selecting one among many possible choices in a Web page using Java

I want to get the HTML code of the following Web Page (http://www.studenti.ict.uniba.it/esse3/ListaAppelliOfferta.do) after:
selecting "Dipartimento di Informatica" among Facoltà
selecting "Informatica" (or one of the others available)
clicking "Avvia Ricerca"
I am not very keen in the matter but I noticed the URL of the page stays the same after each selection!?!
Can anyone help describing, possibly in details, how can I do that? Unfortunately I am not expert in web programming.
Many thanks

After some tests, it refresh the pages with a POST request
fac_id:1012 --
cds_id:197 --
ad_id: -- Attività didattica
docente_id: -- Id of the docent selected
data:06/03/2014 -- Date
Anyway you missed the value of Attività ditattica, Docente and Data esame
Just run a HTTP request using HttpURLConnection (?) with this POST args, and with a XML parser read the output of tplmessage table.
Try this tutorial for HTTP request: click.
Try to read this to understand how to parse response: click
An example using the code of the tutorial:
HttpURLConnection connection = null;
try
{
URL url = new URL("http://www.studenti.ict.uniba.it/esse3/ListaAppelliOfferta.do");
connection = (HttpURLConnection) url.openConnection(); // open the connection with the url
String params =
"fac_id=1012&cds_id=197"; // You need to add ad_id, docente_id and data
connection.setRequestMethod("POST"); // i need to use POST request method
connection.setRequestProperty("Content-Length", "" + Integer.toString(params.getBytes().length)); // It will add the length of params
connection.setRequestProperty("Content-Language", "it-IT"); // language italian
connection.setUseCaches (false);
connection.setDoInput (true);
connection.setDoOutput (true);
DataOutputStream wr = new DataOutputStream(
connection.getOutputStream ());
wr.writeBytes (params); // pass params
wr.flush (); // send request
wr.close ();
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
StringBuilder response = new StringBuilder();
while((line = rd.readLine()) != null) {
response.append(line);
response.append('\r');
}
rd.close();
}
catch (MalformedURLException e)
{
e.printStackTrace();
} catch (IOException e)
{
e.printStackTrace();
}
finally
{
// close connection if created
if (connection != null)
connection.disconnect();
}
In response you will have the DOM of the page.
Anyway, use Chrome developers tool to get request args: