Develop Reference

cURL + Java -- CPAS Request -- Suggestion Please

Explination
I am working on a Java App that sends a text message via a CPAS API (Similar to Tweepy). I am using CURL to request the message send. The request from Java seems to be sent but I am getting a 401 code. I'm assuming there is an issue with the encoding of my Authentication for the request. The code is as follows:
URL url = new URL("https://api.zang.io/v2/Accounts/ACe1889084d37de951ef064200aecbe4b2/SMS/Messages");
String auth = AUTH + ":" + TOKEN;
String encodeedAuth = Base64.getEncoder().withoutPadding().encodeToString(auth.getBytes());
HttpURLConnection http = (HttpURLConnection)url.openConnection();
http.setRequestMethod("POST");
http.setDoOutput(true);
http.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
http.setRequestProperty("Authorization", "Basic " + encodeedAuth);
String data = "From=+14132698029&To=17817381451&Body=New Test";
byte[] out = data.getBytes(StandardCharsets.UTF_8);
OutputStream stream = http.getOutputStream();
stream.write(out);
System.out.println(http.getResponseCode() + " " + http.getResponseMessage());
http.disconnect();
"""

An HTTP 401 response is commonly an HTTP endpoint authorization failure. The best way to see why your auth credentials are failing is to capture the request/response pair in transit (at the receiving end would be optimal) and then compare that request/response pair to one that works.

UPnP ContentDirectory Browse returns null

I have been playing around with UPnP, to get an understanding of how it all works, before I try working with any of the APIs that are out there, or doing anything more substantial. I have been reading through the UPnP documentation, and have used that information to format the messages that I am sending. I am just working from the command line right now, and have gotten discovery messages to work without issue. Now, I'm trying to return content from a ContentDirectory Browse() request (I have also tried TransportAV GetMediaInfo() because it takes only one argument). However, no matter what I try, I am getting a Null response from the MediaServer.
public class SOAPSocket2 {
public static void main(String[] args) {
try {
String xmldata = "<?xml version=\"1.0\"?>" +
"<s:Envelope " +
"xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/ \"" +
"s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\">" +
"<s:Body>" +
"<u:GetMediaInfo xmlns:u=\"urn:schemas-upnp-org:service:AVTransport:1\">" +
"<InstanceID>0</InstanceID>" +
"</u:GetMediaInfo>" +
"</s:Body>" +
"</s:Envelope>";
//Create socket
String hostname = args[0];
int port = Integer.parseInt(args[1]);
Socket sock = new Socket(hostname, port);
//Send header
String path = args[2];
BufferedWriter wr = new BufferedWriter(new OutputStreamWriter(sock.getOutputStream(),"UTF-8"));
// You can use "UTF8" for compatibility with the Microsoft virtual machine.
wr.write("POST " + path + " HTTP/1.1\r\n");
wr.write("HOST: " + hostname + ":" + port +"\r\n");
wr.write("CONTENT-TYPE: text/xml; charset=\"utf-8\"\r\n");
wr.write("SOAPACTION: \"urn:schemas-upnp-org:service:AVTransport:1#GetMediaInfo\"");
wr.write("\r\n");
//Send data
wr.write(xmldata);
wr.flush();
// Response
BufferedReader rd = new BufferedReader(new InputStreamReader(sock.getInputStream()));
String line;
line = rd.readLine();
System.out.println(line);
while((line = rd.readLine()) != null)
System.out.println(line);
} catch (Exception e) {
e.printStackTrace();
}
}
}
I know this isn't the most proper code, but I borrowed it from here: http://users.skynet.be/pascalbotte/rcx-ws-doc/xmlpost.htm. I figured that if I could actually get some sort of data in a response, then I could work on building it properly. I have modified it so that I pass the IP address and Port of the Media Server from the command line, as well as the path to the Control URL. However, I am getting nothing but 'null' from the Media Server. Any thoughts on what I'm doing incorrectly? Thanks

I am getting a Null response from the MediaServer
Is that a response from MediaServer? I would imagine it's just BufferedReader telling you there's nothing to return.
You have two things here you can debug (sending and receiving) but the same tools should help with both. Use wireshark or another network traffic capture tool to see the actual data that goes through the network. Wireshark will tell you if the response is sent (but you are failing to receive it properly) or if the reply never comes (implying your message is incorrect). It will also show your message as it is on the wire, making it easier to notice mistakes.
Doing the above (and pasting the messages here if you can't figure it out) is the best way to continue debugging, but I can see some problems in the code already:
SOAPACTION-line is missing "\r\n" in the end
There is no CONTENT-LENGTH header (this is required in normal cases)
These aren't even UPnP problems really, the message just isn't proper HTTP. Still, the UPnP Device Architecture document will help with problems like this.

Using HttpURLConnection to submit a json POST request

I am trying to submit a json post request using HttpURLConnection in Scala. I followed along to two tutorials and produced this:
def sendPost(url: String, jsonHash: String) {
val conn: HttpURLConnection = new URL(url).openConnection().asInstanceOf[HttpURLConnection]
conn.setRequestMethod("POST")
conn.setRequestProperty("Content-Type", "application/json")
conn.setRequestProperty("Accept", "application/json")
conn.setDoOutput(true)
conn.connect()
val wr = new DataOutputStream(conn.getOutputStream)
wr.writeBytes(jsonHash)
wr.flush()
wr.close()
val responseCode = conn.getResponseCode
println("Sent: " + jsonHash + " to " + url + " received " + responseCode)
val in = new BufferedReader(new InputStreamReader(conn.getInputStream))
var response: String = ""
while(response != null) {
response = in.readLine()
println(response)
}
in.close()
}
It responds with:
Sent: '{"schedule":"R/2014-02-02T00:00:00Z/PT24H", "name":"Scala-Post-Test", "command":"which scalac", "epsilon":"PT15M", "owner":"myemail#thecompany.com", "async":false}' to http://localhost:4040/scheduler/iso8601 received 500
java.io.IOException: Server returned HTTP response code: 500 for URL: http://localhost:4040/scheduler/iso8601
stemming from
val in = new BufferedReader(new InputStreamReader(conn.getInputStream))
but if I rebuild it as a curl request, it works fine:
curl -X POST -H 'Content-Type: application/json' -d '{"schedule":"R/2014-02-02T00:00:00Z/PT24H", "name":"Scala-Post-Test", "command":"which scalac", "epsilon":"PT15M", "owner":"myemail#thecompany.com", "async":false}' http://localhost:4040/scheduler/iso8601
requirement failed: Vertex already exists in graph Scala-Post-Test
(which is what I expect)
Any insight to what is wrong? I'm trying to sniff the packets now to determine what is different.
(Note: I had previously given up on sys.process._)

The issue is here:
Sent: '{"schedule":"R/2014-02-02T00:00:00Z/PT24H", "name":"Scala-Post-Test", "command":"which scalac", "epsilon":"PT15M", "owner":"myemail#thecompany.com", "async":false}' to http://localhost:4040/scheduler/iso8601 received 500
You'll note your JSON is surrounded by single quotes. This makes it invalid.
Also worth noting is that while this code works, you are using a DataOutputStream.writeBytes() to output your data. This would be problematic if your string including anything but single-byte characters; it strips the high 8 bits off each char (Java uses 2-byte chars to hold UTF-16 codepoints).
It's better to use something more suited for String output. The same technique you use for input, for example:
BufferedWriter out =
new BufferedWriter(new OutputStreamWriter(conn.getOutputStream));
out.write(jsonString);
out.close();

Android C2DM: Could not verify SSL certificate javax.net.ssl.SSLHandshakeException

I'm receiving the following SSLHandshakeException when trying to send a push notification through the C2DM servers.
javax.net.ssl.SSLHandshakeException: Could not verify SSL certificate for: https://android.apis.google.com/c2dm/send
The code to send the message is as follows, and is running on App Engine. Everything works fine when I use cURL, so I know that the server authentication code and device registration ID are correct.
public static void sendHttpPostToC2dmService(String msg, PrintWriter out) {
String authCode = "XXXX";
String regID = "YYYY";
try {
URL url = new URL("https://android.apis.google.com/c2dm/send");
String data = URLEncoder.encode("registration_id", "UTF-8") + "="
+ URLEncoder.encode(regID, "UTF-8");
data += "&"
+ URLEncoder.encode("Authorization: GoogleLogin auth",
"UTF-8") + "="
+ URLEncoder.encode(authCode, "UTF-8");
data += "&" + URLEncoder.encode("collapse_key", "UTF-8") + "="
+ URLEncoder.encode("something", "UTF-8");
data += "&" + URLEncoder.encode("data.message", "UTF-8") + "="
+ URLEncoder.encode(msg, "UTF-8");
out.println("data=" + data);
HttpURLConnection connection = (HttpURLConnection) url
.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
OutputStreamWriter writer = new OutputStreamWriter(
connection.getOutputStream());
writer.write(data);
writer.close();
out.println("responseCode=" + connection.getResponseCode());
if (connection.getResponseCode() == HttpURLConnection.HTTP_OK) {
String responseLine = new BufferedReader(new InputStreamReader(
connection.getInputStream())).readLine();
out.println("responseLine=" + responseLine);
} else {
// Server returned HTTP error code.
}
} catch (MalformedURLException e) {
out.println("MalformedURL");
out.println(e.getMessage());
} catch (IOException e) {
out.println("IOException");
out.println(e.getMessage());
e.printStackTrace(out);
}
}
It seems like others have also had this problem, but I haven't been able to find a clear solution (at least not one I'm able to understand). Appreciate any help.

You need to use HttpsURLConnection instead of HttpURLConnection.
AFAIK, HttpURLConnection doesn't verify hostnames.

If you are in the testing phase, you could use the non-secure url to push a notification.
http://android.apis.google.com/c2dm/send
I also got around this issue by adding a certificate validation callback to accept any certificate, but this was on a C# server. I'm sure there is a Java equivalent for this.
In production code you would want to verify the correct certificate or maybe just encrypt the data you are sending.

Java method to log into ASP.NET Web form

I'm working on a java program that will need to log into a ASP.NET web form, then once authenticated, download a file. Normal HTTP GET/POST is not a problem, but it appears that ASP is not giving me a SESSION ID when I connect from java, but it is from the browser.
When I look at the header information in Firefox, I see the cookies being set from the initial login, but then the page is immediately redirected over to a new URL. I'm not sure if it matters, but the page it redirects to after login contains iframes. I've tried loading both the main page and the iframe src inside, but neither give me the cookie in the header.
//Pull up the login page, extract out the hidden input variables __VIEWSTATE, __EVENTVALIDATION
URL url = new URL(loginPage);
HttpURLConnection conn = null;
conn = (HttpURLConnection) url.openConnection();
//This reads the page line-by-line and extracts out all the values from hidden input fields
Map<String,String> formFields = getViewstate(conn);
//Now re-open the URL to actually submit the POST data
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
DataOutputStream out = new DataOutputStream(conn.getOutputStream());
String postValues = URLEncoder.encode("txtUsername", "UTF-8") + "=" + URLEncoder.encode(uid, "UTF-8");
postValues += "&" + URLEncoder.encode("txtPassword", "UTF-8") + "=" + URLEncoder.encode(pwd, "UTF-8");
postValues += "&" + URLEncoder.encode("__EVENTTARGET", "UTF-8") + "=" + URLEncoder.encode("", "UTF-8");
postValues += "&" + URLEncoder.encode("__VIEWSTATE", "UTF-8") + "=" + URLEncoder.encode(formFields.get("viewstate"), "UTF-8");
postValues += "&" + URLEncoder.encode("__EVENTVALIDATION", "UTF-8") + "=" + URLEncoder.encode(formFields.get("eventvalidation"), "UTF-8");
out.writeBytes(postValues);
out.flush();
out.close();
//At this point looking at Firefox sniffer data, it should be sending back the cookie
//However there is no Set-Cookie in the header fields
for (int i = 1; (key = conn.getHeaderFieldKey(i)) != null; i++) {
// get ASP.NET_SessionId from cookie
if (key.equalsIgnoreCase("set-cookie")) {
sessionId = conn.getHeaderField(key);
sessionId = sessionId.substring(0, sessionId.indexOf(";"));
}
}
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = rd.readLine()) != null) {
//The page it prints out is the page it was redirected to when logged in through the browser
System.out.println(line);
}
rd.close();
//At this point, it was a successful login, but I never got the cookie so I'm stuck

HttpClient, which I believe HtmlUnit is based on, has the lower level functionality I think you're looking for. Handles cookies well, though if you need more, then Kurt is right in that you should look for something with more functionality. If you actually need to get full browser functionality, you could try something like Selenium/Webdriver that actually automates a browser under programmatic control.

It looks like the site you are trying to access relies on Cookies which are not supported by HttpURLConnection. A way around this issue is to use a library like HtmlUnit which simulates a browser (supports cookies, javascript, etc..).