Develop Reference

Piwik returns 400 bad request when trying to send tracking data via API

I'm developing an app which I plan on implementing Piwik Analytics into. Although there seems to be no real documentation (unless I'm blind) on how to use the Java Tracking API (this one), I think I managed to figure it out, but when I try it, I get this error
WARNING: Warning:400 Bad Request
org.piwik.PiwikException: error:400 Bad Request
at org.piwik.SimplePiwikTracker.sendRequest(SimplePiwikTracker.java:878)
at test.Main.main(Main.java:32)
Here is my code:
SimplePiwikTracker tracker = new SimplePiwikTracker("http://example.com/piwik/piwik.php");
tracker.setIdSite(2);
tracker.setPageUrl("http://example.com/javatest/test");
tracker.setPageCustomVariable("test", "10");
//This is line 32
tracker.sendRequest(tracker.getLinkTrackURL("http://example.com/piwik/piwik.php"));
I also tried this:
String datURL = tracker.getLinkTrackURL("http://example.com/piwik/piwik.php").toString();
URL url = new URL(datURL);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
System.out.println(connection.getResponseCode());
connection.disconnect();
But this just prints out response code 404 which doesn't make any sense because if I paste the link generated by tracker.getLinkTrackURL into my browser it works perfectly and logs correctly in Piwik (I have debug mode on).
So my question is, why is Piwik returning a bad request and why can't Java find the URL (404) when it works perfectly fine in my browser?

I figured it out. Instead of having the URL point to the http://example.com/piwik/piwik.php, it should just be going to http://example.com/piwik

Socket versus URL website access

I have a Java application which opens an existing company's website using the Socket class:
Socket sockSite;
InputStream inFile = null;
BufferedWriter out = null;
try
{
sockSite = new Socket( presetSite, 80 );
inFile = sockSite.getInputStream();
out = new BufferedWriter( new OutputStreamWriter(sockSite.getOutputStream()) );
}
catch ( IOException e )
{
...
}
out.write( "GET " + presetPath + " HTTP/1.1\r\n\r\n" );
out.flush();
I would read the website with the stream inFile and life is good.
Recently this started to fail. I was getting an HTTP 301 "site has moved" error but no moved-to link. The site still exists and responds using the same original HTTP reference and any web browser. But the above code comes back with the HTTP 301.
I changed the code to this:
URL url;
InputStream inFile = null;
try
{
url = new URL( presetSite + presetPath );
inFile = url.openStream();
}
catch ( IOException e )
{
...
}
And read the site with the original code from inFile stream and it now works again.
This difference doesn't just occur in Java but it also occurs if I use Perl (using IO::Socket::INET approach opening the website port 80, then issuing a GET fails, but using LWP::Simple method get just works). In other words, I get a failure if I open the web page first with port 80, then do a GET, but it works fine if I use a class which does it "all at once" (that just says, "get me web page with such-and-such an HTTP address").
I thought I'd try the different approaches on http://www.microsoft.com and got an interesting result. In the case of opening port 80, followed by issuing the GET /..., I received an HTTP 200 response with a page that said, "Your current user agent
In one case, I tried the "port 80" open followed by GET / on www.microsoft.com and I received an HTTP 200 response page that said, "Your current user agent appears to be from an automated process...". But if I use the second method (URL class in Java, or LWP in Perl) I simply get their web page.
So my question is: how does the URL class (in Java) or the LWP module (in Perl) do its thing under the hood that makes it different from opening the website on port 80 and issuing a GET?

Most servers require the Host: header, to allow virtual hosting (multiple domains on one IP)

If you use a packet capturing software to see what's being sent when URL is used, you'll realize that there's a lot more than just "GET /" being sent. All sorts of additional header information are included. If a server gets just a simple "GET /", it's easy to deduct that it can't be a very sophisticated client on the other end.
Also, HTTP 1.0 is "outdated", the current version is 1.1.

Java URL implementation delegates to HttpURLConnection if it starts with "http:"

PHP receiving a POST request from Java

EDIT: this is what is shown in my website logs
xx.xx.xxx.xx - - [27/Jan/2012:17:42:24 -0500] "POST /dir/addData2.php HTTP/1.1" 200 - www.mywebsites.com "-" "Java/1.7.0" "-"
I am hosting my website at 1&1, and I want to have page blank.php that should take a POST request and upload it to my database. I think I am sending my POST properly, and that somehow I am not handling it properly on my website. Because nothing is being inserted to my database. The response has content length 0, but even if i send a header with the length of the string it wont change. Another option is that the host wont allow me to do remote post requests (still waiting on reply).
I send the post request from a Java application like this:
URL url = new URL("www.mywebsite.com/blank.php");
HttpURLConnection request = (HttpURLConnection)url.openConnection();
request.setRequestProperty("Content-type","application/x-www-form-urlencoded");
request.setRequestMethod("POST");
OutputStreamWriter post = new OutputStreamWriter(request.getOutputStream());
String data = URLEncoder.encode("account", "UTF-8") + "=" + URLEncoder.encode(message[0], "UTF-8");
data += "&" + URLEncoder.encode("message", "UTF-8") + "=" + URLEncoder.encode(message[2], "UTF-8");
data += "&" + URLEncoder.encode("type", "UTF-8") + "=" + URLEncoder.encode(message[0], "UTF-8");
post.write(data);
post.flush();
/*
/ String example
/account=103&message=Feller+1391.88+0&type=103
*/
The response from the server is:
null=[HTTP/1.1 200 OK]
Date=[Fri, 27 Jan 2012 21:59:10 GMT]
Content-Length=[0]
Keep-Alive=[timeout=2, max=200]
Connection=[Keep-Alive]
Content-Type=[text/html]
Server=[Apache]
X-Powered-By=[PHP/5.2.17]
My webpage has this basic PHP code (right now, will improve/check for stuff later)
$link = mysql_connect($hostname, $username, $password);
$db_selected = mysql_select_db($database, $link);
$query = "INSERT INTO newData(account, message, type) VALUES('$_POST[account]', '$_POST[message]', '$_POST[type]')";
mysql_query($query) || die();
I want to point out that I do not have a private server, and I will probably use http://hc.apache.org/httpcomponents-core-ga/examples.html later. Right now I just want to send a string from my Java application, receive it with PHP and insert to MySQL database.

Yes, I'm certain your ISP allows both "post" and "get" HTTP server requests.
Superficially, both your Java and PHP look "OK".
STRONG SUGGESTION:
1) Check your Apache logs on the server.
Make sure the request arrived (it probably did).
Check for errors (there could well be none for your request - but you SHOULD see SOMETHING in the Apache error log.
2) Verify PHP is working.
Writing a "hello world" page with "phpinfo ();" is a good way to do this.
3) Verify that MySQL is working.
Writing a "hello world" PHP page to verify you can 1) connect, 2) make a query, and 3) echo the query back in your web browser is ideal.
4) Verify that you can successfully read the "$_POST[account]" value you THINK you're getting from the client.
5) At that point, you should be able to:
a) have the client communicate with your server
b) read the post request and write it to your database
'Hope that helps .. and please post back with any specific questions/problems you might have.

You haven't told the client code that this is a POST request. It will probably do a GET by default.
url = new URL(targetURL);
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");

It looks like you're not setting the request method on your HttpUrlConnection, and the default is GET. You'd need something like
request.setRequestMethod("POST");
If that doesn't work, I'd consider rather using the Apache HTTP client right away from the beginning, it's easiert to use than Java's standard HTTP client API.

How to read icy protocol in Java ?

I want to read data from a streaming icy protocol.The problem is that all the libraries that I've tried (dsj,MP3SPI) use the HttpUrlConnection to do this.However I've tried it on my windows 7 and I've received "Invalid http response" which is normal cause "HTTP 200 OK" is different from "ICY 200 OK".I know this could be accomplished with sockets but I'm a beginner so if any can provide a few lines o code so I can get an idea I would appreciate.Also if you have some solutions please share them.Thanx and have a nice day!
This is the code that I've tried:
URLConnection connection = new URL("89.47.247.59:8020").openConnection();
InputStream in = connection.getInputStream();
System.out.println("getting lots of bytes");
in.close();
The error is :
Exception in thread "main" java.io.IOException: Invalid Http response
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.jav‌​a:1328)
at javaapplication1.JavaApplication1.main(JavaApplication1.java:46) Java Result: 1
Sorry couldnt figure it out how to format code or add newline.
Edit: I included the code from your comment below..

Try this instead:
URL url=new URL("89.47.247.59:8020");
Socket socket=new Socket(url.getHost(), url.getPort());
OutputStream os=socket.getOutputStream();
String user_agent = "WinampMPEG/5.09";
String req="GET / HTTP/1.0\r\nuser-agent: "+user_agent+"\r\nIcy-MetaData: 1\r\nConnection: keep-alive\r\n\r\n";
os.write(req.getBytes());
is=socket.getInputStream();
This worked for me perfectly!

MP3SPI should work fine on all systems.
If you want to extract ICY metadata, check this code: https://gist.github.com/1008126 There's an IcyInputStream that opens the URL and returns a regular InputStream that you can attach to a decoder and it also extracts metadata like Artist and Track title.
I've written this code using information from here.

java.io.IOException: Server returns HTTP response code 505

I have HTML based queries in my code and one specific kind seems to give rise to IOExceptions upon receiving 505 response from the server. I have looked up the 505 response along with other people who seemed to have similar problems. Apparently 505 stands for HTTP version mismatch, but when I copy the same query URL to any browser (tried firefox, seamonkey and Opera) there seems to be no problem. One of the posts I read suggested that the browsers might automatically handle the version mismatch problem..
I have tried to dig in deeper by using the nice developer tool that comes with Opera, and it looks like there is no mismatch in versions (I believe Java uses HTTP 1.1) and a nice 200 OK response is received. Why do I experience problems when the same query goes through my Java code?
private InputStream openURL(String urlName) throws IOException{
URL url = new URL(urlName);
URLConnection urlConnection = url.openConnection();
return urlConnection.getInputStream();
}
sample link: http://www.uniprot.org/uniprot/?query=mnemonic%3aNUGM_HUMAN&format=tab&columns=id,entry%20name,reviewed,organism,length

There has been some issues in Tomcat with URLs containing space in it. To fix the problem, you need to encode your url with URLEncoder.
Example (notice the space):
String url="http://example.org/test test2/index.html";
String encodedURL=java.net.URLEncoder.encode(url,"UTF-8");
System.out.println(encodedURL); //outputs http%3A%2F%2Fexample.org%2Ftest+test2%2Findex.html

AS a developer at www.uniprot.org I have the advantage of being able to look in the request logs. In the last year according to the logs we have not send a 505 response code. In any case our servers do understand http 1 requests as well as the default http1.1 (though you might not get the results that you expect).
That makes me suspect there was either some kind of data corruption on the way. Or you where affected by a hardware failure (lately we have had some trouble with a switch and a whole datacentre ;). In any case if you ever have questions or problems with uniprot.org please contact help#uniprot.org then we can see if we can help/fix the problem.
Your code snippet seems normal and should work.
Regards,
Jerven Bolleman

Are you behind a proxy? This code works for me and prints out the same text I see through a browser.
final URL url = new URL("http://www.uniprot.org/uniprot/?query=mnemonic%3aNUGM_HUMAN&format=tab&columns=id,entry%20name,reviewed,organism,length");
final URLConnection conn = url.openConnection();
final InputStream is = conn.getInputStream();
System.out.println(IOUtils.toString(is));
conn is an instance of HttpURLConnection

from the API documentation for the URL class:
The URL class does not itself encode or decode any URL components
[...]. It is the responsibility of the caller to encode any fields,
which need to be escaped prior to calling URL, and also to decode any
escaped fields, that are returned from URL.
so if you have any spaces in your url-str encode it before calling new URL(url-str)

#posdef I was having same HTTP error code 505 problem. When I pasted URL that I was using in Java code in Firefox, Chrome it worked. But through code was giving IOException. But at last I came to know that in url string there were brackets '(' and ')', by removing them it worked so it seems I needed URLEncodeing same like browsers.