Develop Reference

How to write a Java code to read fields from a website that requires login and uses POST request?

Need some help with fetching some data from a website.
Previously , we had following code in our application and it used to fetch the required data. We just used to read the required fields by forming a URL by passing username , password and search parameter (DEA number). The same URL (with parameters ) could also be hit from browser directly to see the results. It was a simple GET request:
{URL url = new URL(
"http://www.deanumber.com/Websvc/deaWebsvc.asmx/GetQuery?UserName="+getUsername()+"&Password="+getPassword()+"&DEA="
+ deaNumber
+ "&BAC=&BASC=&ExpirationDate=&Company=&Zip=&State=&PI=&MaxRows=");
Document document = parser.parse(url.toExternalForm());
// Ask the document for a list of all <sect1> tags it contains
NodeList sections = document.getElementsByTagName("DEA");
//Followed by a loop code to get each element by using sections.item(index).getFirstChild() etc.
}
Now, the website URL has got changed to following:
https://www.deanumber.com/RelId/33637/ISvars/default/Home.htm
I am able to login to the URL with credentials , go to the search page , enter the DEA number and search. The login page comes as a pop-up once I click 'Login' link on home page. Also, the final result comes as a pop-up. This is a POST request so I am unable to form the complete URL which I could use in my code.
I am not an expert in Web Services , but I think I need a web service URL like the one mentioned in the code above. Not sure how to get that !! Even if I get the URL , I am not sure how to perform the login through Java code and search the DEA number.
Also, it would be great if I could validate the URL manually before using in Java. Let me know if there is any way.
Or, in case there is any alternate approach in Java; kindly suggest.
Thanks in advance.

First of all, the previous approach provided by the website was completely wrong and insecure, because it passes the username and password as querystring parameters in plain text. I think, they would have realized this thing and changed their way of authentication.
Also, it looks like that they have restricted the direct URL based requests from the client applications like yours. For such requests from clients, they have published the web services. Check this link. They also have mentioned the rates for web service request counts.
So, you may need to open a formal communication channel to get authentication and other details to access their web services for this purpose. Depends on what they use for web service client authentication, you may code your client to access the web services.
I hope this helps.

Android, AccountManager and OAuth

I'm sure this is basic and I'm missing something. I've read through other answers on SO, I've googled, I've read resources and I just can't wrap my head around what I need to do.
I'm trying to figure out how to write an app that connects to Twitch's API, specifically how to authenticate with Twitch's api. Their documentation is here: https://github.com/justintv/Twitch-API/blob/master/authentication.md
I've created an app and stored my keys.
Now comes the part where I want my user to click a button which launches the authentication on their website. From what I can tell I do this by using an AccountManager. Except... I can't figure out what I'm supposed to do.
Here's the excerpt I've found online:
AccountManager am = AccountManager.get(this);
Bundle options = new Bundle();
am.getAuthToken(
myAccount_, // Account retrieved using getAccountsByType()
"Manage your tasks", // Auth scope
options, // Authenticator-specific options
this, // Your activity
new OnTokenAcquired(), // Callback called when a token is successfully acquired
new Handler(new OnError())); // Callback called if an error occurs
According to twitch's documentation I want to send the user to:
https://api.twitch.tv/kraken/oauth2/authorize
?response_type=code
&client_id=[your client ID]
&redirect_uri=[your registered redirect URI]
&scope=[space separated list of scopes]
&state=[your provided unique token]
And I simply have no idea how these two things need to be combined.

Firstly, I recommend to read the OAuth2 RFC. This should cover everything you need to know.
The AccountManager code snippet won't help you much unless there already is an app that provides authentication for Twitch. If that's not the case you either need to use an existing OAuth2 library or implement your own.
You could write your own AccountAuthenticator but that's a different challenge (and you still need some kind of OAuth2 client).
Doing it yourself is not that hard, see below.
Steps to implement it yourself
Twitch recommends to use the "Implicit Grant Flow" for mobile apps. That's what I'm going to describe below.
1. Get a client ID
Register your app as outlined in Developer Setup to get a client ID
As redirect URI you can use something like https://localhost:12398/, the actual port doesn't really matter.
2. Build the authentication URL
In your client app you need to construct the authentication URL like so:
https://api.twitch.tv/kraken/oauth2/authorize?
response_type=token&
client_id=[your client ID]&
redirect_uri=[your registered redirect URI]&
scope=[space separated list of scopes]
Apparently [your client ID] should be replaced by the client ID you've received from Twitch, same goes for [your registered redirect URI] (that's the URL above, i.e. https://localhost:12398/). [space separated list of scopes] is the list of scopes (i.e. features your want to access), see Scopes. Make sure you URL-encode the parameter values properly.
Assuming your client ID is 123456 and the scopes you need are user_read and channel_read your URL would look like this:
https://api.twitch.tv/kraken/oauth2/authorize?
response_type=token&
client_id=123456&
redirect_uri=https%3A%2F%2Flocalhost%3A12398%2F&
scope=user_read%20channel_read
Note that you should also pass a state parameter, just use a randomly generated value. You can also append the (non-standard) force_verify parameter to make sure the user actually needs to log in each time (instead of continuing a previous session), but I think you can achieve the same by clearing the cookie store (given that you open the URL in a webview in the context of your app) before you open the login page.
With a random state the URL would look like this:
https://api.twitch.tv/kraken/oauth2/authorize?
response_type=token&
client_id=123456&
redirect_uri=https%3A%2F%2Flocalhost%3A12398%2F&
scope=user_read%20channel_read&
state=82hdknaizuVBfd9847guHUIhndzhuehnb
Again, make sure the state value is properly URL encoded.
3. Open the authentication URL
Ideally you just open the URL in a WebView inside of your app. In that case you need to intercept all request to load a new URL using WebViewClient.shouldOverrideUrlLoading
Once the client is redirected to your redirect URL you can close the webview and continue with step 4.
Theoretically it's possible to utilize the default browser to do the authentication, but I would have security concerns since an external app could learn about your client ID and the access token.
4. Extract the access token
The actual URL you get redirected to in step #3 will have the form:
https://[your registered redirect URI]/#access_token=[an access token]&scope=[authorized scopes]
or to pick up the example
https://localhost:12398/#access_token=xxx&scope=user_read%20channel_read
Where xxx is the actual access token.
If you passed a state it will be present like so:
https://localhost:12398/#access_token=xxx&scope=user_read%20channel_read&state=82hdknaizuVBfd9847guHUIhndzhuehnb
All you have to do now is to parse the (URL encoded) access token, scope and state. Compare the scopes and state to the ones that you actually sent. If they match you can start using the access_token to authenticate.
Note According to the OAuth2 RFC, the response URL MUST also contain a token_type and it SHOULD contain an expires_in duration in seconds.
Once you received the access token you can use it to authenticate as described here.
Access tokens issued by the Implicit Grant Flow usually expire after a certain time and the user needs to authenticate again. The Twitch documentation doesn't mention any expiration time, so it's possible that the token is valid forever. So make sure your app doesn't store it or store it in a secure way (like using Android's key store provider to generate and store a key to encrypt the access token).
If the implicitly issued access token expires you could consider using the "Authorization Code Flow". That's quite similar but it contains an additional step to receive the access token and a "refresh token" that can be used to renew the access token. I leave it up to you to figure out how that works.

Login authentication programming pattern

i would like to have a discussing with you about a login pattern and ask for your input.
Especially my idea is used for a Androird Applicaion
PHP -> Native Android with AsyncHttpClient -> Activity
I dont need help for the authentication or the login procedure itself. Just about the process afterwards, if a user is already authenticated.
Imaging your having one Activity in Andriod with Login fields, thats refers after a right Login to another ShowData-Activity.
The Cookie of the Weberver (Apache + PHP) is stored in the SharedPreferences.
If the user is coming back to the application but is still logged in, as his PHPSessionID is still valid, how can we bypass the login Activity and redirect directly to the Data-Activity.
Should there be a second cookie that stores something like "logged_in", "true"
and the Android APP then checks
Pseudocode:
(If logged_in-cookie == true) { Start data-activity}
Or should there be another call to a site on the webserver that returns a true value?
Pseudo:
If(webseite_response==true){redirect to data activity}
Im not sure about the Best practise even under a security point of view.
Even if the user session is not active, someone could just send an "true" to the Andorid application, and then the user would be in the Data-Activity (even if no data is showed there)
Looking forward to your answers.
Best regards
Fabian

I would suggest storing two values in the shared preference
1)a boolean for the logged in status
2)the cookie
The during app startup,check if logged in status is true,if true you can then verify the cookie.If the cookie is valid,proceed,if its not valid,display the login interface.
Seems like a very nice and easy solution.

Just have a look at Facebook SDK for android and see how they have implemented the authentication mechanism. It should help.
https://developers.facebook.com/docs/android/

Register with facebook - Java server side authentication - Getting access token

To get user details, facebook docs suggests making this call:
https://graph.facebook.com/me?access_token=YOUR_USER_ACCESS_TOKEN
When a user signs up using facebook's "Register plugin", how do I get this user access_token from the signed request. The latter contains a oauth_token, but I couldn't find documentation on how to obtain the access_token. I use java server side authentication technique (I don't use the javascript way..)
Specifically, the docs say:
https://graph.facebook.com/oauth/access_token?
client_id=YOUR_APP_ID
&redirect_uri=YOUR_REDIRECT_URI
&client_secret=YOUR_APP_SECRET
&code=CODE_GENERATED_BY_FACEBOOK
This answer recommends passing a blank redirect uri. But where does the "CODE_GENERATED_BY_FACEBOOK" come from in the REGISTRATION flow? I tried passing the oauth_token from the signed_request as the "code generated by facebook". But it fails verification.
Pointers will be be helpful. Thanks.
Note: I already have the "sign in with facebook" working & I know how to obtain the access_token in that flow. But I would expect a more straight forward way of doing that when the user registers using facebook.

You may have solved your problem, but anyway I place my case;
You must set exactly the same value for the "redirect_uri" parameters for both dialog/oauth and oauth/access_token pages (and unfortunately, this is not stated clearly on Facebook documentation). In my case the URIs in question were:
https:
//www.facebook.com/dialog/oauth?client_id=23902620775&redirect_uri=http%3a%2f%2fwww.mydomain.com%2fMembership%2fLoginCheckFacebook&scope=user_birthday,email
https
://graph.facebook.com/oauth/access_token?client_id=23902620775&redirect_uri=http%3a%2f%2flocal.www.mydomain.com%2fMembership%2fLoginCheckFacebook&client_secret=80a34471e31e7afa0a7a7cd3cfc7&code=AQDsrevxH_Q3-NVts8VBThXCC0z6cCIhdnTMVCE9McsjJYLLy6ZKnlibmi8tWPcP...

Server-Side Redirect in PHP

I have a Java application that I need to integrate our existing PHP website with. The vendor wants us to do a server-side redirect to allow for secure authentication and single-sign-on, but I'm not sure how to do that in PHP. The vendor explained the workflow as follows:
User clicks on a 'Open Application' link on our PHP site
The PHP application hits a page on the Java application, sending the authentication parameters
If successful, the PHP application sends the headers back to the user's browser, which forces a 'redirect', otherwise the PHP app displays an error
What this will allow would be for our PHP app to securely talk to the Java app, and the client never has to send any sort of authentication.
From what I understand, .NET and Java have this capability built in, but I can't find a way in PHP to do this. Any ideas?
UPDATE
I'm not talking about using the header("Location: ..."); function to do a redirect. The kicker with this server-side redirect is that the app does the authentication and sends all that information back to the client so that the client is then logged in. Using header("Location: ...") just forces the browser to go elsewhere.
UPDATE 2
autologin.php (Simulates the user logging into an external app via curl)
// The login 'form' is at login.php
$ch = curl_init('http://domain.local/login.php');
// We are posting 2 variables, and returning the transfer just so it doesn't dump out
// Headers are processed by the callback function processHeaders()
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'processHeaders');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'username=user&password=pass');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Execute curl, close the connection, and redirect the user to a 'restricted' page
$response = curl_exec($ch);
curl_close($ch);
header("Location: http://domain.local/restricted.php");
function processHeaders($ch, $header) {
// Dump the response headers to the client
header($header);
strlen($header);
}
login.php (Contains the 'login' form)
session_start();
if($_POST) {
if($_POST['username'] == 'user' && $_POST['password'] == 'pass') {
$_SESSION['auth'] = 1;
$_SESSION['token'] = md5(time());
} else {
echo 'Auth failed';
}
} else {
echo 'Invalid access type';
}
restricted.php (Restricted page)
session_start();
if($_SESSION['auth']) {
echo 'Secret Token: '.$_SESSION['token'];
} else {
echo 'Please log in';
}
The idea is that the user wants to ultimately get to 'restricted.php'. 'login.php' contains the code necessary to log in. What I want to simulate is the user filling out the form on 'login.php' and logging the user into 'restricted.php'.
The above snippets of code work together on my local tests (hitting autologin.php redirects to restricted.php and the secret token is printed out), but I can't seem to get it to work cross-application. The apps will be on the same domain (https://domain.com/myapp, https://domain.com:1234/vendorapp).
I've never done this before in any language, I'm just going off of what my vendor has told me they've done. Apparently they've never dealt with PHP before and have no idea what to do.

like this:
header("Location: http://www.example.com/")
But it must come before any other code...see php.net

You just output a normal HTTP redirect header() like this:
<?php header('Location: http://www.example.com/'); ?>
Re Update
If I understand correctly you'd need to do this:
Browser POSTs login request to PHP server
PHP script packages the login information in some specific form for JSP app
PHP script POSTs (via cURL) or SOAPs or whatever is necessary to JSP app
PHP receives the response and parses out the necessary information
PHP sends header and/or body data back to browser
Step 4, parsing the information, depends on how you send and receive the information. If you receive them in the header via cURL, you'll need to set CURLOPT_HEADER to true and parse the necessary data out of the response. This may be as simple as splitting the string on the first blank line or more complicated, that depends on your specific situation.
How this logs in the user in your app is something you need to handle as well. The JSP app probably handles the actual password and username and hands you back a token of some sort which you'll need to keep track of.

It sounds like you are looking for the curl library, which is usually bundled with PHP.
http://php.net/manual/en/book.curl.php
<?php
session_start();
// Receive username / password from $_POST
// Prepare CURL object for post
// Post u/p to java server
// Read response
if($success)
{
header('Location: nextpage.php');
$_SESSION['LoggedInTime'] = time();
exit;
}
else
{
//display error
}
Update:
Later, you can check $_SESSION['LoggedInTime'] + 3600 > time() to see if they are still logged in. Every time they visit a logged in page, do this:
if($_SESSION['LoggedInTime'] + 3600 > time())
{
$_SESSION['LoggedInTime'] = time() + 3600;
}
else
{
header('Location: /login.php?Message=session+expired');
exit;
}
Hope this helps.

If you are trying to integrate php and java on the web, you may want to look into Quercus/Resin. Your PHP can then call java code directly. Since they are running on the same server, the java code could write any cookies, setup any sessions or doing any necessary setup processing.
http://www.caucho.com/resin-3.0/quercus/tutorial/module/index.xtp