Develop Reference

How to write a Java code to read fields from a website that requires login and uses POST request?

Need some help with fetching some data from a website.
Previously , we had following code in our application and it used to fetch the required data. We just used to read the required fields by forming a URL by passing username , password and search parameter (DEA number). The same URL (with parameters ) could also be hit from browser directly to see the results. It was a simple GET request:
{URL url = new URL(
"http://www.deanumber.com/Websvc/deaWebsvc.asmx/GetQuery?UserName="+getUsername()+"&Password="+getPassword()+"&DEA="
+ deaNumber
+ "&BAC=&BASC=&ExpirationDate=&Company=&Zip=&State=&PI=&MaxRows=");
Document document = parser.parse(url.toExternalForm());
// Ask the document for a list of all <sect1> tags it contains
NodeList sections = document.getElementsByTagName("DEA");
//Followed by a loop code to get each element by using sections.item(index).getFirstChild() etc.
}
Now, the website URL has got changed to following:
https://www.deanumber.com/RelId/33637/ISvars/default/Home.htm
I am able to login to the URL with credentials , go to the search page , enter the DEA number and search. The login page comes as a pop-up once I click 'Login' link on home page. Also, the final result comes as a pop-up. This is a POST request so I am unable to form the complete URL which I could use in my code.
I am not an expert in Web Services , but I think I need a web service URL like the one mentioned in the code above. Not sure how to get that !! Even if I get the URL , I am not sure how to perform the login through Java code and search the DEA number.
Also, it would be great if I could validate the URL manually before using in Java. Let me know if there is any way.
Or, in case there is any alternate approach in Java; kindly suggest.
Thanks in advance.

First of all, the previous approach provided by the website was completely wrong and insecure, because it passes the username and password as querystring parameters in plain text. I think, they would have realized this thing and changed their way of authentication.
Also, it looks like that they have restricted the direct URL based requests from the client applications like yours. For such requests from clients, they have published the web services. Check this link. They also have mentioned the rates for web service request counts.
So, you may need to open a formal communication channel to get authentication and other details to access their web services for this purpose. Depends on what they use for web service client authentication, you may code your client to access the web services.
I hope this helps.

Why does my POST to Sagepay returns an Error 400?

I am trying to integrate my application with Sagepay, using the Server Integration Protocol. I have written my code in JAVA and currently I am at the point where I'm sending a POST to Sagepay to be redirected to their payment page. However, I get a blank screen which is a result of an Error 400 (Bad Request).
In their documentation, they specifically state that:
The data should be sent as URL Encoded Name=Value pairs separated with & characters and sent to the Sage Pay Server URL with a Service name set to the message
type in question.
The URL that I have constructed is this:
https://test.sagepay.com/gateway/service/vspserver-register.vsp&VPSProtocol=3.00&TxType=PAYMENT&Vendor=foovendor&VendorTxCode=foovendor-1459865650735-78597&Amount=10&Currency=GBP&Description=This+is+the+description&NotificationURL=http%3A%2F%2Fwww.google.com&BillingSurname=foosurname&BillingFirstnames=fooname&BillingAddress1=fooaddress&BillingCity=foocity&BillingPostCode=foopc&BillingCountry=UK&DeliverySurname=fooname&DeliveryFirstnames=foosurname&DeliveryAddress1=fooaddr&DeliveryCity=foocity&DeliveryPostCode=foopc&DeliveryCountry=UK&CustomerEMail=foo%40foo.com
What am I missing?
Thanks for your help!

Your url doesn't setup the query string properly.
Ithink that
register.vsp&VPSProtocol
should be
register.vsp?VPSProtocol
I.E. Question mark instead of ampersand.
Also, you said a post was required, but pasting that url in a browser will send a GET request, won't it ?

How can I POST using Java and include parameters and a raw request body?

I am communicating with a web service that expects a POST parameter and also expect Request body. I have confirmed that such a POST request can be done using a REST Console I have, but I am unable to make such a request in Java using Apache libraries.
In the code below, I am able to POST to the web service, and it correctly receives the contents of the variable raw_body. If I uncomment the first of the two commented lines, the web service receives the "fname" parameter, but it no longer receives the body of the POST.
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.RequestEntity;
...
HttpClient httpClient = new HttpClient();
String urlStr = "http://localhost:8080/MyRestWebService/save";
PostMethod method = new PostMethod(urlStr);
String raw_body = "This is a very long string, much too long to be just another parameter";
RequestEntity re = new StringRequestEntity(raw_body, "text/xml", "UTF-16");
//method.addParameter("fname", "test.txt");
//httpClient.getParams().setParameter("fname", "test.txt");
method.setRequestEntity(re);
How can I transmit both the parameter and the body?

You could use the setQueryString method to add the parameters to the URL that is being POSTed to. From a RESTful perspective I'd argue you should normally not be doing that, however, since a POST should represent a call to a resource and anything that would qualify for a query parameter should be included in the representation that is being transferred in the request body...or it should represent qualification of the resource itself in which case it should be part of the path that is posted to which could then be extracted by the controller using #PathVariable/#PathParam or something similar. So in your case you could also be looking for something like POST /MyRestWebService/files/test.txt or more fittingly a PUT if you're saving the resource and know the URI. The code on the server could pull the filename out from a URL pattern.

You need to make a POST request using multipart-form. Here is the example:
Apache HttpClient making multipart form post
Alternatively, you can make a POST request with the content (parameters and files) encoded using application/x-www-form-urlencoded but it is not recommended when you want to make a POST request with large content, like files.

How to get user's browser id using JSF?

Is it possible to get the user's browser id number using JSF? I use JBoss 7 for application server.

The browser's user agent string is available as HTTP request header with the name User-Agent. The request headers are in JSF available by ExternalContext#getRequestHeaderMap():
ExternalContext externalContext = FacesContext.getCurrentInstance().getExternalContext();
String userAgent = externalContext.getRequestHeaderMap().get("User-Agent");
No need to haul the raw Servlet API from under the JSF hoods. Always look at the javadoc of ExternalContext first whenever you need to have access to the HTTP servlet request or response.
Keep in mind that request headers (as everything else in a HTTP request) are fully controllable by the enduser. So never assume the information to be correct and valid. Use it for statistics only. If you need to do feature detection, strongly prefer client side languages like JavaScript and/or CSS if possible. They can do that much more reliably.

You can read user-agent header from request to get the detail about the browser
((HttpServletRequest)FacesContext.getCurrentInstance().getExternalContext().getRequest()).getHeaders();

How to take a redirection URL

I have a problem with redirection, in my work I have to make a connection to a URL that automatically go to another URL at this point it takes the credential (username and password) and redirect to a URL that contains a parameter which need me. How can I take this parameter?
To be precise, I have to do this:
Embed a Web browser with the URL https://graph.facebook.com/oauth/authorize with your client_id and the type set to user_agent. Since your application is embedding a small window, you can trigger a compact "popup" version of the authorization dialog with the display parameter:
graph.facebook.com/oauth/authorize?
client_id=...&
redirect_uri=www.facebook.com/connect/login_success.html&
type=user_agent&
display=popup
After the user authorizes your application, we redirect the user back to www.facebook.com/connect/login_success.html with the access token in the URL fragment: www.facebook.com/connect/login_success.html#access_token=...&expires_in=... Intercept the redirect above and read the access token out of the URL. Use the access token to fetch data from the Graph API on behalf of the user:
graph.facebook.com/me?access_token=....
I need access_token.

you can use apache http components httpclient to do this,
it will automatically follow redirects
HttpClient client=new HttpClient();
GetMethod get=new GetMethod("graph.facebook.com/oauth/authorize?client_id=...& redirect_uri=www.facebook.com/connect/login_success.html&type=user_agent&display=popup");
int status=client.exectueMethod(get);
Then you will have the information you need in the Location header of the response which you can access by using:
String location=get.getResponseHeader("location").getValue();
and parse the location-header for the url fragment you want.
hope that helped