I need to obtain the input stream to a HTTPS URL eg. https://baseurl.com/mypdfgenerated.php?param=somevalue. In order to access this URL I need to get through the login page (eg. https://baseurl.com/login.php) by supplying BODY parameters:
user_name, web_pwd and submit_login
I'm assuming the only way to successfully access the first URL is by a POST to the /login.php followed by storing the cookies and then reusing the cookie-session-ID in the next GET request; if this is the correct approach then could someone please share a solution with the correct/recent libraries?
Not sure which is the best way but what helped me achieve this is the CloseableHttpClient class which along with BasicCookieStore retains cookies for subsequent requests once logged in, implemented below:
BasicCookieStore cookieStore = new BasicCookieStore();
CloseableHttpClient httpclient = HttpClients.custom().setDefaultCookieStore(cookieStore).build();
HttpUriRequest login = RequestBuilder.post()
.setUri(new URI(url_login))
.addParameter("login", "loginuname")
.addParameter("password", "pwd")
.addParameter("submit", "sub_mit");
CloseableHttpResponse response = httpclient.execute(login);
List<Cookie> cookies = cookieStore.getCookies();
response.close();
HttpGet httpget2 = new HttpGet(url_to_get_after_login);
CloseableHttpResponse response2 = httpclient.execute(httpget2);
response2.close();
Sample code snippet from Java Samples
try {
System.setProperty("java.protocol.handler.pkgs", "com.sun.net.ssl.internal.www.protocol");
java.security.Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
URL url = new URL("https://www.yourwebsite.com/"); // Some URL
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setFollowRedirects(true);
String query = "UserID=" + URLEncoder.encode("username");
query += "&";
query += "password=" + URLEncoder.encode("password");
query += "&";
// open up the output stream of the connection
DataOutputStream output = new DataOutputStream( connection.getOutputStream() );
// write out the data
output.writeBytes( query );
}catch(Exception err){
err.printStackTrace();
}
Have a look at Usage of cookies
You should use a library which handles cookies for you, such as Apache HTTPClient.
Related
I am trying to send HTTP PUT request to Cloudant database to update a key "BPInc" to "N". Its current value is "Y". I am using HTTPURLConnection to make the connection and send request. I am able to send GET request successfully and retrieve the BPInc value. But when I am sending the PUT request, I am getting error code - 400.
I have also looked into cloudant-client library, but not able to get how to send PUT request and I want to stick to HTTPURLConnection method only.
Here's the code ,
// After retrieving _rev and _id using GET request......
String revID = "_rev of the document";
String docID = "_id of the document";
URL postUrl = new URL("<Cloudant URL>/<DB NAME>/" + docID);
String usernameColonPassword = "<API KEY>:<PASSWORD>";
String basicAuthPayload = "Basic " + Base64.getEncoder().encodeToString(usernameColonPassword.getBytes());
HttpURLConnection conn = (HttpURLConnection)postUrl.openConnection();
conn.setRequestMethod("PUT");
conn.setRequestProperty("Content-Type", "application/json; utf-8");
conn.setRequestProperty("Accept", "application/json");
conn.setRequestProperty("Authorization", basicAuthPayload);
conn.setDoOutput(true);
OutputStreamWriter json = new OutputStreamWriter(conn.getOutputStream());
json.write(String.format("{\"_rev\":revID, \"BPInc\":\"Y\"}"));
json.flush();
json.close();
int putResponseCode = conn.getResponseCode();
I am very new to this so I suppose I might be doing wrong something. Please suggest.
NOTICE UPDATE!!
The problem got solved and i added my own answer in the thread
In short, I have attempted to add the parameter "scan_id" value but since it is a POST i can't add the value directly in the url path.
using the code i already have, how would i go about modifying or adding so that the url is correct, that is, so that it accepts my POST?.
somehow i have been unable to find any examples that have helped me in figuring out how i would go about doing this..
I know how to do a POST with a payload, a GET with params. but a post with Params is very confusing to me.
Appreciate any help. (i'd like to continue using HttpUrlConnection unless an other example is provided that also tells me how to send the request and not only configuring the path.
I've tried adding it to the payload.
I've tried UriBuilder but found it confusing and in contrast with the rest of my code, so wanted to ask for help with HttpUrlConnection.
URL url = new URL("http://localhost/scans/{scan_id}/launch");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("tmp_value_dont_mind_this", "432432");
con.setRequestProperty("X-Cookie", "token=" + "43432");
con.setRequestProperty("X-ApiKeys", "accessKey="+"43234;" + " secretKey="+"43234;");
con.setDoInput(true);
con.setDoOutput(true); //NOT NEEDED FOR GETS
con.setRequestMethod("POST");
con.setRequestProperty("Accept", "application/json");
con.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
//First example of writing (works when writing a payload)
OutputStreamWriter writer = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
writer.write(payload);
writer.close();
//second attemp at writing, doens't work (wanted to replace {scan_id} in the url)
DataOutputStream writer = new DataOutputStream(con.getOutputStream());
writer.writeChars("scan_id=42324"); //tried writing directly
//writer.write(payload);
writer.close();
Exception:
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost/scans/launch
I'd like one of the three response codes because then i know the Url is correct:
200 Returned if the scan was successfully launched.
403 Returned if the scan is disabled.
404 Returned if the scan does not exist.
I've tried several urls
localhost/scans/launch,
localhost/scans//launch,
localhost/scans/?/launch,
localhost/scans/{scan_id}/launch,
So with the help of a friend and everyone here i solved my problem.
The below code is all the code in an entire class explained bit by bit. at the bottom you have the full class with all its syntax etc, that takes parameters and returns a string.
in a HTTP request there are certain sections.
Such sections include in my case, Request headers, parameters in the Url and a Payload.
depending on the API certain variables required by the API need to go into their respective category.
My ORIGINAL URL looked like this: "http://host:port/scans/{scan_id}/export?{history_id}"
I CHANGED to: "https://host:port/scans/" + scan_Id + "/export?history_id=" + ID;
and the API i am calling required an argument in the payload called "format" with a value.
String payload = "{\"format\" : \"csv\"}";
So with my new URL i opened a connection and set the request headers i needed to set.
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
The setDoOutput should be commented out when making a GET request.
con.setDoInput(true);
con.setDoOutput(true);
con.setRequestMethod("POST");
con.setRequestProperty("Accept", "application/json");
con.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
con.setRequestProperty("X-Cookie", "token=" + token);
con.setRequestProperty("X-ApiKeys", "accessKey="+"23243;" +"secretKey="+"45543;");
Here i write to the payload.
//WRITING THE PAYLOAD to the http call
OutputStreamWriter writer = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
writer.write(payload);
writer.close();
After i've written the payload i read whatever response i get back (this depends on the call, when i do a file download (GET Request) i don't have a response to read as i've already read the response through another piece of code).
I hope this helps anyone who might encounter this thread.
public String requestScan(int scan_Id, String token, String ID) throws MalformedInputException, ProtocolException, IOException {
try {
String endpoint = "https://host:port/scans/" + scan_Id + "/export?history_id=" ID;
URL url = new URL(endpoint);
String payload= "{\"format\" : \"csv\"}";
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setDoInput(true);
con.setDoOutput(true);
con.setRequestMethod("POST");
con.setRequestProperty("Accept", "application/json");
con.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
con.setRequestProperty("X-Cookie", "token=" + token);
con.setRequestProperty("X-ApiKeys", "accessKey="+"324324;" +
"secretKey="+"43242;");
//WRITING THE PAYLOAD to the http call
OutputStreamWriter writer = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
writer.write(payload);
writer.close();
//READING RESPONSE
BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuffer jsonString = new StringBuffer();
String line;
while ((line = br.readLine()) != null) {
jsonString.append(line);
}
br.close();
con.disconnect();
return jsonString.toString();
} catch (Exception e) {
throw new RuntimeException(e.getMessage());
}
}
As discussed here the solution would be to change the content type to application/x-www-form-urlencoded, but since you are already using application/json; charset=UTF-8 (which I am assuming is a requirement of your project) you have no choise to redesign the whole thing. I suggest you one of the following:
Add another GET service;
Add another POST service with content type application/x-www-form-urlencoded;
Replace this service with one of the above.
Do not specify the content type at all so the client will accept anything. (Don't know if possible in java)
If there are another solutions I'm not aware of, I don't know how much they would be compliant to HTTP protocol.
(More info)
Hope I helped!
Why you are not using like this. Since you need to do a POST with HttpURLConnection, you need to write the parameters to the connection after you have opened the connection.
String urlParameters = "scan_id=42324";
byte[] postData = urlParameters.getBytes(StandardCharsets.UTF_8);
DataOutputStream dataOutputStream = new DataOutputStream(conn.getOutputStream());
dataOutputStream.write(postData);
Or if you have launch in the end, just change the above code to the following,
String urlParameters = "42324/launch";
byte[] postData = urlParameters.getBytes(StandardCharsets.UTF_8);
DataOutputStream dataOutputStream = new DataOutputStream(conn.getOutputStream());
dataOutputStream.write(postData);
URL url = new URL("http://localhost/scans/{scan_id}/launch");
That line looks odd to me; it seems you are trying to use a URL where you are intending the behavior of a URI Template.
The exact syntax will depend on which template implementation you choose; an implementation using the Spring libraries might look like:
import org.springframework.web.util.UriTemplate;
import java.net.url;
// Warning - UNTESTED code ahead
UriTemplate template = new UriTemplate("http://localhost/scans/{scan_id}/launch");
Map<String,String> uriVariables = Collections.singletonMap("scan_id", "42324");
URI uri = template.expand(uriVariables);
URL url = uri.toURL();
Java project (Eclipse). Sandbox test app. I got all necessary token and secret pairs using the Sample OAuth app and assigned them to String variables: consumerKey, consumerSecret, accessToken, accessTokenSecret, appToken, companyID.
Next. I tried to authorize QBO in two different ways: directly with HTTP connection (try/catch blocks removed):
OAuthConsumer ouathconsumer = new DefaultOAuthConsumer(consumerKey,consumerSecret);
ouathconsumer.setTokenWithSecret(accessToken, accessTokenSecret);
HttpURLConnection urlConnection = null;
URL url = new URL("https://sandbox-quickbooks.api.intuit.com/v3/company/" + companyID + "/customer/4");
urlConnection = (HttpURLConnection)url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setUseCaches(false);
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Connection", "Close");
urlConnection.setRequestProperty("Content-Type", "application/json");
urlConnection.setRequestProperty("Accept", "application/json");
ouathconsumer.sign(urlConnection);
urlConnection.connect();
BufferedReader rd = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
System.out.println(rd.readLine());
rd.close();
And it works well! But I wanna do the SDK calls. Following source gives me error (try/catch removed, secret strings are the same):
IAuthorizer authorizer = new OAuthAuthorizer(consumerKey, consumerSecret, accessToken, accessTokenSecret);
Context context = new Context(authorizer, appToken, ServiceType.QBO, companyID);
DataService service = new DataService(context);
Customer customer=new Customer();
customer.setId("3");
Customer resultCustomer = service.findById(customer);
Last line gives current error (some strings replaced by stars):
com.intuit.ipp.exception.AuthenticationException: ERROR CODE:3200, ERROR MESSAGE:message=ApplicationAuthenticationFailed; errorCode=003200; statusCode=401, ERROR DETAIL:SignatureBaseString: GET&https%3A%2F%2Fsandbox-quickbooks.api.intuit.com%2Fv3%2Fcompany%***%2Fcustomer%2F3&minorversion%3D3%26oauth_consumer_key%***%26oauth_nonce%3D-4488452729022111661%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1450088662%26oauth_token%***%26oauth_version%3D1.0%26requestid%3Dc5bba96ea6894dc3be93588324778891
at com.intuit.ipp.interceptors.HandleResponseInterceptor.execute(HandleResponseInterceptor.java:91)
at com.intuit.ipp.interceptors.IntuitInterceptorProvider.executeResponseInterceptors(IntuitInterceptorProvider.java:94)
at com.intuit.ipp.interceptors.IntuitInterceptorProvider.executeInterceptors(IntuitInterceptorProvider.java:67)
at com.intuit.ipp.services.DataService.executeInterceptors(DataService.java:126)
at com.intuit.ipp.services.DataService.findById(DataService.java:215)
at test_qb.Start_QB.main(Start_QB.java:76)
It says something about auth error, but tokens are ok. Please help me out. What am I doing wrong?
Fixed. NO SLASH MIGHT BE IN THE ENDING OF URL IN CONFIG!
also use logging tools provided by SDK & log4j
I'm trying to login web site using Java and I succeeded. Below is the code I used.
String query = "myquery";
URL url = new URL(loginUrl);
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Content-length", String.valueOf(query.length()));
con.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0;Windows98;DigExt)");
con.setDoOutput(true);
con.setDoInput(true);
DataOutputStream output = new DataOutputStream(con.getOutputStream());
output.writeBytes(query);
output.close();
DataInputStream input = new DataInputStream( con.getInputStream() );
for( int c = input.read(); c != -1; c = input.read() ) {
System.out.print( (char)c );
// this page returns JavaScript code
}
After this, I want to access another web page in same domain, so I tried below code.
URL url = new URL(anotherUrl);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
... similar to above code ...
But this page asks me to login again. I think connection has been disconnected in the process of changing URL. (Onlt login page uses HTTPS protocol and other pages use HTTP protocol)
How can I fix this?
Someone please help
Keep in mind that HTTP is completely stateless. The idea of "logging in" to a site translates to (usually) setting cookies from an HTTP perspective. Those cookies are simply HTTP headers and they are sent with each subsequent request by your browser. So for you to maintain the logged in state its up to you get the cookies from the response headers and send them along with future requests.
Here is how:
Retrieving cookies from a response:
Open a java.net.URLConnection to the server:
URL myUrl = new URL("http://www.hccp.org/cookieTest.jsp");
URLConnection urlConn = myUrl.openConnection();
urlConn.connect();
Loop through response headers looking for cookies:
Since a server may set multiple cookies in a single request, we will need to loop through the response headers, looking for all headers named "Set-Cookie".
String headerName=null;
for (int i=1; (headerName = uc.getHeaderFieldKey(i))!=null; i++) {
if (headerName.equals("Set-Cookie")) {
String cookie = urlConn.getHeaderField(i);
...
Extract cookie name and value from cookie string:
The string returned by the getHeaderField(int index) method is a series of name=value separated by semi-colons (;). The first name/value pairing is actual data string we are interested in (i.e. "sessionId=0949eeee22222rtg" or "userId=igbrown"), the subsequent name/value pairings are meta-information that we would use to manage the storage of the cookie (when it expires, etc.).
cookie = cookie.substring(0, cookie.indexOf(";"));
String cookieName = cookie.substring(0, cookie.indexOf("="));
String cookieValue = cookie.substring(cookie.indexOf("=") + 1, cookie.length());
This is basically it. We now have the cookie name (cookieName) and the cookie value (cookieValue).
Setting a cookie value in a request:
Values must be set prior to calling the connect method:
URL myUrl = new URL("http://www.hccp.org/cookieTest.jsp");
URLConnection urlConn = myUrl.openConnection();
Create a cookie string:
String myCookie = "userId=igbrown";
Add the cookie to a request:
Using the
setRequestProperty(String name, String value);
method, we will add a property named "Cookie", passing the cookie string created in the previous step as the property value.
urlConn.setRequestProperty("Cookie", myCookie);
Send the cookie to the server:
To send the cookie, simply call connect() on the URLConnection for which we have added the cookie property:
urlConn.connect()
I want to hit a URL in php from java servlet.I just want to send a information to that url.I didn't need to go to that url.i want to stay in my page.Anyway to do that.?
You could simply make a post on that url like in the http components doc
PostMethod post = new PostMethod("http://jakarata.apache.org/");
NameValuePair[] data = {
new NameValuePair("user", "joe"),
new NameValuePair("password", "bloggs")
};
post.setRequestBody(data);
// execute method and handle any error responses.
...
InputStream in = post.getResponseBodyAsStream();
// handle response.
Try following code
URL url = new URL("your url");
URLConnection connection = url.openConnection();
connection.setConnectTimeout(5000); // time out
connection.setDoOutput(true);
PrintWriter out = new PrintWriter(new OutputStreamWriter(connection.getOutputStream()));
String postData = "your post data";
out.print(postData);
out.close();
String response connection.getInputStream();
or you can use
Request.Post("your url").bodyForm(
Form.form().add("parameter name", "value")
.add("parameter name1", "value").build()).execute();