I'm attempting to login to twitter using the following code I've written. The issue is on each execution i receive a 400 Bad Request back as the response. I have tried numerous attempts to get this to work to no avail.
public void login(String url) throws ClientProtocolException, IOException{
HttpClient client = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(url);
// add request header
request.addHeader("User-Agent", USER_AGENT);
HttpResponse response = client.execute(request);
System.out.println("Response Code : "
+ response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
// set cookies
setCookies(response.getFirstHeader("Set-Cookie") == null ? "" : response.getFirstHeader("Set-Cookie").toString());
Document doc = Jsoup.parse(result.toString());
System.out.println(doc);
// Get input elements
Elements loginform = doc.select("div.clearfix input[type=hidden][name=authenticity_token]");
String auth_token = loginform.attr("value");
System.out.println("Login: "+auth_token);
List<NameValuePair> paramList = new ArrayList<NameValuePair>();
paramList.add(new BasicNameValuePair("authenticity_token", auth_token));
paramList.add(new BasicNameValuePair("session[username_or_email]", "twitter_username"));
paramList.add(new BasicNameValuePair("session[password]", "twitter_password"));
System.out.println(paramList);
HttpPost post = new HttpPost(url);
// add header
post.setHeader("Host", "twitter.com");
post.setHeader("User-Agent", USER_AGENT);
post.setHeader("Accept", "text/html,application/xhtml;q=0.9,*/*;q=0.8");
post.setHeader("Accept-Language", "en-US,en;q=0.5");
post.setHeader("Keep-Alive", "115");
post.setHeader("Cookie", getCookies());
post.setHeader("Connection", "keep-alive");
post.setHeader("Referer", "https://twitter.com/");
post.setHeader("Content-Type", "application/x-www-form-urlencoded");
post.setEntity(new UrlEncodedFormEntity(paramList));
// Execute POST data
HttpResponse res = client.execute(post);
int responseCode = res.getStatusLine().getStatusCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + paramList);
System.out.println("Response Code : " + responseCode);
System.out.println("Headers: "+res.getAllHeaders().toString());
System.out.println("Response: "+res.getStatusLine());
BufferedReader rd1 = new BufferedReader(
new InputStreamReader(res.getEntity().getContent()));
StringBuffer resul = new StringBuffer();
String line1 = "";
while ((line1 = rd1.readLine()) != null) {
resul.append(line1);
}
Document doc2 = Jsoup.parse(res.toString());
System.out.println(doc2);
}
public static void main(String[] args) throws ClientProtocolException, IOException{
Browser b = new Browser();
b.login("https://twitter.com/login");
}
I believe that everything that needs to be POST'd is being, such as the username, password, as well as the authenticity token.
Turns out i was sending the wrong session information in my POST request! If anyone else has a similar issue i recommend using Chrome Developer tools to inspect the headers being sent/received.
Related
Below is what i tried to send a HTTP POST request which send the json file as payload. The Error I always get is
java.io.FileNotFoundException: test.json (The system cannot find the file specified)
Although the test.json file is in the same folder.
private void sendPost() throws Exception {`
String url = "url";
HttpClient httpClient = HttpClientBuilder.create().build();
HttpPost post = new HttpPost(url);
List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
String postData = AutomaticOnboarding.readFile("test.json");
urlParameters.add(new BasicNameValuePair("data", postData));
StringEntity se = new StringEntity(postData);
post.setHeader(HttpHeaders.CONTENT_TYPE, "application/json");
post.setEntity(se);
HttpResponse response = httpClient.execute(post);
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + post.getEntity());
int responseCode = response.getStatusLine().getStatusCode();
System.out.println("Response Code : " +responseCode);
if(responseCode == 200){
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
}
}
Here follows the readFile method:
public static String readFile(String filename) {
String result = "";
try {
BufferedReader br = new BufferedReader(new FileReader(filename));
StringBuilder sb = new StringBuilder();
String line = br.readLine();
while (line != null) {
sb.append(line);
line = br.readLine();
}
result = sb.toString();
} catch(Exception e) {
e.printStackTrace();
}
return result;
}
Use the class loader to get resources inside the jar
getClass().getClassLoader().getResourceAsStream(filename)
I was trying to upload files to OneDrive using multipart request. I've tried many ways but none of them worked.
The request is of the form :-
POST /drive/items/{folder-id}/children
Content-Type: multipart/related; boundary="A100x"
--A100x
Content-ID: <metadata>
Content-Type: application/json
{
"name": "newfile.txt",
"file": {},
"#content.sourceUrl": "cid:content",
"#name.conflictBehavior": "rename"
}
--A100x
Content-ID: <content>
Content-Type: text/plain
Contents of the file to be uploaded.
--A100x--
I've tried many ways. The snippet I've done for this added below. Any help would be appreciated.
Snippet :-
HttpClient httpClient = HttpClientBuilder.create().build();
URIBuilder uriBuilder = new URIBuilder(URI);
HttpPost post = new HttpPost(uriBuilder.build());
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
Charset chars = Charset.forName("utf-8");
builder.setCharset(chars);
post.setEntity(builder.build());
builder.addPart("content", new FileBody(new File("/home/myfiles/test"), ContentType.APPLICATION_OCTET_STREAM, "test"));
builder.addPart("metadata", new StringBody(metaJson, ContentType.APPLICATION_JSON));
post.setHeader("Content-Type", "multipart/form-data");
post.addHeader("Authorization", "Bearer " + ACCESS_TOKEN);
try {
HttpResponse response = httpClient.execute(post);
if (response.getStatusLine().getStatusCode() == 200) {
br = new BufferedReader(new InputStreamReader((response.getEntity().getContent())));
responseBuilder = new StringBuilder();
while ((output = br.readLine()) != null) {
responseBuilder.append(output);
}
} else {
System.out.println("Failed : HTTP error code : " + response.getStatusLine().getStatusCode());
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
URI :
https://api.onedrive.com/v1.0/drive/root:/myfiles/children
Later on I realized that I can use simple file upload provided my files are simple doc files. And achieved solution using Apache REST client.
Code snippet :-
BufferedReader br = null;
String output;
StringBuilder responseBuilder = null;
HttpClient httpClient = HttpClientBuilder.create().build();
URIBuilder uriBuilder = new URIBuilder(<UPLOAD_URL>);
HttpPut request = new HttpPut(uriBuilder.build());
request.addHeader("Authorization", "Bearer " + oneDriveConnection.getAccessToken());
request.addHeader("Content-Type", mimeType);
HttpEntity entity = new ByteArrayEntity(bytes);
request.setEntity(entity);
HttpResponse response = httpClient.execute(request);
int responseCode = response.getStatusLine().getStatusCode();
if (responseCode == 201 || responseCode == 200) {
br = new BufferedReader(new InputStreamReader((response.getEntity().getContent())));
responseBuilder = new StringBuilder();
while ((output = br.readLine()) != null) {
responseBuilder.append(output);
}
} else {
logger.error("Failed : HTTP error code : " + response.getStatusLine().getStatusCode());
throw new UploadException("Upload failure, Status code : " + response.getStatusLine().getStatusCode());
}
The main idea is. I do an httpget request to take the form of login page from moodle. (html code)
Then i find the form params in the html code and i fill them.
Then i send httpPost request to the login page with the parameters (email and pass)
And in the end trying to get the html code from a new website that needed to login first.
Buts i am always getting the same code . The one from login Page.
I have tried this to gmail and i think it works. I dont get the same code after ther post request at least.
If anyone can help me it would be great becase i am in a little hurry.
Thanks in advance :)
public class MyHttpClient {
private String cookies;
private HttpClient client = getNewHttpClient() ;
private final String USER_AGENT = "Chrome/39.0.2171.65";
#SuppressLint("NewApi")
/* this method is called from the
* mainactivity .
*courseUrl is the page i want to go after i login
*/
public String getHtmlFromElearn(String courseURL) throws Exception{
String url = "https://elearn.uoc.gr/login/index.php"; //the login page
CookieHandler.setDefault(new CookieManager());
MyHttpClient http = new MyHttpClient();
String page = http.GetPageContent(url); //gets the html code of login page
List<NameValuePair> postParams =
http.getFormParams(page, "mymail#csd.uoc.gr","mypass"); //pasing the email and pass params to post request
http.sendPost(url, postParams); // sends the post request with the params found in form
String result = http.GetPageContent(courseURL);
System.out.println("Done");
return result;
}
/*
* sends post request with params to login to the website
*/
private void sendPost(String url, List<NameValuePair> postParams)
throws Exception {
//was getting exception for main thred without this
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);
HttpPost post = new HttpPost(url);
// add header details
post.setHeader("Host", "elearn.uoc.gr");
post.setHeader("User-Agent", USER_AGENT);
post.setHeader("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
post.setHeader("Accept-Language", "el-GR,el;q=0.8");
post.setHeader("Cookie", getCookies());
post.setHeader("Connection", "keep-alive");
post.setHeader("Referer", "https://elearn.uoc.gr/login/index.php");
post.setHeader("Content-Type", "application/x-www-form-urlencoded");
post.setEntity(new UrlEncodedFormEntity(postParams,HTTP.UTF_8));
HttpResponse response = client.execute(post);
// THE RESPONSE I AM GETTING IS 200
int responseCode = response.getStatusLine().getStatusCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + postParams);
System.out.println("Response Code : " + responseCode);
//copy the response page in to a string
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line).append("\n");
}
if(result.toString().contains("login failed")){ // checks if login succeeded
System.out.println("login faild");
}
System.out.println(result.toString());
}
#SuppressLint("NewApi")
/*
* gets the html code from the url given
*/
private String GetPageContent(String url) throws Exception {
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);
//httpget request to get the html of page
HttpGet request = new HttpGet(url);
request.setHeader("User-Agent", USER_AGENT);
request.setHeader("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
request.setHeader("Accept-Language", "el-GR,el;q=0.8");
HttpResponse response = client.execute(request);
int responseCode = response.getStatusLine().getStatusCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
//copies response html code to string
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
String result = new String();
String line = "";
while ((line = rd.readLine()) != null) {
result+=line+"\n";
}
// set cookies
setCookies(response.getFirstHeader("Set-Cookie") == null ? "" :
response.getFirstHeader("Set-Cookie").toString());
return result;
}
/*gets the email and pass and returns a lists withh all the paramas
* for post request to login
*/
public List<NameValuePair> getFormParams(
String html, String username, String password)
throws UnsupportedEncodingException {
System.out.println("Extracting form's data...");
Document doc = Jsoup.parse(html);
Element loginform = doc.getElementById("login");
Elements inputElements = loginform.getElementsByTag("input");
List<NameValuePair> paramList = new ArrayList<NameValuePair>();
for (Element inputElement : inputElements) {
String key = inputElement.attr("name");
String value = inputElement.attr("value");
if (key.equals("username")){
value = username;
}
else if (key.equals("password")){
value = password;
}
paramList.add(new BasicNameValuePair(key, URLEncoder.encode(value, "UTF-8")));
}
return paramList;
}
public String getCookies() {
return cookies;
}
public void setCookies(String cookies) {
this.cookies = cookies;
}
/*
*return an http client that trusts all cerfificates
*/
public HttpClient getNewHttpClient() {
try {
KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
trustStore.load(null, null);
SSLSocketFactory sf = new MySSLSocketFactory(trustStore);
sf.setHostnameVerifier(SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);
SchemeRegistry registry = new SchemeRegistry();
registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
registry.register(new Scheme("https", sf, 443));
ClientConnectionManager ccm = new ThreadSafeClientConnManager(params, registry);
return new DefaultHttpClient(ccm, params);
} catch (Exception e) {
return new DefaultHttpClient();
}
}
}
I used Jsoup.connect and logged in successfully to any web site
Response res = Jsoup
.connect("loginPage.com/login.php")
.data("username", "myUserName", "password", "myPass")
.method(Method.POST)
.execute();
//This will get you cookies
Map<String, String> loginCookies = res.cookies();
And then if i want to visit any other web page i use the cookies that login page returned me.
Document doc = Jsoup.connect("urlYouNeedToBeLoggedInToAccess")
.cookies(loginCookies)
.get()
There is a .Net web service and I have to send XML data from my local applications. My local application are running on Java & Sql.
Web service is accepting xml type. would you please help me how should I do? is there an example for this case?
I am giving you 2 examples from your java application, you post an file to service.
Apache HttpClient :
String url = "https://yoururl.com";
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
// add header
post.setHeader("User-Agent", USER_AGENT);
List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
urlParameters.add(new BasicNameValuePair("xml", xmlString));
post.setEntity(new UrlEncodedFormEntity(urlParameters));
HttpResponse response = client.execute(post);
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + post.getEntity());
System.out.println("Response Code : " +
response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
Here's an example how to do it with java.net.URLConnection:
String url = "http://example.com";
String charset = "UTF-8";
String param1 = URLEncoder.encode("param1", charset);
String param2 = URLEncoder.encode("param2", charset);
String query = String.format("param1=%s¶m2=%s", param1, param2);
URLConnection urlConnection = new URL(url).openConnection();
urlConnection.setUseCaches(false);
urlConnection.setDoOutput(true); // Triggers POST.
urlConnection.setRequestProperty("accept-charset", charset);
urlConnection.setRequestProperty("content-type", "application/x-www-form-urlencoded");
OutputStreamWriter writer = null;
try {
writer = new OutputStreamWriter(urlConnection.getOutputStream(), charset);
writer.write(query); // Write POST query string (if any needed).
} finally {
if (writer != null) try { writer.close(); } catch (IOException logOrIgnore) {}
}
InputStream result = urlConnection.getInputStream();
// Now do your thing with the result.
Thanks
Shiva Kumar SS
I require to send an HL7 message, which is parsed into an XML message to the below end point.
Host: iol.sandbox.ohie.org
Port: 5001
username: admin
password: admin
HTTP method: POST
HTTP Path: /ws/rest/v1/patients/
I am using Apache HttpClient to achieve this. Below is the code which I am using.
public void simpleHttpMessage() throws Exception{
String url = "iol.sandbox.ohie.org";
String USER_AGENT = "/ws/rest/v1/patients/";
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
// add header
post.setHeader("User-Agent", USER_AGENT);
List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
urlParameters.add(new BasicNameValuePair("username", "admin"));
urlParameters.add(new BasicNameValuePair("password", "admin"));
post.setEntity(new UrlEncodedFormEntity(urlParameters));
HttpResponse response = client.execute(post);
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + post.getEntity());
System.out.println("Response Code : " +
response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
}
I am not sure where I should pass the actual xml message which I want to send. I am basically trying to send a HL7 message which is parsed into XML, over HTTP POST.
I would be grateful if someone who help me with what I am missing from the above code snippet.
So if you are trying to put the xml into the body
String xml = "<xml>xxxx</xml>";
HttpEntity entity = new ByteArrayEntity(xml.getBytes("UTF-8"));
post.setEntity(entity);
I believe that should work. But I really only deal with C#. Where is the xml?