I'm a bit new to Java and more to connections stuff with it. I'm trying to create a program to connect to a website ("www.buybackprofesional.com") where I would like to download pictures and get some text from cars (after the login you have to enter a plate number to access a car's file).
This is what I have right now, but it always says that the session has expired, I need a way to login using the username and password of the mainpage, am I right? can someone give me some advice? Thanks
Note: I want to do it in Java, maybe I was not clear in the question.
//URL web = new URL("http://www.buybackprofesional.com/DetallePeri.asp?mat=9073FCV&fec=27/07/2010&tipo=C&modelo=4582&Foto=0");
URL web = new URL("http://www.buybackprofesional.com/");
HttpURLConnection con = (HttpURLConnection) web.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; JVM)");
con.setRequestProperty("Pragma", "no-cache");
con.connect();
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
A colleage helped me with this so I'll post the code that works:
public static URLConnection login(String _url, String _username, String _password) throws IOException, MalformedURLException {
String data = URLEncoder.encode("Usuario", "UTF-8") + "=" + URLEncoder.encode(_username, "UTF-8");
data += "&" + URLEncoder.encode("Contrase", "UTF-8") + "=" + URLEncoder.encode(_password, "UTF-8");
// Send data
URL url = new URL(_url);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();
return conn;
}
This will submit the form info on the page I need and after that, using cookies I can stay connected!
To connect to a website using java consider using httpunit or httpcore (offered by apache). They handle sessions much better then you (or I) could do on your own.
Edit: Fixed the location of the link. Thanks for the correction!
Related
I have a problem on HttpURLConnection in post method. Everything is working fine on get method however, when I try to use Post method. I'm getting this error message.
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL
Here's my code snippet. I hope you could help me about this.
URL url = new URL(my url/userInfo);
String encoding = Base64.getEncoder().encodeToString(("username:password").getBytes("UTF-8"));
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "text/plain");
connection.setRequestProperty("Authorization", "Basic " + encoding);
connection.setRequestProperty("x-csrf-token", "fetch");
String csrfToken = connection.getHeaderField("x-csrf-token");
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String inputLine;
String output = in.readLine();
in.close();
String content = data // expected data to retrieve
URL url2 = new URL(my URL);//another url to push the data retrieve
HttpURLConnection connection2 = (HttpsURLConnection) url2.openConnection();
connection2.setDoInput(true);
connection2.setDoOutput(true);
connection2.setRequestMethod("POST");
connection2.setRequestProperty("Authorization", "Basic " + encoding);
connection2.setRequestProperty("Accept", "application/json");
connection2.setRequestProperty("x-CSRFToken", csrfToken);
connection2.setRequestProperty("cache-control", "no-cache");
OutputStream os = connection2.getOutputStream();
OutputStreamWriter osw = new OutputStreamWriter(os, "UTF-8");
osw.write(data);//this is where the data will be pushed
osw.flush();
osw.close();
os.close();
the idea is, we need first to get the x-csrf-token and data from the first link, which is okay. After GET Method execution, the POST method will occur. unfortunately, the post method is not working. I'm getting the error message shown above. By the way, we tried to do a post method in POSTMAN and it' working fine.
Hoping you could help me about this.
So I have a problem where if I type this link on the browser and hit enter, an activation happens. I just want to do the same through Java. I don't need any kind of response from the URL. It should just do the same as entering the URL on a browser. Currently my code doesn't throw an error, but I don't think its working because the activation is not happening. My code:
public static void enableMachine(String dns){
try {
String req= "http://"+dns+"/username?username=sputtasw";
URL url = new URL(req);
URLConnection connection = url.openConnection();
connection.connect();
/*BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream()));
String strTemp = "";
while (null != (strTemp = br.readLine())) {
System.out.println(strTemp);
}*/
} catch (Exception ex) {
ex.printStackTrace();
}
}
What's the problem?
If you want to do that with an URLConnection, it isn't sufficient to just open the connection with connect, you also have to send e.g. an HTTP request etc.
That said, i think it would be easier, if you use an HTTP client like the one from Apache HttpComponents (http://hc.apache.org/). Just do a GET request with the HTTP client, this would be the same as visiting the page with a browser (those clients usually also supports redirection etc.).
You may use HttpUrlConnectionClass to do the job:
URL url = new URL("http://my.url.com");
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
httpCon.setRequestProperty("Content-Type", "application/json");
httpCon.setDoOutput(true);
httpCon.setRequestMethod("POST");
String params = "foo=42&bar=buzz";
DataOutputStream wr = new DataOutputStream(httpCon.getOutputStream());
wr.writeBytes(params);
wr.flush();
wr.close();
httpCon.connect();
int responseCode = httpCon.getResponseCode();
You may as well use "GET" request method and just append parameters to the url.
So there is this line of code
String data = URLEncoder.encode("image", "UTF-8") + "=" + URLEncoder.encode(Base64.encodeBase64String(baos.toByteArray()).toString(), "UTF-8");
data += "&" + URLEncoder.encode("key", "UTF-8") + "=" + URLEncoder.encode(YOUR API KEY GOES HERE, "UTF-8");
and when I registered for the Imgur API I was given a client_id and a client_secret and was wondering which one I use for where it says "YOUR API KEY GOES HERE" also in the first part in the second line where it says "key" what do I enter there? Also is the site to upload it http://imgur.com/api/upload because I have seen a few different ones.
try this out:
public static String getImgurContent(String clientID) throws Exception {
URL url;
url = new URL("https://api.imgur.com/3/image");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
String data = URLEncoder.encode("image", "UTF-8") + "="
+ URLEncoder.encode(IMAGE_URL, "UTF-8");
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Authorization", "Client-ID " + clientID);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
conn.connect();
StringBuilder stb = new StringBuilder();
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
// Get the response
BufferedReader rd = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
stb.append(line).append("\n");
}
wr.close();
rd.close();
return stb.toString();
}
was almost like humpty dumpty, getting every piece back together, codes from everywhere, at least it worked as expected, its a shame they don't have examples...
enjoy.
ps: ou can also make with FILES (haven't tried yet) but you need to convert an image to base64 and then to utf8 (to replace the url)
edit, use this instead of the URL, so you can upload files:
//create base64 image
BufferedImage image = null;
File file = new File(imageDir);
//read image
image = ImageIO.read(file);
ByteArrayOutputStream byteArray = new ByteArrayOutputStream();
ImageIO.write(image, "png", byteArray);
byte[] byteImage = byteArray.toByteArray();
String dataImage = Base64.encode(byteImage);
String data = URLEncoder.encode("image", "UTF-8") + "="
+ URLEncoder.encode(dataImage, "UTF-8");
The site to upload to is - https://api.imgur.com/3/image or you can alternatively use the same link with "upload" instead of image.
I am currently trying to use the Imgur API myself and although I have not got it completely right yet (I can't seem to parse the URL response) I have looked at quite a few code examples for it. Are you definitely using version 3 of the API?
Because the homepage of the API says that you should give your client ID in this format "Authorization Client-ID YOUR_CLIENT_ID", not using "key" like you are.
Have a look at http://api.imgur.com/
Edit: you might find the following useful - Anonymous Uploading File object to Imgur API (JSON) gives Authentication Error 401
I am trying to login to a website and get page source of a page site after I login to the web site with java URLConnection. The problem I am facing is I can't maintain session so server gives me this warning and doesn't let me to get connected:
This system requires the use of HTTP cookies to verify authorization information.
Our system has detected that your browser has disabled HTTP cookies, or does not support them.
Please refer to the Help page in your browser for more information on how to correctly configure your browser for use with this system.
At first I am trying to send empty cookie to let server to understand I am handling sessions but it doesn't give me session id either.
This is my source code:
try {
// Construct data
String data = URLEncoder.encode("usr", "UTF-8") + "=" + URLEncoder.encode("usr", "UTF-8");
data += "&" + URLEncoder.encode("password", "UTF-8") + "=" + URLEncoder.encode("pass", "UTF-8");
// Send data
URL url = new URL("https://loginsite.com");
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
conn.setRequestProperty("Cookie", "SESSID=");
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
wr.close();
rd.close();
String headerName=null;
for (int i=1; (headerName = conn.getHeaderFieldKey(i))!=null; i++) {
if (headerName.equals("Set-Cookie")) {
String cookie = conn.getHeaderField(i);
System.out.println(cookie.split(";", 2)[0]);
}
}
} catch (Exception e) {
}
You should use an HTTP library which handles session management and other details of the HTTP protocol for you, e.g. supports Cookies and things like Keep-Alive, Proxies etc. out of the box. Try Apache HttpComponents
I am new to facebook app development and i have been trying to post a simple message on the wall of the user.i have managed to get the access token .Here is the code for the POST request.I am using java servlets
String data = URLEncoder.encode("access_token", "UTF-8") + "=" + URLEncoder.encode(accessToken, "UTF-8");
data += "&" + URLEncoder.encode("message", "UTF-8") + "=" + URLEncoder.encode("finally", "UTF-8");
out.println("data is\n"+data);
// Send data
String u="https://graph.facebook.com/me/feed";
URL urls = new URL(u);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
Well this code is not working and i can not post on the wall.Any suggestion as to where i might be wrong?
I'm pretty sure that it's because you don't specify the application/x-www-form-urlencoded content type, try this:
URLConnection connection = new URL("https://graph.facebook.com/me/feed").openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
out.writeBytes(data);
out.flush();
out.close()
Edit
Ok, so there are two more things that might cause this problem:
You also need to specify the content length.
You might need to read the response to make it count..
This code was tested and it works:
StringBuffer buffer = new StringBuffer();
buffer.append("access_token").append('=').append(ACCESS_TOKEN);
buffer.append('&').append("message=").append('=').append("YO!");
String content = buffer.toString();
URLConnection connection = new URL("https://graph.facebook.com/me/feed").openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Content-Length", Integer.toString(content.length()));
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
out.writeBytes(content);
out.flush();
out.close();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
}
in.close();